Databricks on Google Cloud, Part 8: Better together, BigQuery, Gemini and Google Cloud services

Acme’s question has always needed two halves of data. Marketing keeps campaign data in BigQuery. Supply keeps delivery data in Databricks. “Did the Acme Pure campaign work?” means putting campaign_performance next to supply_service, week by week, and for most of the last decade that meant copying one of them into the other platform every night.

That is no longer the only option. As of October 2026, BigQuery can read a Unity Catalog table in place, Databricks can read Iceberg tables that BigQuery manages, and Databricks can push queries down to BigQuery. Gemini models are served inside Databricks, Databricks streams directly from Pub/Sub, Looker connects to Databricks SQL, and OpenSharing hands live tables to partners. Each of these has a different status, a different identity model and a different bill.

This post walks through every one of those links for Acme, from the basics to the commands. It builds on Part 2, which covered the identities each link uses, and it ends with a reference architecture where Databricks builds the bronze and silver layers and BigQuery serves the gold table to marketing without a copy.

If you prefer to watch first, here is the heart of this part as a short animated explainer.

Video: how Databricks and BigQuery read each other's tables without a copy, in three minutes. The narration is a synthetic voice; the script and animation are mine.

What are the key ideas?

Seven terms carry this post.

Catalog. A service that stores table names, schemas and where each table’s files live. Unity Catalog is Databricks’ catalog; the Lakehouse runtime catalog (formerly BigLake metastore) is Google’s catalog for Apache Iceberg tables.
Catalog federation. One platform’s catalog mirrors the table list and metadata of another platform’s catalog, and its own engine then reads the files directly. One copy of the data, and the reading side pays for the compute.
Query federation. Your platform sends a query to the other platform, which runs it on its own storage and streams rows back. It works on any table, but both sides do work and rows cross the network.
Iceberg REST catalog. The standard HTTP interface, defined in the Iceberg REST catalog specification, that an engine calls to find an Iceberg table’s current metadata file. It is called REST because it is a set of plain HTTP requests on resources such as /v1/namespaces/{namespace}/tables/{table}.
Credential vending. When a catalog answers a request for a table, it also hands out a short-lived storage token scoped to that table’s files, so the reading engine needs no standing permission on the bucket.
Iceberg reads (UniForm). A Delta table setting that makes Databricks write Iceberg metadata next to the Delta log, so Iceberg readers such as BigQuery can read the same Parquet files.
OpenSharing. The name Databricks now uses for Delta Sharing: an open protocol for sharing live tables with other organisations without copying them.

The first three ideas are shapes that keep coming back, so here they are side by side.

three ways to use another platform's data without copying it: catalog federation, where your engine borrows the other catalog's metadata and reads the files itself; query federation, where the other platform runs your SQL and streams rows back; and sharing, where a provider hands a recipient short-lived links to its files

The table at the centre of this post is supply’s gold table. Here is a quick look:

a quick look at supply_service, the gold table this post shares with BigQuery: acme_supply.supply_gold.supply_service has one row per brand, region and week, owned by supply, in Databricks; three real rows

The series overview shows it next to marketing’s campaign_performance, and the agents series’ business and data prerequisite explains every column and how the two tables join.

Which Google Cloud services does Databricks work with?

A workspace runs without these, but they are often the reason to choose Google Cloud.

BigQuery is Google’s data warehouse, where Acme’s marketing data lives. Databricks and BigQuery can now read each other’s tables in both directions without copying them; the next section shows the four ways.
Gemini models and Gemini Enterprise Agent Platform, the platform formerly called Vertex AI (Google lists the name changes from Vertex AI). Databricks also serves Gemini models in its Foundation Model APIs, so a notebook or agent can call Gemini without leaving Databricks.
Pub/Sub is Google’s messaging service. Databricks has a built-in Structured Streaming connector for it.
Looker is Google’s business intelligence platform. It connects to a Databricks SQL warehouse with its Databricks dialect.

the Google Cloud services Databricks works with: BigQuery through Lakehouse Federation one way and Google Cloud Lakehouse federation the other, Gemini models and Gemini Enterprise Agent Platform, Pub/Sub as a streaming source, and Looker as a BI tool on Databricks SQL

For Acme, the BigQuery link matters most, because its agents need both halves of the data, so it gets the most space below.

This has changed a lot since 2025, starting with the names. On 20 April 2026 Google renamed BigLake to Lakehouse, and BigLake metastore to the Lakehouse runtime catalog, Google’s catalog service for Apache Iceberg tables, which speaks the Iceberg REST protocol (the standard HTTP interface an engine calls to find an Iceberg table’s current metadata file). Google’s banner adds that “Lakehouse APIs, client libraries, CLI commands, and IAM names remain unchanged and still reference BigLake”, which is why you still grant roles such as roles/biglake.viewer and run gcloud biglake commands. Dataplex Universal Catalog is now Knowledge Catalog (since 10 April 2026).

The two ideas from the key ideas section, catalog federation and query federation, make the options easier to tell apart. The bottom line: there are four paths, but only two shapes: borrow the other side’s catalog and read the files yourself, or send the other side a query and take the rows back. Anything else is a copy.

A simple example. You need a book that another library holds. You can look it up in their catalogue, which tells you which shelf it is on, and go and read it yourself. You can phone their librarian, who reads the passage and tells you the answer. Or you can photocopy it and keep your own copy, which goes out of date the day they publish a new edition. Looking it up and reading it yourself is catalog federation, phoning the librarian is query federation, and the photocopy is a copy.

How does this apply to Acme? The two libraries are Unity Catalog and BigQuery, and each can visit the other. With those, there are four paths.

BigQuery reads Unity Catalog. Google calls this cross-cloud data access, part of “borderless Lakehouse”. You set up a connection to Databricks Unity Catalog, and the Lakehouse runtime catalog creates a federated catalog that mirrors Unity Catalog’s Iceberg REST endpoint. BigQuery then scans the Parquet files of Delta tables that have Iceberg reads (UniForm) turned on, in place. Google’s page says federated catalogs “are read-only views of the remote catalog”. It has been generally available since 2 November 2026 (it was in Preview before that). This is the path that lets Acme’s marketing analysts join supply’s supply_service table in BigQuery without a copy.
Databricks reads Google Lakehouse tables. Databricks calls this Google Cloud Lakehouse catalog federation: a GOOGLE_CLOUD_LAKEHOUSE connection in Unity Catalog, so Databricks compute reads Apache Iceberg tables that BigQuery manages, in place. It was labelled Beta in October 2026, with a Previews page switch; as of 8 November 2026 the page no longer shows either, but there is no general availability announcement yet (more on this below).
Databricks queries any BigQuery table. Lakehouse Federation for BigQuery creates a foreign catalog in Unity Catalog that mirrors BigQuery. Queries are pushed down to BigQuery, which runs them, and the rows come back. It is generally available and works on native BigQuery tables too, so it suits occasional reads; heavy, repeated scans of Iceberg tables are cheaper through path 2.
BigQuery reads a Delta folder directly. The older option, BigLake external tables for Delta Lake, points BigQuery at a Delta table’s folder in Cloud Storage and still works. It bypasses Unity Catalog, so Unity Catalog’s permissions and lineage do not apply to those reads; for tables that Databricks governs, path 1 is now the better fit.

four ways to connect Databricks and BigQuery as of November 2026: BigQuery reading Unity Catalog through Lakehouse cross-cloud data access (GA), Databricks reading Google Lakehouse Iceberg tables through catalog federation (Beta), Databricks querying any BigQuery table through Lakehouse Federation (GA), and BigQuery reading a Delta folder as a BigLake external table for Delta Lake (GA); every path is read-only across the boundary

A beginner question is worth answering here: does any path let BigQuery write to a Databricks table, or Databricks write to a BigQuery table? Not across a federated catalog. Each engine writes only to tables its own catalog owns. Writing into BigQuery from Databricks is a copy, done with the Spark BigQuery connector, which I cover after path 4.

How does BigQuery read a Unity Catalog table?

The bottom line: BigQuery keeps a read-only mirror of Unity Catalog’s table list, and for each query Unity Catalog hands BigQuery a short-lived pass to read the files in place. No data is copied, and Unity Catalog still decides what can be read.

A simple example first. A branch library keeps a copy of the main library’s index cards, refreshed every ten minutes, so readers at the branch can see what exists. When a reader asks for a book, the main library checks the request against its own rules (by default the branch’s library card; optionally the reader’s own) and issues a one-hour pass to its own stacks; the reader goes and reads it there. The branch never holds the book. The copied index cards are the federated catalog, the ten minutes is the refresh interval, and the one-hour pass is credential vending: a temporary storage token scoped to one table.

How does this apply to Acme? Path 1 is the one Acme cares about most, so it is worth following one query end to end. Marketing’s analyst runs a query in project acme-analytics. The table lives in Unity Catalog in workspace acme-prod-euw2-supply, and its files live in gs://acme-lake-supply.

how BigQuery reads supply_service through Lakehouse cross-cloud data access: in the background the federated catalog signs in to Unity Catalog as Acme's service principal and syncs the table list; at query time BigQuery resolves the table through the federated catalog, Unity Catalog's Iceberg REST endpoint returns the current metadata file and a temporary Cloud Storage token, and BigQuery reads the Parquet files in place

Nothing in that picture copies data. The federated catalog holds a mirror of names and metadata pointers; BigQuery’s own workers read the Parquet files. The token Unity Catalog hands out is the same mechanism any Iceberg client gets: Databricks’ Iceberg REST catalog page shows the response to a loadTable call, and if it were printed it would look like this, with a gs:// metadata location and a temporary Cloud Storage token that expires after one hour by default:

{
  "metadata-location": "gs://acme-lake-supply/<table path>/metadata/00042-<uuid>.metadata.json",
  "metadata": { "format-version": 2, "current-snapshot-id": 4120, "...": "..." },
  "config": {
    "expires-at-ms": "<epoch ms>",
    "gcs.oauth2.token": "<temporary-oauth-token>",
    "gcs.oauth2.token-expires-at": "<epoch ms>"
  }
}

What must be true on the Databricks side?

Google’s setup page and Databricks’ external access pages list four conditions. All four must hold, and the first two catch most people.

The table must be in an external location, not default storage. Google’s page says only Unity Catalogs “that use an external location” are supported, and that catalogs using Databricks default storage (storage that Databricks manages for you, which Part 5 explains) are not. Acme’s acme_supply catalog has its managed storage in gs://acme-lake-supply, an external location, so it qualifies.
External data access must be on for the metastore. It is off by default “to prevent unauthorized external access”, per Enable external data access to Unity Catalog. A metastore admin turns it on in Catalog Explorer, under the metastore’s Details tab.
A service principal must hold the right grants. Acme creates a Databricks service principal for BigQuery, sp-bq-federation, and grants it USE CATALOG, USE SCHEMA and SELECT, plus EXTERNAL USE SCHEMA. That last privilege is special: only the catalog owner can grant it, and “ALL PRIVILEGES does not include the EXTERNAL USE SCHEMA privilege”, to avoid accidental exfiltration (Part 6 covers the privilege model).
The table must have Iceberg reads turned on. The next section shows how.

-- Run by the owner of acme_supply. The application ID is the service principal's UUID.
GRANT USE CATALOG ON CATALOG acme_supply TO `<sp-bq-federation application id>`;
GRANT USE SCHEMA, SELECT ON SCHEMA acme_supply.supply_gold TO `<sp-bq-federation application id>`;
GRANT EXTERNAL USE SCHEMA ON SCHEMA acme_supply.supply_gold TO `<sp-bq-federation application id>`;

Granting on supply_gold only, rather than the whole catalog, keeps bronze and silver invisible to BigQuery. That is the main governance lever on this path: in the default service identity mode, BigQuery sees exactly what the service principal may see, so row filters and column masks that apply to the principal apply to every BigQuery reader. The optional user identity federation mode, described below, runs each query as the signed-in analyst instead.

What must be true on the Google Cloud side?

In acme-analytics, Acme enables the BigLake and Secret Manager APIs, and the person creating the catalog needs BigLake Admin (roles/biglake.admin). Google offers two ways to authenticate to Databricks. OIDC (OpenID Connect) is the one Google recommends: no secret is stored, because the federated catalog’s own Google service account signs in through Databricks OAuth token federation, which Part 2 introduced. The alternative stores the service principal’s OAuth client ID and secret in a regional Secret Manager secret.

This is the documented command, with Acme’s names. I have not run it myself. Since general availability it is gcloud biglake rather than gcloud alpha biglake:

gcloud biglake iceberg catalogs create acme_supply_uc \
  --project="acme-analytics" \
  --primary-location="europe-west2" \
  --catalog-type="federated" \
  --federated-catalog-type="unity" \
  --unity-service-principal-application-id="<sp-bq-federation application id>" \
  --unity-instance-name="<workspace host of acme-prod-euw2-supply>" \
  --unity-catalog-name="acme_supply" \
  --refresh-interval="600s" \
  --namespace-filters="supply_gold"

Four flags deserve a sentence each. For a Unity Catalog on Google Cloud, Google’s page says to pick the exact same region in --primary-location, so europe-west2 for Acme, and the regional secret or Service Directory namespace, if you use one, must be in that region too. --refresh-interval is how often metadata is synchronised: it must be 0s (off) or at least 300 seconds, and “changes made in the remote catalog … aren’t visible in Lakehouse until the next background metadata refresh completes”. --namespace-filters limits which schemas are mirrored. And there is no write flag at all, because the catalog is read-only.

After creation, Acme reads the catalog’s service account ID with gcloud biglake iceberg catalogs describe acme_supply_uc --format='value(biglake-service-account-id)' and adds a federation policy to sp-bq-federation in the Databricks account: issuer https://accounts.google.com, subject that numeric ID, and audience https://accounts.cloud.databricks.com, exactly as Google’s page lists. Then the analyst queries a four-part name, which Google’s query remote data page describes as project.federated_catalog.namespace.table:

SELECT s.week_start, s.otif_pct, c.media_spend_eur
FROM `acme-analytics.acme_supply_uc.supply_gold.supply_service` AS s
JOIN `acme-analytics.campaigns.campaign_performance` AS c
  USING (brand, region, week_start)
WHERE s.brand = 'Acme Pure' AND s.region = 'Western Europe';

The analyst needs BigQuery Data Viewer, BigQuery Job User and BigLake Viewer (roles/biglake.viewer), the three roles the query page lists. BigQuery bills the scan to acme-analytics like any other query.

Two more details from Google’s limitations list matter for Acme. Cross-cloud caching (Lakehouse keeping a copy of the remote file blocks it has read in the Google Cloud region where the query runs, so a repeated query does not fetch them again) became generally available on 2 November 2026 and is off by default: you opt in per federated catalog or connection with --cross-cloud-cache="enabled". The cache does not support customer-managed encryption keys; if the constraints/gcp.restrictNonCmekServices organisation policy applies to a table in the query, caching is switched off for that query, which still runs. And the cache stores data at rest in the query’s region, which is a residency question when the remote data is in another jurisdiction. For a Unity Catalog on Google Cloud in the same region, as at Acme, that second point does not change anything, and because Acme’s files are already in europe-west2 there is little egress for the cache to save.

The second detail is new with general availability: user identity federation. The bottom line: in the default mode Unity Catalog sees one service principal for every BigQuery reader; in user identity federation mode it sees, checks and logs each analyst. Think of a shared company account at a car hire desk versus each driver showing their own licence: with the shared account the desk only knows the company rented the car, with licences it knows who drove. By default the federated catalog runs in service identity mode, so both the background metadata refresh and every query reach Unity Catalog as sp-bq-federation; Unity Catalog’s audit log shows that service principal, never the analyst. With --federated-catalog-identity-mode="user-identity-federation", the refresh still uses the catalog’s service account, but each query carries the end user’s identity, so “the remote catalog [can] attribute queries from Google Cloud to end users, allowing permission enforcement and audit log generation”. The setup page lists what it needs: the identity provider puts the user’s email in the token’s sub claim, the same user exists in the Databricks account (SCIM sync is the recommended way), and an account-wide federation policy in Databricks trusts the catalog’s issuer (https://www.gstatic.com/biglake/federation/organizations/<org id>) and the audience databricks-<metastore id>. Google’s note is explicit that this mode is “currently supported only for Databricks Unity Catalog”.

A worked example. A marketing analyst signed in as analyst@acme.example runs the supply_service join above. In service identity mode Unity Catalog checks sp-bq-federation’s grants and the system.access.audit row names the service principal. In user identity federation mode Unity Catalog evaluates the analyst’s own permissions, so if she has no access to supply_gold.supply_service the query is refused for her even though the service principal could read the table, and the audit row names her (Google warns that auditable events “can take 15 minutes or longer to appear”). Google’s page does not list which Unity Catalog privileges the end user needs in this mode, so test it with one analyst before rolling it out. My view: for a gold table read by many analysts, user identity federation is worth the extra setup, because it keeps one set of grants and one audit trail in Unity Catalog.

How do you turn on Iceberg reads for a Delta table?

The bottom line: Iceberg reads add a second set of metadata over the same Parquet files, so Delta and Iceberg readers share one copy; the price is a few table features Iceberg v2 readers cannot understand.

A simple example. A reference book has one set of pages and two indexes at the back, one in English and one in French. Both indexes point to the same pages, so nobody prints the book twice. For that to work, each paragraph needs a fixed number, because the French index cannot rely on English headings. Now suppose an editor marks deleted lines with sticky notes rather than reprinting the page: an English reader honours the notes, but a French reader whose instructions never mention sticky notes reads the deleted lines as if they were still there. The pages are the Parquet files, the two indexes are the Delta log and the Iceberg metadata, the paragraph numbers are column mapping field IDs, and the sticky notes are deletion vectors, which an Iceberg v2 reader cannot apply.

How does this apply to Acme? The book is supply_service, and the French reader is BigQuery. Path 1 only works on tables that Iceberg readers can understand, so this is the configuration that decides whether marketing can see supply_service at all. Prerequisite 1 explains the formats: Delta Lake and Apache Iceberg both store data as Parquet, and UniForm (Universal Format) makes Databricks write Iceberg metadata next to the Delta log, so one copy of the files serves both kinds of reader. It also draws the Iceberg metadata tree that UniForm generates. Databricks now calls this setting Iceberg reads, supports versions 1, 2 and 3 of the Iceberg specification, and generates the metadata asynchronously on the same compute that wrote the data. This section covers what Prerequisite 1 leaves out: the exact properties, the requirements and the limits that decide whether BigQuery can read supply_service.

Turning it on is three table properties. This is the example from the Databricks documentation for a new table, which I have not run myself:

CREATE TABLE T (c1 INT) TBLPROPERTIES (
  'delta.columnMapping.mode' = 'id',
  'delta.enableIcebergCompatV2' = 'true',
  'delta.universalFormat.enabledFormats' = 'iceberg'
);

the three table properties that turn on Iceberg reads (UniForm): column mapping gives columns stable IDs that Iceberg uses, IcebergCompatV2 keeps every write readable by Iceberg v2 readers, and universalFormat.enabledFormats turns on Iceberg metadata generation

Column mapping deserves one more sentence. Without it, Delta finds columns in Parquet files by name. With column mapping on, each column gets a fixed physical name and a numeric field ID, and the Delta protocol requires writers to store that ID in the field_id of each Parquet file’s schema. Iceberg identifies columns by field ID, so this is what lets an Iceberg reader line the columns up. The documentation recommends id mode for new tables; its example for an existing table uses name mode, and IcebergCompatV2 accepts either.

supply_service already exists, so Acme uses the existing-table form, which needs Databricks Runtime 15.4 LTS or above:

ALTER TABLE acme_supply.supply_gold.supply_service SET TBLPROPERTIES (
  'delta.columnMapping.mode' = 'name',
  'delta.enableIcebergCompatV2' = 'true',
  'delta.universalFormat.enabledFormats' = 'iceberg'
);
DESCRIBE EXTENDED acme_supply.supply_gold.supply_service;  -- look for "Delta Uniform Iceberg"

The requirements page adds conditions that are easy to miss. The table must be registered in Unity Catalog (managed or external). The Delta protocol must be at minReaderVersion 2 and minWriterVersion 7 or above. Every write must use Databricks Runtime 14.3 LTS or above. Turning it on “adds the IcebergCompatV2 write protocol feature”, so older Delta writers outside Databricks may no longer be able to write to the table, and the table must be written by name, not by path, for metadata generation to trigger.

Two limits are worth knowing early, and both follow from how Iceberg readers work.

The table must be registered in Unity Catalog. An Iceberg reader finds a table through a catalog that points at its current metadata file. Unity Catalog provides “an implementation of the Iceberg REST catalog API specification”, so engines such as Apache Spark, Apache Flink and Trino can look the table up by name. Some clients, such as BigQuery, instead register the table with the path of one metadata JSON file, which UniForm writes under the table’s metadata/ folder. Path 1 uses the catalog route; path-based registration is the older route, and the documentation warns that path-based clients may need their metadata path updated by hand to see new versions.
No deletion vectors. Prerequisite 1 explains deletion vectors (merge-on-read marks instead of rewritten files) and why an Iceberg v2 reader cannot apply them: it would return rows that had been deleted. The documentation is direct: “You cannot enable deletion vectors on a table with Iceberg reads enabled.” Turning Iceberg reads on does not remove existing ones for you: on a table that has them, you run REORG TABLE ... APPLY (UPGRADE UNIFORM (ICEBERG_COMPAT_VERSION = 2)), which turns them off, purges them and rewrites the files. From then on, every DELETE and UPDATE rewrites whole files.

There is a newer option. Iceberg v3 supports deletion vectors, and Databricks documents delta.enableIcebergCompatV3 for managed tables on Databricks Runtime 18 LTS and above, so a Delta table can keep deletion vectors and still be read as Iceberg v3. Your Iceberg readers then need to support version 3 of the specification. One honest gap: I did not find a public statement of which Iceberg versions BigQuery’s federated catalogs read, so test v3 with BigQuery before you move supply_service to it.

Three operational details close the loop. Iceberg reads switch the Parquet compression codec from Snappy to Zstandard. Metadata generation is asynchronous, so the metadata fields converted_delta_version and converted_delta_timestamp tell you which Delta version Iceberg readers can see; if a cluster died before generation finished, MSCK REPAIR TABLE ... SYNC METADATA regenerates it. And old Iceberg metadata files accumulate in metadata/ until VACUUM removes them (Databricks Runtime 17.2 and above), or predictive optimisation does it for you.

How does Databricks read Iceberg tables that BigQuery manages?

Path 2 is the mirror image: the same library visit, in the other direction. The bottom line: Databricks borrows Google’s Iceberg catalog and reads BigQuery-managed Iceberg files itself, paying only for its own compute. Marketing could keep some tables as Apache Iceberg tables managed by BigQuery, with files in Cloud Storage, and supply’s data scientists want to read them from Databricks without a copy. The Google Cloud Lakehouse catalog federation page describes the setup. It was labelled Beta in October 2026, with a step to turn it on from the Previews page. One honest gap: as of 8 November 2026 the page shows neither the Beta label nor that step, but Databricks has not announced general availability in its release notes, so check the page before you rely on its stage.

The requirements are specific: a workspace on Google Cloud with Unity Catalog, Databricks Runtime 18 LTS or above, and pro or serverless SQL warehouses. Dedicated clusters (formerly single-user clusters) are not supported. If you enforce VPC Service Controls or firewall rules in Google Cloud, you must allow Databricks’ egress IP ranges for both the control plane and the compute plane, which Part 4 covers for serverless.

There are four steps.

Google side. A service account with roles/biglake.viewer on acme-analytics, and a JSON key for it. Part 2 explains why this path still needs a key and how to keep it in a secret.
A connection of type GOOGLE_CLOUD_LAKEHOUSE, separate from the BigQuery connection type.
Storage credentials and external locations. One read-write location for the catalog’s metadata storage root, which can be any bucket, and read-only locations over the table buckets, whose service accounts need roles/storage.objectViewer, plus roles/bigquery.dataViewer when the tables are BigQuery-managed.
A foreign catalog that names the warehouse, the authorised paths and the storage root.

CREATE CONNECTION acme_lakehouse TYPE GOOGLE_CLOUD_LAKEHOUSE
OPTIONS (
  gcp_project_id 'acme-analytics',
  service_account_json secret('google', 'lakehouse_viewer_key')
);

CREATE FOREIGN CATALOG marketing_iceberg USING CONNECTION acme_lakehouse
OPTIONS (
  warehouse 'bq://projects/acme-analytics',
  authorized_paths 'gs://acme-marketing-iceberg',
  storage_root 'gs://acme-lake-supply/lakehouse-federation-metadata'
);

The warehouse value takes three forms: bl://projects/<project>/catalogs/<catalog> for a Lakehouse runtime catalog, a gs:// path for a Cloud Storage bucket catalog, and bq://projects/<project> for the built-in catalog of BigQuery-managed Iceberg tables. Namespaces become Unity Catalog schemas and tables become foreign tables, so a BigQuery Iceberg table orders in namespace analytics appears as marketing_iceberg.analytics.orders. The bucket names above are Acme’s; I have not run these statements.

Foreign tables are read-only, tables with a TIME column are not supported, and Unity Catalog grants (USE CATALOG, USE SCHEMA, SELECT) decide who reads them, including tag-based and fine-grained access control. Because the query runs entirely on Databricks compute against files in place, Databricks says it “can provide better performance and lower cost than query federation”.

How does Lakehouse Federation run a query on BigQuery?

Path 3 is the general-purpose one. It works on any BigQuery table, native or not, and it is generally available. Part 2 showed the connection, the key and the grants, so here is just the snapshot:

CREATE CONNECTION bq_acme TYPE bigquery
OPTIONS (GoogleServiceAccountKeyJson secret('bq', 'acme_federation_key'));
CREATE FOREIGN CATALOG marketing USING CONNECTION bq_acme
OPTIONS (dataProjectId 'acme-analytics');

What Part 2 did not cover is how a query actually runs, which decides both speed and cost. The bottom line: in a direct read BigQuery only sends the columns and rows you asked for and Databricks does the rest; with materialisation BigQuery does more of the work and bills for it.

A simple example. You order stock from a wholesaler’s warehouse. Option one: they send you only the boxes for the products and dates you listed, and you count and sort them in your own shop. Option two: their staff count and summarise everything first and send you one sheet of totals, and the invoice includes their labour. Sending the right boxes is a direct read with projections and filters pushed down; the summary sheet is materialisation, and the labour is the extra BigQuery compute charge.

How does this apply to Acme? Marketing’s audience_reach table is the warehouse, and supply’s notebook is the shop.

Unlike most federation connectors, which use JDBC (a standard database driver interface), the BigQuery connector uses the BigQuery Storage API, Google’s high-throughput interface for reading table data directly from BigQuery’s storage. There are two modes.

two ways a federated BigQuery query runs: a direct read opens a BigQuery Storage API read session and streams filtered columns to Databricks compute; a materialised read runs the SQL as a BigQuery job, writes the small result to a temporary table in a materialisation dataset, then streams that back

Direct read, the default. Databricks opens a Storage API read session and pushes down only projections (the columns you select) and simple filters, which become Storage API row restrictions. Everything else, including joins and aggregates, runs on Databricks after the rows arrive. The documentation says direct reads “offer better performance for large scans”.
Materialisation. BigQuery runs more of the query as SQL, writes the result to a temporary table in a materialisation dataset, and Databricks reads that back. It adds limits, aggregates, sorting with a limit, joins and some functions, and it “incurs additional BigQuery compute charges”. Views and external tables, including BigLake tables, are always materialised.

what Lakehouse Federation pushes down to BigQuery: direct reads through the Storage API push projections and simple filters only; materialisation compiles more of the query to BigQuery SQL, adding limits, sorting with a limit, aggregates, joins and some functions in filters; window functions are never pushed, and views and external tables are always materialised

You choose per query or per catalog. Per query, add an option after the table name; per catalog, set forceMaterialization, which clusters support from Databricks Runtime 16.4 LTS. If the service account cannot create datasets, or you want the temporary tables in a dataset you control, name one with materializationDataset:

I have not run these two statements; the option syntax is the documented one.

-- One query: a small aggregate from a large table
SELECT region, SUM(reach) FROM marketing.campaigns.audience_reach
WITH ('materializationEnabled' 'true') GROUP BY region;

-- Every query on the catalog, into a dataset Acme created in europe-west2
ALTER CATALOG marketing OPTIONS (forceMaterialization 'true', materializationDataset 'dbx_tmp');

The documentation suggests materialisation when you need advanced pushdowns, read small results from large tables, or read data in another region. As a litmus test:

1. A large scan, simple filters, columns you need?          -> direct read (the default)
2. A small aggregate or top-N from a very large table?      -> materialisationEnabled on that query
3. Views, external or BigLake tables, or another region?    -> materialisation (views always are)

My view: for Acme’s occasional lookups of audience_reach, the default is fine; for anything run every hour, the BigQuery job cost and the Databricks cost both belong in the estimate.

Four operational facts from the same page are worth pinning on the wall:

The compute needs four endpoints on port 443: bigquery.googleapis.com for metadata, bigquerystorage.googleapis.com for data, and oauth2.googleapis.com and accounts.google.com for authentication. With a restricted virtual IP for Google APIs, accounts.google.com must still resolve. Part 3 covers that routing.
The Test connection button does not prove the data path. It checks the REST API and authentication, but “doesn’t read data through the Storage API”. The documented check is a short shell loop from a notebook: for host in bigquery.googleapis.com bigquerystorage.googleapis.com oauth2.googleapis.com accounts.google.com; do nc -zv "$host" 443; done. Serverless SQL warehouses use a separate egress path, so a passing cluster test proves nothing about them.
Some types are awkward. INTERVAL columns are not supported and make the whole foreign table fail to load, and BIGNUMERIC maps to DecimalType(38, 38) unless you set bigNumericDefaultScale.
BigQuery bills the connection’s project. The optional billing project on the connection is where queries, and the auto-created materialisation dataset, land.

What about the Spark BigQuery connector?

The bottom line: the connector is the photocopier from the library example: the right tool when you genuinely need a native BigQuery copy, and the wrong one for everyday reads.

Before Lakehouse Federation existed, notebooks read and wrote BigQuery with the open-source Spark BigQuery connector, using spark.read.format("bigquery"). It still works on classic compute, and it is still the documented way to write a DataFrame into a native BigQuery table. Be aware of how Databricks labels it now: its BigQuery page is marked Experimental, says this legacy documentation has been retired, and recommends Lakehouse Federation where it supports your source.

Three facts decide whether Acme uses it. It authenticates as the cluster service account, which is keyless but outside Unity Catalog, so everyone on the cluster shares its Google permissions; Part 2 drew that trade-off. Serverless compute has no cluster service account, so this is a classic compute tool. And reading needs BigQuery Read Session User and BigQuery Data Viewer, while writing needs BigQuery Job User and BigQuery Data Editor. A write either stages files in a temporary bucket and runs a load job (temporaryGcsBucket) or, per the connector’s README, streams through the Storage Write API with writeMethod set to direct:

(spark.table("acme_supply.supply_gold.supply_service")
   .write.format("bigquery")
   .option("writeMethod", "direct")
   .option("parentProject", "acme-analytics")
   .mode("overwrite")
   .save("acme-analytics.supply_copy.supply_service"))

That code makes a second copy, with its own freshness and its own permissions in BigQuery. I would use it only when marketing needs a native BigQuery table for a BigQuery-only feature, and I would schedule it as a Lakeflow job, never by hand.

When does a BigLake external table for Delta Lake still make sense?

The bottom line: a BigLake external table for Delta is a side door into the files that skips Unity Catalog’s checks, so it suits folders nothing else governs and not governed tables.

A simple example. An archive has a front desk that checks every visitor’s permissions and logs every visit. A side door with its own key opens straight into the stacks. Anyone holding that key reads whatever they like, and the front desk never knows. The front desk is Unity Catalog, the side door is the BigLake external table, and its key is the Cloud resource connection’s service account.

How does this apply to Acme? Path 4 points BigQuery straight at a Delta folder. Google’s Delta Lake page shows the statement, which needs a Cloud resource connection whose service account can read the bucket:

CREATE EXTERNAL TABLE `acme-analytics.supply_ext.supply_service`
WITH CONNECTION `acme-analytics.europe-west2.acme-lake-conn`
OPTIONS (format = "DELTA_LAKE", uris = ['gs://acme-lake-supply/<table path>']);

BigQuery reads the _delta_log itself to find the current files, so no Iceberg metadata is needed, and schema changes are picked up with bq update --autodetect_schema. The cost is governance: the connection’s service account reads the bucket, BigQuery’s own access controls decide who sees the table, and Unity Catalog’s grants, row filters and lineage play no part. For a Delta folder that nothing else governs it is fine. For supply_service it would bypass everything Part 6 set up.

Which link should Acme use for which job?

which link Acme uses for which job: path 1 for marketing's daily joins in BigQuery, path 2 for supply reading marketing's Iceberg tables, path 3 for occasional reads of native BigQuery tables, the Spark BigQuery connector for heavy loads into BigQuery, path 4 only for Delta folders outside Unity Catalog, and OpenSharing for partners

The pattern is simple once the shapes are clear. Read in place wherever a catalog can describe the table (paths 1 and 2). Push a query across only for occasional reads of tables that only the other side can describe (path 3). Copy only when a platform-specific feature needs a native table. In the library terms from earlier: read it yourself, phone the librarian only now and then, photocopy only when you must. As a litmus test, cheapest and best governed first:

1. Is it a Unity Catalog table with Iceberg reads on?       -> path 1, BigQuery reads in place
2. Is it a BigQuery-managed Iceberg table?                  -> path 2, Databricks reads in place
3. Is it a native BigQuery table, read now and then?        -> path 3, Lakehouse Federation
4. Does a BigQuery-only feature need a native table?        -> copy with the Spark BigQuery connector
5. A Delta folder that nothing else governs?                -> path 4, BigLake external table

How do Gemini models work with Databricks?

Gemini meets Databricks in three places, and each answers a different need. The bottom line: where the model runs, and therefore where the prompt and its rows are processed, decides which of the three you use.

three ways Gemini and Databricks work together: Gemini models served by Databricks Foundation Model APIs and called with ai_query; an external model endpoint that calls Gemini in Acme's own Gemini Enterprise Agent Platform project; and agents in Gemini Enterprise that call Databricks through managed MCP servers; Unity Catalog and Unity Gateway govern the Databricks side of all three

Gemini inside Databricks. Databricks serves Gemini models through its Foundation Model APIs, the pay-per-token model endpoints that come with a workspace. As of October 2026 the list includes several Gemini models, for example databricks-gemini-3-8-flash, with the model service name system.ai.gemini-3-8-flash. The SQL function ai_query calls a model on every row of a query, which is how a data team runs batch inference without writing Python:

SELECT brand, region, week_start, otif_pct,
       ai_query('system.ai.gemini-3-8-flash',
                CONCAT('In one sentence, explain an OTIF of ', otif_pct,
                       '% for ', brand, ' in ', region, '.')) AS note
FROM acme_supply.supply_gold.supply_service
WHERE week_start >= '2026-05-04';

One condition matters for Acme. The Gemini 3.8 Flash entry, like several others, says the model “is hosted on a global endpoint and requires cross geography routing to be enabled”, and Databricks’ regional availability table marks pay-per-token Foundation Model APIs in europe-west2 as supported “only when cross geography routing is enabled”. So the prompt, and the rows inside it, may be processed outside the UK. That is a decision for Acme’s data protection team, made once at account level, before anyone writes the query above.

Gemini in Acme’s own project. If processing must stay in a region Acme chooses, Databricks external models can front Gemini Enterprise Agent Platform. The provider is still named google-cloud-vertex-ai in Databricks’ configuration, and it takes a project_id, a region and the full service account key JSON, stored as a Databricks secret. The endpoint then behaves like any other model endpoint, including inside ai_query. The cost is one more service account key, which Part 2’s least-privilege checklist covers.

Agents outside Databricks. The third direction is Gemini Enterprise agents reaching into Databricks. Databricks now provides managed MCP servers (MCP, the Model Context Protocol, is an open standard for exposing tools to agents) at https://<workspace-hostname>/ai-gateway/mcp-services/<mcp-name>. For analytics questions the documentation recommends system.ai.genie_one_mcp, which answers natural-language questions through Genie, grounded in a governed semantic layer, and it is generally available. My post wiring Gemini Enterprise to Databricks Genie walks through four ways to connect them so every answer runs as the user who asked, and the Production-ready agents series builds Acme’s supply agent on that pattern.

As a decision guide for Acme:

1. Is processing outside the UK acceptable for this data?   -> Foundation Model APIs and ai_query
2. Must prompts stay in a region Acme chooses?              -> external model to Agent Platform
3. Does the agent live in Gemini Enterprise?                -> managed MCP, such as genie_one_mcp

All three are governed on the Databricks side by Unity Catalog grants and Unity Gateway, which sets who may call which model or MCP service and caps spend. Part 7 covers Unity Gateway and Agent Bricks; here it is enough to know that the Gemini calls above are governed objects, not loose API keys.

How does Databricks stream from Pub/Sub?

Acme’s warehouse system publishes a scan event for every parcel. Those events feed supply_bronze, and eventually the units_delivered column of supply_service. Pub/Sub is the natural transport on Google Cloud: publishers send messages to a topic, and each subscription on that topic delivers every message to its subscribers, as the Pub/Sub overview explains.

streaming shipment events from Pub/Sub into Databricks: a warehouse system publishes to the topic acme-shipment-events, the subscription acme-dbx-shipments delivers messages to a Structured Streaming query that reads with format pubsub and a service credential, and the rows land in a bronze table with a checkpoint in a volume; Pub/Sub may redeliver, so the silver step removes duplicates by messageId; the stream's columns are messageId, payload, attributes and publishTimestampInMillis

Databricks reads a subscription with its built-in connector. The documented example authenticates with a service credential, the Unity Catalog object Part 2 introduced for Google APIs, which needs Databricks Runtime 16.1 or above. With Acme’s names, in Python:

events = (spark.readStream.format("pubsub")
    .option("subscriptionId", "acme-dbx-shipments")
    .option("topicId", "acme-shipment-events")
    .option("projectId", "acme-dbx-prod")
    .option("serviceCredential", "pubsub_shipments")
    .load())

(events.writeStream
    .option("checkpointLocation", "/Volumes/acme_supply/supply_bronze/checkpoints/shipments")
    .toTable("acme_supply.supply_bronze.shipment_events"))

The same read in a Lakeflow pipeline (formerly Delta Live Tables) is a streaming table over the read_pubsub table-valued function, using the same named arguments:

CREATE OR REFRESH STREAMING TABLE shipment_events AS
SELECT * FROM STREAM read_pubsub(
  subscriptionId => 'acme-dbx-shipments', topicId => 'acme-shipment-events',
  projectId => 'acme-dbx-prod', serviceCredential => 'pubsub_shipments');

The service credential’s Google service account needs Pub/Sub Viewer (to check that the subscription exists) and Pub/Sub Subscriber (to fetch messages). Pub/Sub Editor is optional: it lets the connector create the subscription if it is missing and delete it when the stream stops. If you grant the two required roles on the topic and subscription rather than the project, grant both on both. Without service credentials, the connector accepts a service account’s key fields (clientEmail, clientId, privateKey, privateKeyId), but you cannot attach a Google service account to compute in standard access mode.

On delivery guarantees, the page says two things that sound contradictory until you separate them. The common misconception is that “exactly-once” means “no duplicates”. It does not.

A simple example. The post office occasionally delivers the same letter twice, and sometimes a Tuesday letter arrives before a Monday one. Your mailroom logs every envelope it receives exactly once, so nothing it received is lost or logged twice. But if the post office delivered two copies, both are logged, and it is your clerk’s job to bin the second by checking the reference number. The post office is Pub/Sub, the mailroom log is the connector’s exactly-once processing, and the reference number is the messageId.

The connector “has exactly-once processing semantics for rows from the subscriber”: a row it fetched is processed once. But “Pub/Sub might publish duplicate rows, or rows might arrive to the subscriber out of order”, and handling that is your job. How does this apply to Acme? The clerk is the silver step: it removes duplicates by messageId within a watermark, using Spark’s dropDuplicatesWithinWatermark:

from pyspark.sql import functions as F
silver = (spark.readStream.table("acme_supply.supply_bronze.shipment_events")
    .withColumn("published_at", F.timestamp_millis("publishTimestampInMillis"))
    .withWatermark("published_at", "1 hour")
    .dropDuplicatesWithinWatermark(["messageId"]))

How does Looker connect to Databricks?

Looker connects to a Databricks SQL warehouse with its Databricks dialect, and both vendors document the connection: Databricks’ Looker page and Looker’s own Databricks page. The host is the workspace URL, the port 443, and the additional JDBC parameters must include transportMode=http;httpPath=<the warehouse's HTTP path>.

The bottom line: Looker is just another SQL client of a warehouse, so the two questions are the ones from Part 7: whose identity does it query as, and does it let the warehouse stop?

A simple example for the second question. A warehouse with automatic lights switches them off when nobody has moved for ten minutes. A security guard who walks through every five minutes, just to check, keeps the lights on all night. Looker’s default check on persistent derived tables is that guard, and a SQL warehouse’s auto-stop is the lights.

Three settings decide whether it is governed and affordable.

Catalog. With Unity Catalog, set the connection’s Catalog field to acme_supply; without it, Looker only sees schemas in the default catalog.
Authentication. A database account means the username token and a personal access token, ideally a service principal’s, so every Looker user queries as one identity. OAuth makes “each Looker user” authorise Looker “with the user’s own OAuth user account”, so Unity Catalog applies each person’s own grants. The catch: persistent derived tables are not supported with OAuth, and caching becomes per user.
Maintenance schedule. Persistent derived tables (PDTs) are tables Looker builds and refreshes in a scratch schema. By default Looker checks them every five minutes, and Looker’s page warns this “can prevent Databricks clusters from shutting down, which may cause unexpected costs”. Set a cron schedule that matches how often the data really changes.

In Acme’s architecture, marketing’s Looker dashboards sit on BigQuery, which already reads supply_service through path 1. A Databricks connection in Looker is for supply’s own dashboards on silver and gold tables.

Acme’s largest retailer wants weekly service levels for Acme’s brands in its stores, refreshed as soon as supply publishes them. The retailer runs Databricks on another cloud. Copying files to it every week would create a second copy to secure and reconcile. OpenSharing (formerly Delta Sharing) shares the live table instead. The bottom line: OpenSharing gives a partner read access to the live table in place, through short-lived links, instead of sending them a copy.

A simple example. Rather than posting a printed price list to a customer every week, a shop gives the customer a link that opens the current price list and expires after a short time; each time they want it, they ask for a fresh link. The shop keeps one list, and the customer never reads an old version by accident. The shop is the provider, the customer is the recipient, the list of what the customer may open is a share, and the expiring link is a pre-signed URL.

How does this apply to Acme? The shop is Acme, the customer is the retailer, and the price list is supply_service.

OpenSharing has a provider (Acme), one or more recipients (the retailer), and shares, which are read-only collections of tables, views and other assets registered in Unity Catalog. There are two protocols. Databricks-to-Databricks sharing works between Unity Catalog metastores, needs no token and can also share volumes, models, notebooks and views. Databricks-to-Open sharing serves any client, such as Apache Spark, pandas or Power BI, using a bearer token or OIDC federation.

how OpenSharing serves supply_service to a partner: the recipient's client asks Acme's sharing server for the table, the server checks the share and recipient in Unity Catalog, returns short-lived pre-signed URLs to the Parquet files of the current version, and the client reads those files directly from Acme's bucket; reads from another region or cloud create egress charged to Acme

The flow is what keeps it to one copy. The recipient’s client asks the provider’s sharing server for a table, the server checks the share in Unity Catalog, and the data “is delivered to recipients using pre-signed short-lived URLs” to the Parquet files of the current version. The client reads those files directly from Acme’s bucket. The setup is four statements, from the create share, create recipient and grant access pages:

CREATE SHARE IF NOT EXISTS supply_share;
ALTER SHARE supply_share ADD TABLE acme_supply.supply_gold.supply_service WITH HISTORY;
CREATE RECIPIENT IF NOT EXISTS retail_partner USING ID '<cloud>:<region>:<metastore uuid>';
GRANT SELECT ON SHARE supply_share TO RECIPIENT retail_partner;

The USING ID value is the recipient metastore’s sharing identifier, which the retailer sends Acme. WITH HISTORY lets the recipient time travel, stream from the table and read its change data feed.

Egress is the Google Cloud architect’s part. “OpenSharing within a region incurs no egress cost”, but recipients read from the provider’s bucket, so reads from another region or cloud are charged by the storage vendor, here to Acme’s Cloud Storage bill. For heavy, repeated reads Databricks’ egress page suggests replicas near the recipient, kept fresh with DEEP CLONE or a change data feed. One overlap with the Iceberg section: some table features that Iceberg reads use are not supported by every OpenSharing client, so test a UniForm table with the retailer’s client before you promise it.

What does Acme’s reference architecture look like?

Put together, Acme’s design keeps one copy of supply data in its own bucket and lets each engine do what it is best at.

Acme's reference architecture on Google Cloud: events, files and SaaS data are ingested into Databricks, Lakeflow pipelines build bronze and silver tables and a gold table with Iceberg reads, all as Delta files in Acme's Cloud Storage bucket governed by Unity Catalog; BigQuery reads the gold table in place and joins it with marketing data for Looker and analysts, Gemini Enterprise agents reach Databricks through managed MCP, and partners get live data through OpenSharing; Cloud KMS, VPC Service Controls, IAM and Cloud Logging sit underneath

Reading it left to right:

Ingest. Shipment events arrive through Pub/Sub, files land in Cloud Storage for Auto Loader, and SaaS and database sources come through Lakeflow Connect, which Part 7 covers.
Process. Lakeflow pipelines build supply_bronze and supply_silver in Databricks, then the gold table supply_gold.supply_service, with Iceberg reads turned on. Bronze and silver are Databricks-only; nothing outside Databricks needs them.
Store and govern. All of it is Delta in gs://acme-lake-supply, governed by Unity Catalog, which also serves the Iceberg REST endpoint.
Serve. BigQuery is the gold serving layer for marketing: it reads supply_service in place through path 1 and joins it with campaign_performance, and marketing’s Looker dashboards and analysts work there. Gemini Enterprise agents reach Databricks through managed MCP. The retailer gets live data through OpenSharing. Gemini models run inside Databricks for enrichment.

In short: Databricks builds bronze, silver and gold, and BigQuery serves gold to the people who live in BigQuery, without a copy. The layer underneath, keys, perimeters, IAM and logging, belongs to Part 3 and Part 6.

What can go wrong?

troubleshooting the Google Cloud integrations: empty federated catalogs, stale data in BigQuery, Iceberg reads that will not turn on, federated queries that pass the connection test but fail, missing BigQuery User, 403 errors in Lakehouse catalog federation, Gemini endpoints unavailable in the region, duplicate Pub/Sub rows and warehouses kept awake by Looker

A few of these deserve a sentence more.

Treating UniForm as free. Iceberg reads add a write protocol feature, switch Parquet compression to Zstandard, and rule out deletion vectors. Turn it on for tables that other engines really need to read.
Forgetting that federated catalogs lag. BigQuery sees a new version of supply_service only after Databricks has generated Iceberg metadata and the federated catalog has refreshed. A dashboard that must match Databricks to the minute needs a short refresh interval and a check on converted_delta_version.
Building on default storage. A catalog on Databricks default storage cannot be federated to BigQuery today. Decide where gold tables live before you build them.
Granting the federation principal too much. In the default service identity mode, BigQuery’s readers inherit whatever the service principal can read. Grant EXTERNAL USE SCHEMA on gold schemas only.
Testing federation from the wrong place. A passing Test connection, or a passing check from a classic cluster, says nothing about the Storage API path or about serverless egress.
Assuming Gemini stays in region. Pay-per-token Gemini in europe-west2 relies on cross geography routing. If that is not acceptable, use an external model endpoint in your own region.
Paying egress by surprise. OpenSharing to another region or cloud bills every read to the provider’s bucket.

What would I do now?

List every table that another engine must read, and who reads it. For most estates it is a handful of gold tables, not the lakehouse.
Pick one table that another engine must read, and check whether it meets the UniForm requirements: Unity Catalog, an external location, column mapping, protocol versions and no deletion vectors (or a reader that supports Iceberg v3).
Turn on external data access, create the federation service principal with EXTERNAL USE SCHEMA on that one schema, and build a federated catalog in a sandbox project with OIDC. Query it from BigQuery and compare row counts with Databricks.
For tables that only BigQuery can describe, set up Lakehouse Federation with a dedicated service account, run the four-endpoint check from each compute type, and decide on materialisation per catalog.
Agree the Gemini routing decision (Foundation Model APIs with cross geography routing, or an external model in your region) with your data protection team before the first ai_query reaches production.

What did we learn?

Every link between Databricks and Google Cloud’s data services is one of three shapes: catalog federation reads files in place, query federation moves rows, and sharing hands short-lived file links to another organisation.
BigQuery reads Unity Catalog tables in place through a read-only Lakehouse federated catalog (generally available since 2 November 2026), which needs an external location, external data access, a service principal with EXTERNAL USE SCHEMA and tables with Iceberg reads turned on.
Iceberg reads (UniForm) are three table properties that make Databricks write Iceberg metadata next to the Delta log, at the cost of a protocol upgrade, Zstandard compression and no deletion vectors unless readers support Iceberg v3.
Databricks reads BigQuery-managed Iceberg tables in place through Lakehouse catalog federation (Beta), and queries any BigQuery table through Lakehouse Federation (GA), where materialisation trades extra BigQuery compute for more pushdown.
Gemini works with Databricks in three ways: Foundation Model APIs inside Databricks, external model endpoints in your own project, and Gemini Enterprise agents calling Databricks through managed MCP.
Pub/Sub, Looker and OpenSharing each have one setting that matters most: deduplicate redelivered messages, set Looker’s maintenance schedule, and watch cross-region egress.

What comes next?

Every link in this post has a bill attached: BigQuery slots on path 1, BigQuery jobs on materialisation, DBUs on Databricks, egress on OpenSharing. In Databricks on Google Cloud, Part 9: Commercials, quotas and operations, we look at how Databricks is bought and billed through Google Cloud Marketplace, the quotas and limits that stop a launch, and how to run the platform from day two. The series overview lists every part.

Sources

Opinions are my own. Every fact in this post comes from the public documentation linked above. Acme Consumer Brands and its data are fictional.