Cloud Blog

Google Cloud named a Leader in The Forrester Wave™: Sovereign Cloud Platforms, Q2 2026

Wed, 08 Apr 2026 17:00:00 +0000

In today’s global economy, data is a strategic asset. For many organizations — particularly those in highly regulated industries and the public sector — the ability to innovate with AI is often balanced against the rigorous requirements of data sovereignty, residency, and operational autonomy.

We are proud to announce that Google Cloud has been named a Leader in The Forrester Wave™: Sovereign Cloud Platforms, Q2 2026.

The Forrester Wave™: Sovereign Cloud Platforms, Q2 2026

As organizations move beyond simple data residency toward full digital sovereignty, this report validates our commitment to providing a sovereignty-by-design approach. "Google is an ideal choice for organizations that need a full range of sovereign cloud options for their deployments," Forrester said in their report.

Meeting customers where they are: A platform of choice

There's no one-size-fits-all approach for achieving digital sovereignty. Our strategy is built on providing a consistent experience, including AI solutions, across three distinct sovereign cloud platforms, so that enterprise and government organizations can innovate and meet their compliance obligations.

Google Cloud Data Boundary, delivered with Assured Workloads, provides a sovereign data and access boundary in the public cloud, including controls over data residency, access, and personnel. It’s designed to give you the agility and scale of global infrastructure while enforcing strict rules about where your data lives and who can access it. By using customer-managed encryption keys, external key manager, and localized access policies, administrative actions remain transparent and restricted. This option is a strong fit for commercial enterprises, regulated industries, and public sector organizations that need to meet regional compliance obligations without the complexity of isolated infrastructure and operational sovereignty.

Google Cloud Dedicated, designed for organizations seeking a higher level of control, provides complete regional data and operational sovereignty delivered by a regional independent operator — and is designed to be survivable up to a year even without Google. This environment is managed by a trusted local partner who oversees operations. This creates a functional buffer between your organization and Google, helping ensure that your cloud remains compliant with specific local governance. It is specifically targeted at organizations that require a cloud with operational sovereignty, offering the peace of mind that critical infrastructure can continue to function even if the connection with Google is interrupted. For example, in France, S3NS, a standalone entity, offers PREMI3NS built on Google Cloud Dedicated. PREMI3NS has achieved the SecNumCloud 3.2 qualification from the French National Agency for the Security of Information Systems (ANSSI), one of the most demanding sovereignty standards in the world.

Google Distributed Cloud, an on-premises solution offered to organizations with strict compliance, latency, and data sovereignty requirements that prevent public cloud adoption. Designed for maximum flexibility, Google Distributed Cloud (GDC) offers both connected and air-gapped configurations to meet your sovereignty requirements. The fully air-gapped deployment option operates without any external connection to the public internet or the Google network. Because it is physically self-contained in your own facility, it is designed to prevent remote access, updates, and shut downs by Google. This solution is the preferred choice for defense, intelligence, and the most security-conscious customers in highly regulated sectors who cannot risk any external exposure.

Sovereign by design

One of the key differentiators that Forrester noted is Google Cloud's roadmap, which calls for delivering sovereignty as a standard feature. Forrester said that Google Cloud's roadmap involves delivering sovereignty as a standard feature, ensuring consistency across all three sovereign cloud offerings.

This consistency is especially prominent in our AI capabilities. Forrester highlighted that our AI offering is a "true differentiator" and that Google Cloud excels "at AI sovereign development services and applications services across all three sovereign environments.”

Looking ahead

Being named a Leader in the Forrester Wave™: Sovereign Cloud Platforms, 2026 is a milestone in our journey to help every organization achieve digital autonomy. We remain committed to our partnerships with local players and our "sovereignty-by-design" philosophy.

Want to dive deeper into the report? Download the full Forrester Wave™: Sovereign Cloud Platforms, Q2 2026 report here.

New GKE Cloud Storage FUSE Profiles take the guesswork out of configuring AI storage

Wed, 08 Apr 2026 16:30:00 +0000

In the world of AI/ML, data is the fuel that drives training and inference workloads. For Google Kubernetes Engine (GKE) users, Cloud Storage FUSE provides high-performance, scalable access to data stored in Google Cloud Storage. However, we learned from customers that getting the maximum performance out of Cloud Storage FUSE can be complex.

Today, we are excited to introduce GKE Cloud Storage FUSE Profiles, a new feature designed to automate performance tuning and accelerate data access for your AI/ML workloads (training, checkpointing, or inference) with minimal operational overhead. With these profiles, tuned for your specific workload needs, you can enjoy high performance of Cloud Storage FUSE out of the box.

Before (manual tuning)

code_block: <ListValue: [StructValue([('code', 'apiVersion: v1\r\nkind: PersistentVolume\r\nmetadata:\r\n name: serving-bucket-pv\r\nspec:\r\n accessModes:\r\n - ReadWriteMany\r\n capacity:\r\n storage: 64Gi\r\n persistentVolumeReclaimPolicy: Retain\r\n storageClassName: ""\r\n claimRef:\r\n name: serving-bucket-pvc\r\n mountOptions:\r\n - implicit-dirs\r\n - metadata-cache:ttl-secs:-1\r\n - metadata-cache:stat-cache-max-size-mb:-1\r\n - metadata-cache:type-cache-max-size-mb:-1\r\n - file-cache:max-size-mb:-1\r\n - file-cache:cache-file-for-range-read:true\r\n - file-system:kernel-list-cache-ttl-secs:-1\r\n - file-cache:enable-parallel-downloads:true\r\n - read_ahead_kb=1024\r\n csi:\r\n driver: gcsfuse.csi.storage.gke.io\r\n volumeHandle: BUCKET_NAME\r\n volumeAttributes:\r\n skipCSIBucketAccessCheck: "true"\r\n gcsfuseMetadataPrefetchOnMount: "true"\r\n---\r\napiVersion: v1\r\nkind: PersistentVolumeClaim\r\nmetadata:\r\n name: serving-bucket-pvc\r\nspec:\r\n accessModes:\r\n - ReadWriteMany\r\n resources:\r\n requests:\r\n storage: 64Gi\r\n volumeName: serving-bucket-pv\r\n storageClassName: ""\r\n–--\r\napiVersion: v1\r\nkind: Pod\r\nmetadata:\r\n name: gcs-fuse-csi-example-pod\r\n annotations:\r\n gke-gcsfuse/volumes: "true"\r\nspec:\r\n containers:\r\n # Your workload container spec\r\n ...\r\n volumeMounts:\r\n - name: serving-bucket-vol\r\n mountPath: /serving-data\r\n readOnly: true\r\n serviceAccountName: KSA_NAME \r\n volumes:\r\n - name: gke-gcsfuse-cache # gcsfuse file cache backed by RAM Disk\r\n emptyDir:\r\n medium: Memory \r\n - name: serving-bucket-vol\r\n persistentVolumeClaim:\r\n claimName: serving-bucket-pvc'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5078ce9820>)])]>

After (Cloud Storage FUSE mount options, CSI configs, and file cache medium automatically configured!)

code_block: <ListValue: [StructValue([('code', 'apiVersion: v1\r\nkind: PersistentVolume\r\nmetadata:\r\n name: serving-bucket-pv\r\nspec:\r\n accessModes:\r\n - ReadWriteMany\r\n capacity:\r\n storage: 64Gi\r\n persistentVolumeReclaimPolicy: Retain\r\n storageClassName: gcsfusecsi-serving\r\n claimRef:\r\n name: serving-bucket-pvc\r\n csi:\r\n driver: gcsfuse.csi.storage.gke.io\r\n volumeHandle: BUCKET_NAME\r\n---\r\napiVersion: v1\r\nkind: PersistentVolumeClaim\r\nmetadata:\r\n name: serving-bucket-pvc\r\nspec:\r\n accessModes:\r\n - ReadWriteMany\r\n resources:\r\n requests:\r\n storage: 64Gi\r\n volumeName: serving-bucket-pv\r\n storageClassName: gcsfusecsi-serving\r\n–--\r\napiVersion: v1\r\nkind: Pod\r\nmetadata:\r\n name: gcs-fuse-csi-example-pod\r\n annotations:\r\n gke-gcsfuse/volumes: "true"\r\nspec:\r\n containers:\r\n # Your workload container spec\r\n ...\r\n volumeMounts:\r\n - name: serving-bucket-vol\r\n mountPath: /serving-data\r\n readOnly: true\r\n serviceAccountName: KSA_NAME \r\n volumes: \r\n - name: serving-bucket-vol\r\n persistentVolumeClaim:\r\n claimName: serving-bucket-pvc'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5078ce9bb0>)])]>

The trouble with optimizing Cloud Storage FUSE

Optimizing Cloud Storage FUSE for high-performance workloads is a multi-dimensional problem. Historically, users had to navigate manual configuration guides that could span dozens of pages. And as AI/ML has evolved, Cloud Storage FUSE’s capabilities have also increased, with new mount options available to accelerate your workloads. The "right" settings were never static; they depended heavily on a variety of dynamic factors:

Bucket characteristics: The total size of your dataset and the number of objects significantly impact metadata and file cache requirements.
Infrastructure variability: Optimal configurations change based on whether you are using GPUs, TPUs, or general-purpose compute.
Node resources: Available RAM and Local SSD capacity determine how much data can be cached locally to minimize expensive round-trips to Cloud Storage.
Workload patterns: A training workload (high-throughput reads of large datasets) requires different tuning than a checkpointing workload (bursty, high-throughput writes) or a serving workload (latency-sensitive model loading).

In fact, many customers leave available performance on the table or face reliability issues (e.g., Pod Out-of-Memory kills) due to unoptimized or misconfigured Cloud Storage FUSE settings.

Introducing Cloud Storage FUSE Profiles for GKE

GKE Cloud Storage FUSE Profiles simplify this complexity with pre-defined, dynamically managed StorageClasses tailored for specific AI/ML patterns. Instead of manually adjusting dozens of mount options, you simply select a profile that matches your workload type.

These profiles operate on a layered model. They take the base best practices from Cloud Storage FUSE and add a GKE-specific intelligence layer. When you deploy a Pod using a profile, GKE automatically:

Scans your bucket (or a specific directory) to understand its size and object count.
Analyzes the target node to check for available RAM, Local SSD, and accelerator types.
Calculates optimal cache sizes and selects the best backing medium (RAM or Local SSD) automatically.

We are launching with three primary profiles:

gcsfusecsi-training: Optimized for high-throughput reads to keep GPUs and TPUs fed with data.
gcsfusecsi-serving: Optimized for model loading and inference, with automated Rapid Cache integration.
gcsfusecsi-checkpointing: Optimized for fast, reliable writes of large multi-gigabyte checkpoint files.

Using GKE Cloud Storage FUSE Profiles delivers several benefits:

Simplified tuning: Replace complex, error-prone manual configurations with three simple, purpose-built StorageClasses.
Dynamic, resource-aware optimization: The CSI driver automatically adjusts cache sizes based on real-time environment signals, so that you can maximize performance without risking node stability.
Accelerated read performance: The serving profile automatically triggers Rapid Cache, placing your data closer to your compute for faster cold-start model loading.
Granular performance insights: Gain visibility into automated tuning decisions through structured logs that detail exactly why specific cache sizes and mediums were selected for your Pod.

Using GKE Cloud Storage FUSE Profiles inference profile, we were able to reduce model loading time for a Qwen3-235B-A22B workload on TPUs (480GB) from 39 hours to just 14 minutes, helping customers achieve the maximum benefit of Cloud Storage FUSE GCSFuse out-of-the-box.

How to use Cloud Storage FUSE Profiles on GKE

To get started, ensure your cluster is running GKE version 1.35.1-gke.1616000 or later with the Cloud Storage FUSE CSI driver enabled.

1. Identify the StorageClass

GKE comes pre-installed with the profile-based StorageClasses. You can verify them with:

code_block: <ListValue: [StructValue([('code', 'kubectl get sc -l gke-gcsfuse/profile=true'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5078ce9340>)])]>

2. Create your PV and PVC

When creating your PersistentVolume, point it to your Cloud Storage bucket. GKE automatically initiates a bucket scan to determine the optimal configuration.

code_block: <ListValue: [StructValue([('code', 'apiVersion: v1\r\nkind: PersistentVolume\r\nmetadata:\r\n name: gcs-pv\r\nspec:\r\n accessModes:\r\n - ReadWriteMany\r\n capacity:\r\n storage: 5Gi\r\n persistentVolumeReclaimPolicy: Retain \r\n storageClassName: gcsfusecsi-training\r\n mountOptions:\r\n - only-dir=my-ml-dataset-subdirectory # Optional\r\n csi:\r\n driver: gcsfuse.csi.storage.gke.io\r\n volumeHandle: my-ml-dataset-bucket\r\n---\r\napiVersion: v1\r\nkind: PersistentVolumeClaim\r\nmetadata:\r\n name: gcs-pvc\r\nspec:\r\n accessModes:\r\n - ReadWriteMany\r\n resources:\r\n requests:\r\n storage: 5Gi\r\n storageClassName: gcsfusecsi-training\r\n volumeName: gcs-pv'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5078ce9c40>)])]>

3. Create your Deployment

Once your Persistent Volume Claim (PVC) is bound, simply consume it in your Deployment as you would any other volume. GKE mounts the volume with the precise settings your hardware and dataset require.

code_block: <ListValue: [StructValue([('code', 'apiVersion: apps/v1\r\nkind: Deployment\r\nmetadata:\r\n name: my-deployment\r\nspec:\r\n replicas: 3\r\n selector:\r\n matchLabels:\r\n app: my-app\r\n template:\r\n metadata:\r\n labels:\r\n app: my-app\r\n annotations:\r\n gke-gcsfuse/volumes: "true"\r\n spec:\r\n serviceAccountName: my-ksa\r\n containers:\r\n - name: my-container\r\n image: busybox\r\n volumeMounts:\r\n - name: my-gcs-volume\r\n mountPath: "/data"\r\n volumes:\r\n - name: my-gcs-volume\r\n persistentVolumeClaim:\r\n claimName: gcs-pvc'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5078ce93d0>)])]>

After it's deployed, the CSI driver automatically calculates optimal cache sizes and mount options based on your node's resources, such as GPUs or TPUs, memory, Local SSD, the bucket or sub-directory size, and the sidecar resource limits.

Get started today

GKE Cloud Storage FUSE Profiles remove the guesswork from configuring your cloud storage for high performance. By moving from manual "knob-turning" to automated, workload-aware profiles, you can spend less time debugging storage throughput and more time building the next generation of AI.

Ready to get started? GKE Cloud Storage FUSE Profiles are generally available in version 1.35.1-gke.1616000. Explore the official documentation to configure Cloud Storage FUSE profiles in GKE for your AI/ML workloads!

Openness without compromises for your Apache Iceberg lakehouse

Wed, 08 Apr 2026 16:00:00 +0000

Today, at the Apache Iceberg Summit in San Francisco, we are announcing the preview of read and write interoperability between BigQuery and Iceberg-compatible engines, including Trino, Spark, and others in Apache Iceberg tables in Google-managed Iceberg REST Catalog. With this new capability, you get the benefits of enterprise-grade native storage for your lakehouse without sacrificing Iceberg's openness and flexibility.

Why it matters: If you're building a lakehouse today, you're probably using Apache Iceberg, which has gained massive popularity among data platform teams that need to support multiple compute engines (like Spark and BigQuery) that access the same data for different workloads. However, we consistently hear from customers that achieving openness often requires compromises.

Compared to using enterprise storage, there’s often price-performance overhead on using Iceberg, wiping out the cost benefits of a single-copy architecture. In order to make Iceberg work for all production use cases, data teams have to invest in custom infrastructure to handle real-time streaming, build complex pipelines to replicate operational data, and navigate fragmented governance across different compute engines. Ultimately, these limitations become bottlenecks to innovation.

Over the years, Google has purpose-built storage infrastructure to solve these exact challenges at scale, powered by highly scalable, real-time metadata, unified governance, and deep vertical integration across Cloud Storage, metadata, and various query engines. We are making this infrastructure available directly in Iceberg.

This enables access to BigQuery's advanced runtime, automatic table management, partitioning, multi-statement transactions, and change data replication for Google-managed Iceberg REST catalog tables. These features will be available in preview for Google-managed Iceberg REST catalog tables and will be generally available (GA) for BigQuery-managed Iceberg tables, coming next month.

Write and read interoperability across engines

Previously, customers building lakehouses chose between Iceberg tables in the Google-managed Iceberg REST catalog or tables managed by BigQuery based on their primary ETL engine. That means that customers relying on Apache Spark for ETL to Iceberg REST Catalog tables couldn’t write through BigQuery or use its storage management features.

With this preview, you can create, update, and query Iceberg tables in the Google serverless Iceberg REST catalog with BigQuery or other Iceberg-compatible engines such as Spark, Flink, Trino and others. This two-way read and write interoperability enables data teams to implement multi-engine use cases on a single table type in a fully open manner, using native Iceberg libraries.

Additionally, Iceberg REST Catalog offers table-level access controls using credential vending for uniform governance across BigQuery, Spark and other compute engines that query or modify your Iceberg tables.

Google Cloud also supports a robust ecosystem of partners integrated with the Iceberg REST Catalog across data platforms and engines, transformation and ingestion services, and governance platforms. We work closely with the Iceberg ecosystem to strengthen these partnerships with many more to come.

Improved price-performance with BigQuery and Spark

Automate table management

Achieving strong query performance on Apache Iceberg tables out of the box can be hard. You need to choose the optimal target file size (which tends to be different for different compute engines), data organization strategy (partitioning and sort-order choices have their tradeoffs), and take care of table management to avoid small files problems and metadata bloat.

Apache Iceberg lakehouse customers can now offload table maintenance — compaction and garbage collection — to Google Cloud BigLake, which optimizes performance for you. In addition to Iceberg tables in BigQuery, it will be available for Google-managed Iceberg REST catalog tables in preview, coming next month. You can opt-in to table management by setting a single property, and improve your BigQuery performance on the industry standard TPC-DS 10T benchmark by ~40%.

Improve BigQuery price-performance with advanced runtime

BigQuery advanced runtime offers a set of performance enhancements designed to automatically accelerate analytical workloads without requiring user action or code changes. In particular, it extends the vectorized query execution enhancements in BigQuery to open table formats. Advanced runtime will be available in preview for Google-managed Iceberg REST catalog tables and in GA for BigQuery-managed Iceberg tables, coming next month. According to an internal TCP-DS 10T benchmark, advanced runtime can help additionally accelerate BigQuery query performance on Iceberg tables, providing 2x faster performance vs. a self-managed approach based on internal benchmarking.

Chart based on benchmarks from internal data and testing.

Accelerate Spark performance with Lightning Engine

Apache Spark is a leading compute engine for Apache Iceberg lakehouses, for use cases ranging from ETL to feature engineering. However, achieving high performance and cost efficiency for Spark workloads at scale can be challenging. Lightning Engine accelerates Apache Spark query performance by over 4 times compared to open source Spark (based on a TPCH-like benchmark).

Optimize table layout with BigQuery partitioning and clustering

Many open-source libraries and engines rely on Iceberg table partitioning for effective data pruning. BigQuery time-based partitioning will be available in preview for Google-managed Iceberg REST catalog tables and will be generally available (GA) for BigQuery-managed Iceberg tables, coming next month. Additionally, when you are creating Iceberg tables in BigQuery, you can define clustering columns to organize data in Parquet files, helping to achieve optimal query performance and avoiding common issues with partitioning such as high-cardinality columns, small partition inefficiencies, and multiple filter columns. For example, one common pattern is to combine time-based table partitioning with clustering on other dimensions that are frequently used for query filtering, such as region, store, etc.

Advanced analytics with Apache Iceberg

Streaming with Apache Iceberg

To operationalize real-time analytics with Iceberg, you can leverage BigQuery’s Vortex streaming infrastructure for high-throughput ingestion with zero-read latency. This removes the need for bespoke infrastructure, addresses small file issues, and lets you query data immediately from the streaming buffer to achieve near-zero read latency. This feature is generally available for BigQuery-managed Iceberg tables and will be available in preview for Google-managed Iceberg REST catalog tables, coming next month.

Replicate data from operational databases to Iceberg tables with Datastream

You can now easily replicate data from a variety of operational datastores, including MySQL, Postgres, SQLserver, Oracle, Salesforce, and MongoDB , into managed Iceberg tables in BigQuery using Datastream integration (GA).

Illustration of Datastream creation to replicate MySQL data to managed Iceberg tables in BigQuery.

Incremental processing with change data capture ingestion to Iceberg tables

The BigQuery storage write API’s change data replication feature lets you stream insert, update, and delete changes from OLTP databases to Iceberg tables in real time, removing the need for complex MERGE-based ETL pipelines. This feature will be available in preview for Google-managed Iceberg REST catalog tables and generally available (GA) for BigQuery-managed Iceberg tables, coming next month.

Illustration of change data capture ingestion to a managed Iceberg table in BigQuery.

Multi-statement transactions

Many analytics workloads require transactions that span multiple tables to commit or roll back changes atomically. This provides consistency across logical groups of tables, synchronizes dimensions and fact tables, and simplifies multi-stage ETLs. You can now leverage BigQuery multi-statement transactions to radically simplify complex multi-table processing with Iceberg. This feature will be available in preview for Google-managed Iceberg REST catalog tables and generally available (GA) for BigQuery-managed Iceberg tables, coming next month.

Illustration of a multi-statement transaction in a managed Iceberg table in BigQuery.

Get started

With bidirectional interoperability across BigQuery and other Iceberg-compatible engines on Google-managed Iceberg REST catalog tables, you can modernize your lakehouse with Apache Iceberg without compromising on performance, governance, or advanced analytics.

Ready to start building today? Learn more about our lakehouse capabilities and explore our quickstart guides.

Experimenting with GPUs: GKE managed DRANET and Inference Gateway AI Deployment

Wed, 08 Apr 2026 10:05:00 +0000

Building and serving models on infrastructure is a strong use case for businesses. In Google Cloud, you have the ability to design your AI infrastructure to suit your workloads. Recently, I experimented with Google Kubernetes Engine (GKE) managed DRANET while deploying a model for inference with NVIDIA B200 GPUs on GKE. In this blog, we will explore this setup in easy to follow steps.

What is DRANET

Dynamic Resource Allocation (DRA) is a feature that lets you request and share resources among Pods. DRANET allows you to request and allocate networking resources for your Pods, including network interfaces that support TPUs & Remote Direct Memory Access (RDMA). In my case, the use of high-end GPUs.

How GPU RDMA VPC works

The RDMA network is set up as an isolated VPC, which is regional and assigned a network profile type. In this case, the network profile type is RoCEv2. This VPC is dedicated for GPU-to-GPU communication. The GPU VM families have RDMA capable NICs that connect to the RDMA VPC. The GPUs communicate between multiple nodes via this low latency, high speed rail aligned setup.

Design pattern example

Our aim was to deploy a LLM model (Deepseek) onto a GKE cluster with A4 nodes that support 8 B200 GPUs and serve it via GKE Inference gateway privately. To set up an AI Hypercomputer GKE cluster, you can use the Cluster Toolkit, but in my case, I wanted to test the GKE managed DRANET dynamic setup of the networking that supports RDMA for the GPU communication.

This design utilizes the following services to provide an end-to-end solution:

VPC: Total of 3 VPC. One VPC manually created, two created automatically by GKE managed DRANET, one standard and one for RDMA.
GKE: To deploy the workload.
GKE Inference gateway: To expose the workload internally using a regional internal Application Load Balancers type gke-l7-rilb.
A4 VM’s: These support RoCEv2 with NVIDIA B200 GPU.

Putting it together

To get access to the A4 VM a future reservation was used. This is linked to a specific zone.

Begin: Set up the environment

Create a standard VPC, with firewall rules and subnet in the same zone as the reservation.
Create a proxy-only subnet this will be used with the Internal regional application load balancer attached to the GKE inference gateway

Next: Create a standard GKE cluster node and default node pool.

code_block: <ListValue: [StructValue([('code', 'gcloud container clusters create $CLUSTER_NAME \\\r\n --location=$ZONE \\\r\n --num-nodes=1 \\\r\n --machine-type=e2-standard-16 \\\r\n --network=${GVNIC_NETWORK_PREFIX}-main \\\r\n --subnetwork=${GVNIC_NETWORK_PREFIX}-sub \\\r\n --release-channel rapid \\\r\n --enable-dataplane-v2 \\\r\n --enable-ip-alias \\\r\n --addons=HttpLoadBalancing,RayOperator \\\r\n --gateway-api=standard \\\r\n --enable-ray-cluster-logging \\\r\n --enable-ray-cluster-monitoring \\\r\n --enable-managed-prometheus \\\r\n --enable-dataplane-v2-metrics \\\r\n --monitoring=SYSTEM'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f50781d1580>)])]>

Once that is complete you can connect to your cluster:

code_block: <ListValue: [StructValue([('code', 'gcloud container clusters get-credentials $CLUSTER_NAME --zone $ZONE --project $PROJECT'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f50781d12b0>)])]>

Create a GPU node pool (this example uses, A4 VM with reservation) and additionals flags:

---accelerator-network-profile=auto (GKE automatically adds the gke.networks.io/accelerator-network-profile: auto label to the nodes)

--node-labels=cloud.google.com/gke-networking-dra-driver=true (Enables DRA for high-performance networking)

code_block: <ListValue: [StructValue([('code', 'gcloud beta container node-pools create $NODE_POOL_NAME \\\r\n --cluster $CLUSTER_NAME \\\r\n --location $ZONE \\\r\n --node-locations $ZONE \\\r\n --machine-type a4-highgpu-8g \\\r\n --accelerator type=nvidia-b200,count=8,gpu-driver-version=latest \\\r\n --enable-autoscaling --num-nodes=1 --total-min-nodes=1 --total-max-nodes=3 \\\r\n --reservation-affinity=specific \\\r\n--reservation=projects/$PROJECT/reservations/$RESERVATION_NAME/reservationBlocks/$BLOCK_NAME \\\r\n --accelerator-network-profile=auto \\\r\n--node-labels=cloud.google.com/gke-networking-dra-driver=true'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f50781d1fd0>)])]>

Next: Create a ResourceClaimTemplate, which will be used to attach the networking resources to your deployments. The deviceClassName: mrdma.google.com is used for GPU workloads:

code_block: <ListValue: [StructValue([('code', 'apiVersion: resource.k8s.io/v1\r\nkind: ResourceClaimTemplate\r\nmetadata:\r\n name: all-mrdma\r\nspec:\r\n spec:\r\n devices:\r\n requests:\r\n - name: req-mrdma\r\n exactly:\r\n deviceClassName: mrdma.google.com\r\n allocationMode: All'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f50781d17f0>)])]>

Deploy model and inference

Now that a cluster and node pool is setup, we can deploy a model and serve it via Inference gateway. In my experiment I used DeepSeek but this could be any model.

Deploy model and services

The nodeSelector: gke.networks.io/accelerator-network-profile: auto is used to assign to the GPU node
The resourceClaims: attaches the resource we defined for networking

Create a secret (I used Hugging Face token):

code_block: <ListValue: [StructValue([('code', 'kubectl create secret generic hf-secret \\\r\n --from-literal=hf_token=${HF_TOKEN}'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f50781d16a0>)])]>

Deployment

code_block: <ListValue: [StructValue([('code', 'apiVersion: apps/v1\r\nkind: Deployment\r\nmetadata:\r\n name: deepseek-v3-1-deploy\r\nspec:\r\n replicas: 1\r\n selector:\r\n matchLabels:\r\n app: deepseek-v3-1\r\n template:\r\n metadata:\r\n labels:\r\n app: deepseek-v3-1\r\n ai.gke.io/model: deepseek-v3-1\r\n ai.gke.io/inference-server: vllm\r\n examples.ai.gke.io/source: user-guide\r\n spec:\r\n containers:\r\n - name: vllm-inference\r\n image: us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/pytorch-vllm-serve:20250819_0916_RC01\r\n resources:\r\n requests:\r\n cpu: "190"\r\n memory: "1800Gi"\r\n ephemeral-storage: "1Ti"\r\n nvidia.com/gpu: "8"\r\n limits:\r\n cpu: "190"\r\n memory: "1800Gi"\r\n ephemeral-storage: "1Ti"\r\n nvidia.com/gpu: "8"\r\n claims:\r\n - name: rdma-claim\r\n command: ["python3", "-m", "vllm.entrypoints.openai.api_server"]\r\n args:\r\n - --model=$(MODEL_ID)\r\n - --tensor-parallel-size=8\r\n - --host=0.0.0.0\r\n - --port=8000\r\n - --max-model-len=32768\r\n - --max-num-seqs=32\r\n - --gpu-memory-utilization=0.90\r\n - --enable-chunked-prefill\r\n - --enforce-eager\r\n - --trust-remote-code\r\n env:\r\n - name: MODEL_ID\r\n value: deepseek-ai/DeepSeek-V3.1\r\n - name: HUGGING_FACE_HUB_TOKEN\r\n valueFrom:\r\n secretKeyRef:\r\n name: hf-secret\r\n key: hf_token\r\n volumeMounts:\r\n - mountPath: /dev/shm\r\n name: dshm\r\n livenessProbe:\r\n httpGet:\r\n path: /health\r\n port: 8000\r\n initialDelaySeconds: 1800\r\n periodSeconds: 10\r\n readinessProbe:\r\n httpGet:\r\n path: /health\r\n port: 8000\r\n initialDelaySeconds: 1800\r\n periodSeconds: 5\r\n volumes:\r\n - name: dshm\r\n emptyDir:\r\n medium: Memory\r\n nodeSelector:\r\n gke.networks.io/accelerator-network-profile: auto\r\n resourceClaims:\r\n - name: rdma-claim\r\n resourceClaimTemplateName: all-mrdma\r\n---\r\napiVersion: v1\r\nkind: Service\r\nmetadata:\r\n name: deepseek-v3-1-service\r\nspec:\r\n selector:\r\n app: deepseek-v3-1\r\n type: ClusterIP\r\n ports:\r\n - protocol: TCP\r\n port: 8000\r\n targetPort: 8000\r\n---\r\napiVersion: monitoring.googleapis.com/v1\r\nkind: PodMonitoring\r\nmetadata:\r\n name: deepseek-v3-1-monitoring\r\nspec:\r\n selector:\r\n matchLabels:\r\n app: deepseek-v3-1\r\n endpoints:\r\n - port: 8000\r\n path: /metrics\r\n interval: 30s'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f50781d1550>)])]>

Deploy GKE Inference Gateway

This install needed Custom Resource Definitions (CRDs) in your GKE cluster:

For GKE versions 1.34.0-gke.1626000 or later, install only the alpha InferenceObjective CRD:

code_block: <ListValue: [StructValue([('code', 'kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/raw/v1.0.0/config/crd/bases/inference.networking.x-k8s.io_inferenceobjectives.yaml'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f50781d1460>)])]>

Create Inference pool

code_block: <ListValue: [StructValue([('code', 'helm install deepseek-v3-pool \\\r\n oci://registry.k8s.io/gateway-api-inference-extension/charts/inferencepool \\\r\n --version v1.0.1 \\\r\n --set inferencePool.modelServers.matchLabels.app=deepseek-v3-1 \\\r\n --set provider.name=gke \\\r\n --set inferenceExtension.monitoring.gke.enabled=true'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f50781d18b0>)])]>

Create the Gateway, HTTPRoute and InferenceObjective

code_block: <ListValue: [StructValue([('code', '# 1. The Regional Internal Gateway (ILB)\r\napiVersion: gateway.networking.k8s.io/v1\r\nkind: Gateway\r\nmetadata:\r\n name: deepseek-v3-gateway\r\n namespace: default\r\nspec:\r\n gatewayClassName: gke-l7-rilb\r\n listeners:\r\n - name: http\r\n protocol: HTTP\r\n port: 80\r\n allowedRoutes:\r\n namespaces:\r\n from: Same\r\n---\r\n# 2. The HTTPRoute (Routing to the Pool)\r\napiVersion: gateway.networking.k8s.io/v1\r\nkind: HTTPRoute\r\nmetadata:\r\n name: deepseek-v3-route\r\n namespace: default\r\nspec:\r\n parentRefs:\r\n - name: deepseek-v3-gateway\r\n rules:\r\n - matches:\r\n - path:\r\n type: PathPrefix\r\n value: /\r\n backendRefs:\r\n - group: inference.networking.k8s.io\r\n kind: InferencePool\r\n name: deepseek-v3-pool\r\n---\r\n# 3. The Inference Objective (Performance Logic)\r\napiVersion: inference.networking.x-k8s.io/v1alpha2\r\nkind: InferenceObjective\r\nmetadata:\r\n name: deepseek-v3-objective\r\n namespace: default\r\nspec:\r\n poolRef:\r\n name: deepseek-v3-pool'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f50781d1a90>)])]>

Once complete, you can create a test VM in your main VPC and make a call to the IP address of the GKE Inference Gateway:

code_block: <ListValue: [StructValue([('code', 'curl -N -s -X POST "http://$GATEWAY_IP/v1/chat/completions" \\\r\n -H "Content-Type: application/json" \\\r\n -d \'{\r\n "model": "deepseek-ai/DeepSeek-V3.1",\r\n "messages": [{"role": "user", "content": "Box A: red. Box B: blue. Box C: empty. Move A to C, Move B to A, Swap B and C. Where is red?"}],\r\n "stream": true\r\n }\' | stdbuf -oL grep "data: " | sed -u \'s/^data: //\' | grep -v "\\[DONE\\]" | \\\r\n jq --unbuffered -rj \'.choices[0].delta | (.reasoning_content // .reasoning // .content // empty)\''), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f50781d1b20>)])]>

Next Steps

Take a deeper dive into GKE managed DRANET and GKE Inference Gateway, review the following.

Blog: DRA: A new era of Kubernetes device management with Dynamic Resource Allocation
Document set: DRANET
Documentation: AI Hypercomputer

Want to ask a question, find out more or share a thought? Please connect with me on Linkedin.

Claude Mythos Preview: Available in private preview on Vertex AI

Tue, 07 Apr 2026 18:00:00 +0000

Claude Mythos Preview, Anthropic’s newest and most powerful model, is now available in Private Preview to a select group of Google Cloud customers, as part of Project Glasswing.

The availability of Claude Mythos Preview on Vertex AI underscores our commitment to offer our customers access to models from frontier AI labs. Combined with the enterprise-grade power of Vertex AI to build, scale, and govern AI applications and agents, this new general-purpose model offers high performance capabilities across a variety of use cases, with new focus on reducing cybersecurity risk.

For more information about this release, visit Anthropic’s blog.

Build with other Claude models — including Claude Opus 4.6 and Claude Sonnet 4.6—today on Vertex AI.

See beyond the IP and secure URLs with Google Cloud NGFW

Tue, 07 Apr 2026 17:30:00 +0000

In a cloud-first world, traditional IP-based defenses are no longer enough to protect your perimeter. As services migrate to shared infrastructure and content delivery networks, relying on static IP addresses and FQDNs can create security gaps.

Because single IP addresses can host multiple services, and IPs addresses can change frequently, we are introducing domain filtering with a wildcard capability in Cloud Next Generation Firewall (NGFW) Enterprise. This new capability provides increased security and granular policy controls.

Why domain and SNI filtering matters

The Cloud NGFW URL filtering service performs deep inspections of HTTP payloads to secure workloads against threats from both public and internal networks. This service elevates security controls to the application layer and helps restrict access to malicious domains.

Key use cases include:

Granular egress control: This capability enables the precise allowing and blocking of connections based on domain names and SNI information found in egress HTTP(S) messages. By inspecting Layer 7 (L7) headers, it offers significantly finer control than traditional filtering based solely on IP addresses and FQDNs, which can be inefficient when a single IP hosts multiple services.
Control access without decrypting: For organizations that prefer not to perform full TLS decryption on their traffic, Cloud NGFW can still enforce security policies by controlling traffic based on SNI headers provided during the TLS handshake. This allows for effective domain-level filtering while maintaining end-to-end encryption for privacy or compliance reasons.
Reduced operational overhead: Implementing domain-based filtering helps reduce the constant maintenance typically required to track frequently changing IP addresses and DNS records. By focusing on stable domain identities rather than dynamic network attributes, security teams can minimize the manual effort involved in updating firewall rulebases.
Flexible matching: The service utilizes matcher strings within URL lists, supporting limited wildcard domains to define criteria for both domains and subdomains. For example, using a wildcard like *.example.com allows a single filter to cover all associated subdomains, providing a more scalable solution than defining thousands of individual FQDN entries.
Improved security: URL filtering significantly enhances the security posture by protecting against sophisticated flaws like SNI header spoofing. By evaluating L7 headers before allowing access to an application, Cloud NGFW ensures that attackers cannot bypass security controls by simply spoofing lower-layer identifiers.

How Cloud NGFW URL filtering works

The URL filtering service functions by inspecting traffic at L7 using a distributed architecture.

Cloud NGFW URL filtering service

You can get started with URL filtering in three simple steps.

Deploy Cloud NGFW endpoints:

The first step is to create and deploy a Cloud NGFW endpoint in a zone. The NGFW endpoint is an organization level resource. Please ensure you have the right permission before deploying the endpoint.
Once the endpoint is deployed you can associate it to one or more VPCs of your choice.

Create security profiles and security profile groups:

The URL filtering security profile holds the URL filters with matcher strings and an action (allow or deny).
The security profile group acts as a container for these security profiles, which is then referenced by a firewall policy rule. Create URL filtering security profiles with desired URLs, wildcard FQDNs and add them to a security profile group.
Once the security profile group is created, you will need to reference the security profile group in firewall policies.

Policy enforcement:

You enable the service by configuring a hierarchical or global network firewall policy rule using the apply_security_profile_group action, specifying the name of your security profile group.

For more information about configuring a firewall policy rule, see the following:

Getting started

Get started with Cloud NGFW URL filtering by visiting our documentation and codelab.

Ultimate prompting guide for Lyria 3 models

Tue, 07 Apr 2026 16:00:00 +0000

Lyria 3, Google's family of music generation models, is designed to give you granular control over vocals, instrumentation, and arrangement. So we spent weeks testing against every musical genre and use case we could imagine.

We put together this guide to share exactly what we learned and how you can get the best results.

What you'll learn in this guide:

Model overview
Breakdown of tech specs
Best practices for effective prompting
The core prompting framework
Mastering vocals and lyrics
Advanced creative workflows
How Lyria 3 models work with other generative media models

Model overview

Lyria 3 and Lyria 3 Pro are music generation models designed to support your creative workflows. The models excel in three key areas:

Structural control: Prompt for specific elements like intros, verses, choruses, and bridges to build a complete arrangement.
High-quality audio: Both models deliver high-fidelity stereo audio
Precision control: Dictate structural changes using timed lyrics, descriptive tempo conditioning, and multimodal inputs.

Breakdown of tech specs for Lyria 3 and Lyria 3 Pro

Here is a breakdown of what the models can handle via the API on Vertex AI:

Track length: Lyria 3 generates 30-second long songs, ideal for rapid prototyping and short-form assets. Lyria 3 Pro supports compositions up to three minutes long.
Vocal support: Both models feature improved realism and expressiveness for vocals, supporting multi-vocal conditioning and generation in eight languages (English, German, Spanish, French, Hindi, Japanese, Korean, and Portuguese).
Controls and conditioning: Lyria 3 Pro includes advanced controls for timed lyrics and tempo control through natural language descriptions.
Multimodal inputs: You can generate music using text, PDF files, or up to 10 reference images.
Trust and safety: All outputs include SynthID watermarking and support the C2PA open standard for cryptographically signed metadata.

For more, visit Lyria 3 models card.

Best practices for effective prompting

There are a few guidelines to ensure your generated audio matches your intent:

Be descriptive and specific: Use adjectives to create a clear description. The more detail you provide, the better Lyria understands your prompt.
Reference genres and eras: Clearly state the musical category (for example, Rock or Pop) and stylistic timeframe (e.g. the 1950s, early 90s).
Specify key instruments: Mention the important instruments driving the track, or Lyria chooses defaults based on the genre.
Iterate: If the first result isn't perfect, refine your prompt by adjusting keywords.

The core prompting framework

A simple list of keywords will generate great songs, but to control the models, use this framework.

[Genre and style] + [Mood] + [Instrumentation] + [Tempo and rhythm] + [Vocal style & language] + [Lyrics]

Genre and style: Define the primary category, for example, "cinematic orchestral fantasy".
Mood: Describe the emotional intent, for example, "tense and suspenseful".
Instrumentation: Name the specific instruments, for example, "guitar", "piano".
Tempo and rhythm: Set the speed, pace, and groove using descriptive terms, such as, "a fast, energetic pace with a driving beat".
Instrumental vs. vocal: Specify "instrumental" to exclude vocals.
Vocal style & language: Specify gender, tone (e.g., raspy, smooth), delivery (e.g. rapping), and language.
Lyrics: Either provide a theme for Lyria to generate the words (e.g., "song about a cross-cultural connection"), or provide your exact lyrics in quotes for the model to perform.

Example prompt: “A romantic fusion of classic Bossa Nova and modern R&B. The mood is intimate, warm, and deeply affectionate. Features a gentle acoustic nylon-string guitar, warm electric piano chords, and a crisp, laid-back modern hip-hop drum beat. A slow, swaying tempo. Featuring a vocal duet: a smooth male vocalist singing in English, and a soft, breathy female vocalist singing in French. The lyrics are a beautiful love song about an undeniable, cross-cultural connection”

If you want instrumental only songs, write in the prompt “instrumental”.

Prompt example: “A warm, modern lofi hip-hop beat for studying, featuring a muffled drum break and dusty jazz piano samples. Instrumental.”

Mastering vocals and lyrics

Lyria 3 models give you control over both the lyrics and the vocal performance.

Incorporating specific lyrics

Syntax for lyrics: To use your own lyrics, write the "Lyrics:" before the lines you want the model to sing.
Backing vocals: If you want backing singers to echo the main vocals, mention where you want the backing vocals in the prompt.
Lyrics: If you prefer the model to write the lyrics for you, clearly describe the theme in your prompt, such as asking for "a love song" or a "new happy birthday song", or provide your lyrics to the model.

Example prompt: “A smooth, moody jazz ballad featuring piano and upright bass. The vocals should be a female singer with a breathy, soulful soprano range. The vocal pattern should start out confident but get calmer and quieter as the track progresses. Song lyrics about meeting the love of her life in New York.”

Controlling the voice

Define the desired vocal style in detail to get the performance you want:

Singer demographics and range: Specify whether you want a male or female singer, and dictate their vocal range. For example, you can ask for "commanding baritone vocals" or a "clear and high soprano range."
Voice texture (timbre): Describe the texture of the voice, you can ask for vocals that are "gravelly," "soulful," or "breathy."
Vocal patterns and styles: Describe the specific vocal pattern you want to hear, such as a "fast-paced" or "laid-back" groove. You can also experiment with layering different vocal styles or having the vocals change dynamically, such as getting "calmer and quieter as the track progresses."
Language: You can specify the language you want the vocals to be sung in. The model supports multi-vocal generation in eight languages: English, German, Spanish, French, Hindi, Japanese, Korean, and Portuguese (more languages coming soon).

Example prompt: “An upbeat, high-energy J-pop track with bright, sparkling synths, electric guitar, and a driving bassline. Featuring a clear, expressive male tenor vocal singing in Japanese. The vocal style should be fast-paced and melodic, with a sweet and highly polished texture.”

Advanced creative workflows

Workflow 1: Timestamp prompting

This workflow is ideal for creating dynamic genre shifts or scoring video content by assigning actions to timed segments.

Prompt example:

[00:00] Begin immediately with a massive gospel choir singing a powerful, uplifting harmony about being kind to yourself.

[00:15] A heavy, modern hip-hop drum beat and a deep 808 bassline drop in, matching the energy of the choir.

[00:30] A male lead vocalist begins rapping a confident verse about overcoming life's challenges, while the large choir punctuates his lines in the background.

[01:10] Transition into a huge, triumphant chorus celebrating victory and winning. The gospel choir sings at full volume, layering rich, soulful harmonies over the driving hip-hop beat and triumphant brass horns.

[01:50] The beat strips back to just a gentle Hammond B3 organ. The rapper delivers a quiet, emotional bridge about giving yourself grace, supported by soft, warm hums from the massive choir.

[02:10] The full hip-hop beat and the giant choir return at maximum energy for an uplifting final chorus, before ending on a resonant, sustained choir chord at [03:00].

Workflow 2: Multimodal generation

Lyria 3 models allow you to upload reference images or PDFs to establish the emotional baseline for the track.

Prompt example: “A deeply emotional, modern Bollywood song in English. The lyrics and mood should match the story in the images attached.”

Go further

Lyria 3 and Lyria 3 Pro can be used with our other generative media models on Vertex AI.

Lyria + Veo: Generate video assets using Veo, and then dictate the exact structural timing in Lyria 3 Pro to score a custom soundtrack that matches every scene transition.
Lyria + Nano Banana: Generate images of a storyboard or vibe, and let Lyria create a song based on those images.
Lyria + Gemini: If you are struggling to define your desired sound, use Gemini to analyze your creative brief and output a highly descriptive prompt to feed into Lyria 3 models. Gemini can also create lyrics for you based on your creative brief.
Lyria + Agents: If you’re using these models with our GenMedia MCP tools, you can provide domain-specific sound design knowledge via this Agent Skill.

To get started, access the Lyria 3 models today via the API documentation, Gen AI SDK for Python notebook, and in our playground Vertex AI Media Studio.

_{Thanks to Khulan Davaajav, Russ Khaimov, and Sandeep Gupta for their contributions to prompting guidance for customers.}

Under one roof: Rightmove reinvents property search with unified data

Tue, 07 Apr 2026 16:00:00 +0000

At Rightmove, we want to make home moving easier for everyone, from house hunters and homeowners to estate agents and brokers. Behind every search, listing, and connection on our platform lies a complex network of users, partners, and properties — and we’ve built our data and AI strategy to serve all three.

To deliver on this mission, we migrated from siloed, on-premises databases to Google Cloud. This move wasn’t just about technology. It was about unlocking smarter, faster, more personalized experiences for our users and partners, and helping them find the right match for each property more efficiently.

Our strategy is guided by four core data and AI value areas:

Delighting consumers with personalized search and discovery
Empowering partners, such as estate agents, with smarter tools and insights
Monetizing data through innovations such as property price prediction
Driving operational efficiency across our platform

Today, we're building this future with a unified analytics and AI stack — BigQuery, Vertex AI, and Looker — that we call “the data hive.” Already, around 300 team members (a third of our workforce) are tapping into its capabilities to turn data into action and insights into impact.

Making it easier to find a home with personalized, dynamic suggestions

When someone’s looking for a new home, they often have a wish list: a garden, a modern kitchen, maybe a home office. We’re using Vertex AI to make that search feel more intuitive and tailored than ever. By extracting metadata from property descriptions and images, we automatically create listing features and keywords, even ones that weren’t manually tagged before, to provide more accurate search results.

We’re also exploring ways to streamline communication. Recently released is an AI-powered feature that uses Gemini to help estate agents respond to inquiries faster. With context-aware, automatically generated replies, agents can keep conversations moving, and potential buyers and sellers can get answers faster, even during the busiest periods.

Helping partners work smarter with AI-powered recommendations

Our partners — which include estate agents,new home developers, mortgage lenders, and other industry professionals — rely on Rightmove to connect with the right audience at the right time. With Vertex AI and Gemini models working behind the scenes, we’re helping them do that more efficiently and effectively.

Take lead generation, for example. We’ve built a vendor scoring engine that analyzes user search patterns and on-site behavior to predict the likelihood that someone is a homeowner. This insight helps partners focus their time and marketing efforts on high-conversion leads, while offering more relevant products — such as mortgage options — to the right people at the right moment.

Next, we’re excited to use generative AI to build agentic, conversational user interfaces, enabling anyone across our network to interact with data or find insights using natural language. Whether it's a business user running a query, or a partner navigating market trends, we’re working toward a more natural, accessible way to engage with data.

Turning data into insight and insight into value

Another exciting way we’re unlocking the value of our data is through an Automated Valuation Model (AVM). This AI-powered tool predicts the sale and rental price of every property in the UK, every month, by analyzing a wide range of signals including market trends, supply and demand, and the condition of individual homes.

Traditionally, valuations were fairly static, based on fixed data points that didn’t reflect recent improvements or shifting market conditions. Vertex AI makes them dynamic. Whether it’s a newly renovated kitchen or a shift in local market conditions, we can factor in real-time changes to properties on our website, delivering more accurate, up-to-date valuations for both homeowners and estate agents.

These monthly valuations are invaluable to our partners. Mortgage lenders, and estate agents use them as trusted pricing guides to understand local markets and assess risk, especially when managing large property portfolios or backbooks.

Behind the scenes, the hive gives us access to both structured and unstructured data, including more than 25 years of property images that were previously siloed. Now stored securely in Cloud Storage, this rich visual data is fueling advanced use cases, including these enhanced valuation models and deeper market analysis.

Driving operational efficiency with a smarter, unified data platform

With our migration to the cloud, we’ve embraced a “hub and spoke” model to ensure both consistency and flexibility in how data and AI are used across the business. The “hubs” are our central teams — experts in BigQuery, Looker, and Vertex AI — who set best practices and help scale innovation. The “spokes” are our vertical business units, such as the New Homes department, that tap into the hive platform to run their own business intelligence and AI use cases, tailored to their specific needs.

By consolidating multiple legacy business intelligence tools into a single platform with Looker, we’ve simplified our tech stack and created operational gains. For example, the New Homes team has cut down meeting prep with developers from hours to minutes, thanks to easily accessible, self-serve Looker dashboards.

And we’re constantly discovering new ways to create value from our new platform. For example, as Google Cloud rolls out new features such as BigQuery’s Series FM function for low-code forecasting, our teams are quickly adopting them to move from descriptive to predictive analytics. Forecasting leads, time spent on site, or whatever KPI a business unit has, was previously unthinkable, with siloed data and manual processes for developing models and ingesting data somewhere else for analysis. In our new platform, we can quickly trial this kind of forecasting in a spoke using just 10 lines of code and our BigQuery-Looker integration. It only took weeks for many of our business units to start using Series FM for forecasting.

Helping users at every stage of homeownership with AI

As we look to the future, our AI strategy is expanding beyond helping people find a home. Rightmove is evolving into a smart, supportive companion for every stage of the home journey: find, afford, transact, move, and live.

That means using data and AI to continue helping users to find properties but also to make better-informed decisions throughout the entire lifecycle of homeownership after that. We’re already rolling out new capabilities, such as smarter mortgage in principle matching and insights into the total cost of ownership, including broadband, energy, and utility costs, so buyers can move with confidence.

One exciting step in this direction is using Vertex AI to power the models and data behind our Track a Property feature — a way for home-owners to regularly check the value of their home. This upgrade means the valuation models are built and trained faster, will improve their accuracy over the long term with added model engineering and tuning, as well as taking advantage of better cloud architecture to host them.

This is just the beginning. As Google AI continues to evolve, so does our platform, becoming a home and living assistant that supports not just the move, but the life that follows it.

Build music generation into your apps with Lyria 3 models on Vertex AI

Tue, 07 Apr 2026 16:00:00 +0000

What’s new: Lyria 3, Google's family of music generation models, is available on Vertex AI in public preview. With Lyria 3 models, you can generate high-quality and high-fidelity stereo audio from text prompts and from images with a vocal support.

To meet your diverse needs, we offer two distinct models via the API:

Lyria 3 Pro: Generates complete compositions up to three minutes long. It understands musical architecture, allowing for specific structural elements like intros, verses, choruses, and bridges.
Lyria 3: Generates tracks up to 30 seconds long. It’s ideal for rapid prototyping, social media assets, and short-form audio generation.

Why this matters for your business: These models deliver structural coherence, including vocals, timed lyrics, and full instrumental arrangements. You can build studio-quality audio production directly into your apps. With Lyria 3 models, you can create:

Multi-modal input: Generate audio using standard text prompts or by using reference images to guide the story, mood and style.
Vocal & lyrics: Generate vocals and timed lyrics, or use user-provided lyrics to guide the track. Need a pure soundbed? Write instrumental in the prompt.
Flexible compositions: Utilize duration controls to generate full songs with distinct intros, verses, choruses, and bridges.

Where you can access Lyria: Access the models via the Vertex AI API and Vertex AI Media Studio.

How Lyria 3 models are helping customers bring their audio experiences to life

“Artlist is built on profound music expertise and a commitment to setting the global sound trends that shape the digital world. Our extensive experience allows us to utilize Lyria 3 at its highest potential - combining our 'human-in-the-loop' musicology with Google’s most advanced generative music model to date. Lyria 3 provides an unprecedented level of creative control and high-fidelity output, resulting in a powerful fusion of technical innovation and the authentic sound that defines the Artlist brand.” - Roee Peled, Chief Product and Technology Officer, Artlist

"With Lyria 3, we're seeing a clear shift from pure generation to true creative control — enabling Freepik's clients to produce music that aligns more precisely with their vision and workflows. What stands out in Lyria 3 is the level of control it brings — reducing iteration time and making music generation far more predictable and usable in real-world creative pipelines" - Carlos Perez, Engineering Lead, Freepik

Commercial safety and responsible creation

Responsibility is foundational, and remains integral in the design and training of Lyria 3 models, using materials that YouTube and Google has a right to use under our terms of service, partner agreements, and applicable law. Additionally, we employ filters to check outputs against existing content and users must adhere to the Terms of Service and Gen AI prohibited use policies, which prohibit violating others' intellectual property and privacy rights. All Lyria 3 and Lyria 3 Pro outputs are embedded with SynthID watermarking and support C2PA.

Start building today

To dive deeper into the best practices, explore the following resources:

Model card
Ultimate prompting guide for Lyria
Gen AI SDK for Python notebook
If you’re using these models with our Gen Media MCP tools, you can provide domain-specific sound design knowledge via this Agent Skill

If you're a Google Workspace customer or Google AI subscriber, you can also experience Lyria 3 models in Google Vids to give your videos a custom track that matches your brand’s style.

AI infrastructure efficiency: Ironwood TPUs deliver 3.7x carbon efficiency gains

Mon, 06 Apr 2026 16:00:00 +0000

At Google, we are committed to being transparent about the environmental impact of our AI infrastructure, publishing metrics on the lifetime emissions of our chips — from manufacturing to powering these chips in the data center. Today, we are updating these metrics for our seventh-generation TPU, Ironwood, which demonstrates an approximately 3.7x improvement in Compute Carbon Intensity (CCI) compared to TPU v5p, the previous generation of performance-optimized TPUs.¹

In other words, despite the fact that AI is driving demand for additional compute resources, our ongoing work to optimize AI hardware is helping to improve the energy consumption and emissions of AI workloads.

Measuring AI accelerator efficiency: Compute Carbon Intensity (CCI)

To help manage the environmental impact of AI workloads, we monitor the Compute Carbon Intensity (CCI) of our AI accelerator hardware. CCI is defined in An Introduction to Life-Cycle Emissions of Artificial Intelligence Hardware²as the estimated amount of CO2 equivalent emitted for every utilized floating-point operation (CO2e/FLOP). This metric provides a holistic, chip-level view by including both the embodied emissions associated with manufacturing, transportation, and data center construction (Scope 3), as well as the operational emissions associated with running these chips in data centers (Scope 1 and 2).

The Ironwood advantage: high performance, low footprint

Google’s TPU CCI continues to improve with each chip generation. Drawing from empirical data measured in January 2026, Ironwood demonstrates a remarkable 3.7x improvement in CCI relative to TPU v5p. This accelerates efficiency gains from the 1.2x CCI improvement of TPU v5p relative to TPU v4, and demonstrates continued carbon efficiency optimization of Google’s performance-optimized TPU architecture.

These efficiency gains are driven by outsized compute performance increases between TPU generations relative to growth in machine energy consumption and manufacturing emissions. In fact, fleetwide measurements demonstrate a 5x improvement in utilized FLOPs across generations, from TPU v5p to Ironwood.³ Because the performance denominator in our CCI equation (CO2e/FLOP) is scaling faster than emissions, the net carbon cost per operation drops significantly with every new chip.

^{Figure 1: Ironwood’s accelerating CCI improvement measured on Google’s performance-optimized TPU cohort, considering January 2026 workloads.4}

Operating Google’s TPU fleet more efficiently

Updated TPU CCI metrics also offer a direct comparison to the measurement we published in 2025. Specifically, from October 2024 to January 2026, Google’s versatile TPU cohort ran more efficiently than what we reported previously:

TPU v5e achieved a 43% reduction in total CCI over 15 months, dropping to 228 gCO2e/EFLOP. This was driven by a 72% increase in average utilization.
Trillium, the sixth-generation TPU, saw a 20% reduction in total CCI over the same time period, bringing its emissions intensity down to 125 gCO2e/EFLOP.

^{Figure 2: Google’s versatile TPU cohort demonstrates deployment efficiency gains for the same TPU generations between October 2024 and January 2026.5}

These results demonstrate that Google continues to improve the carbon-efficiency of our AI infrastructure. While the massive scale of AI demand requires a significant and growing amount of power, our innovations allow us to deliver substantially more compute performance for every unit of energy consumed.

Decoupling energy and emissions from performance

To what can we attribute these improvements? Beyond Ironwood’s raw hardware capabilities, these CCI gains are further enabled by deep software and system-level optimizations across our infrastructure:

Software efficiency (MoE): The widespread adoption of sparse architectures, such as Mixture of Experts (MoE), routes computation only to necessary parameters. This drastically reduces the active FLOPs required per inference or training step without sacrificing model capacity or quality.
Lower precision math (FP8): By heavily leveraging 8-bit floating-point (FP8) formats, we effectively double compute throughput and halve memory bandwidth requirements compared to 16-bit formats. This shows that we can maintain output quality while exponentially decreasing the energy cost per mathematical operation.
Workload mix and intelligent scheduling: Advanced fleet orchestration continuously balances the workload mix across our infrastructure. By intelligently scheduling tasks, we ensure high continuous utilization rates, optimize duty cycles, and minimize the carbon penalty of idle power draw.

Scale sustainably with Google Cloud

AI’s trajectory requires infrastructure that can scale exponentially without an equivalent surge in carbon emissions. The 3.7x carbon efficiency improvement from TPU v5p to Ironwood demonstrates that we can achieve greater compute density while minimizing the growth of our energy and environmental footprint through deliberate hardware and software codesign. To learn more and get started with Ironwood, register your interest with this form.

_{1. Following the methodology published in an August 2025 technical report, we quantified the full lifecycle emissions of TPU hardware as a point-in-time snapshot across Google’s generations of TPUs as of January 2026. The functional unit for this study is one AI computer deployed in the data center, which includes one or more accelerator trays (containing TPUs) connected to one host tray (i.e., a computing server). Peripheral components beyond the tray (e.g., rack, shelf, and network equipment) and auxiliary computing and storage resources are excluded from the calculation of embodied and operational emissions. We include the electricity used in data center cooling in operational emissions. To estimate operational emissions from electricity consumption of running workloads, we used a one month sample of observed machine power data from our entire TPU fleet, applying Google’s 2024 average fleetwide carbon intensity. To estimate embodied emissions from manufacturing, transportation, and retirement, we performed a life-cycle assessment of the hardware. Data center construction emissions were estimated based on Google’s disclosed 2024 carbon footprint. These findings do not represent model-level emissions, nor are they a complete quantification of Google’s AI emissions. Based on the TPU location of a specific workload, CCI results of specific workloads may vary.
2. The authors would like to thank and acknowledge the co-authors of this paper for their important contributions to enable these results: Ian Schneider, Hui Xu, Stephan Benecke, Parthasarathy Ranganathan, and Cooper Elsworth.
3. This comparison considers the utilized FLOPS (BF16) between deployed TPU v5p and Ironwood chips in Google’s fleet in January 2026. This trend is consistent with the improvement in peak FLOPS (BF16) between v5p (459 FLOPS) and Ironwood (2,307 FLOPS).
4.The GHG protocol offers two accounting standards for operational emissions. Results presented here consider market-based emissions, which includes the impact of carbon-free energy purchases. Location-based accounting, which excludes carbon-free energy purchases, would raise operational CCI to 793, 712, and 195 gCO2e/EFLOP, respectively. The ratio of CCI improvements would be at a similar level, and Ironwood’s embodied CCI would drop from 23% to 8% of its total CCI.
5. To ensure a fair comparison across varying TPU utilizations, this analysis replicates the propensity score weighting methodology from the August 2025 technical report and compares January 2026 results to the results published in 2025. This statistical technique adjusts for duty cycle variations to balance the comparison of TPUs during a given time period. This empirical methodology results in small variations in calculated CCI between temporal periods, reflecting fluctuations in real-world energy consumption and hardware utilization across the global infrastructure.}

How a leading consumer insight brand uses Dataproc to hyper-personalise faster

Mon, 06 Apr 2026 16:00:00 +0000

At RVU, we have a clear and vital mission: empower people, transform industries.

For our market-leading home management and switching brands — Confused.com, Uswitch, Tempcover, Money.co.uk, and Mojo Mortgages — transparency and accurate information are everything. Today’s consumer expects more than a simple comparison table; they want personalized recommendations tailored to their unique circumstances.

Delivering on that promise — building a true personalization engine that powers all our brands — requires a data foundation capable of processing massive, complex datasets for sophisticated ML models. Today, our platform powers hundreds of automated personalization campaigns, optimized with billions of data points from across all our brands. We tackled this challenge using the power of Google Cloud and its two solutions for Apache Spark: Dataproc and Google Cloud Serverless for Apache Spark. Together, we’re making our mission a reality.

The high-speed engine for feature engineering

Our relationship with Google Cloud isn’t new. In fact, we've used BigQuery as our unified data platform for over a decade. Coming from a performance marketing background, we’ve always dealt with a lot of data, but we recognized early on that we’re not a digital infrastructure company. Instead, our focus must always be on where the value is. Managed solutions like BigQuery that eliminate infrastructure and capacity headaches were a natural fit from the start.

The key challenge was stitching together a meaningful and coherent picture of customer behavior across our brands — turning countless fragmented interactions into something that genuinely reflects how a user behaves, clicks, and makes decisions. Instead of relying on isolated events and aggregate views, we’ve had to build a platform capable of connecting these signals into a narrative that works for our machine learning models.

Using Dataproc to support this was a gamechanger. The biggest impact has been its role as our core high-speed Spark processing engine, primarily for feature engineering for our ML model development. Feature engineering, which is the crucial process of shaping all that raw customer data for our data science models, is a real value-driver for us. It’s where we have a marked competitive edge.

The result has been a significant leap in our innovation velocity. With Serverless for Apache Spark, we now have the ability to shape our customer data for feature engineering in just a matter of days. Previously, this would have taken weeks. We’ve also dramatically reduced our time-to-market, which also used to take weeks. Now, a new contractor can join the team and deliver a model, including exploratory data analysis and all feature engineering, in only a week and a half. That’s incredibly fast.

Delivering personalized experiences

By improving our speed of innovation, we’re better positioned to deliver a personalized user experience to our customers and partners.

Our hyper-personalization journey accelerated once we moved to Spark. We can now run heavyweight data processing jobs that crunch vast amounts of behavioral and contextual data, allowing us to build models that generate genuinely meaningful predictions.

These models help us understand not just what to say to a customer, but when and how to say it — selecting the right moment and the right channel to deliver personalized insight that genuinely resonates.

Building a future vision

Google’s Data Cloud directly aligns with our culture of prioritizing value, and its impact on our business is profound. I call it the network effect, where everything seamlessly connects within the same ecosystem: Our data resides in BigQuery, our ability to validate, enrich, and transform that data is tied to Dataproc and Serverless for Apache Spark, and our capacity to deploy the ML models spans the network. It’s all co-located and integrated, powering the real-time accuracy of our consumer brands and giving us a competitive advantage.

For our engineers, the big win is the lack of infrastructure they have to deal with. They can press a button that processes all the data in 10 minutes, rather than having to set up a network of clusters and servers and make them talk to each other. It’s incredibly efficient and frees up time for more valuable work like building and iterating data products.

Dataproc has upped our speed, scale, and agility. It also gives us the tools to innovate with AI as we build the future of hyper-personalization. Today, we’re proud to say RVU’s cutting-edge tech and data are helping millions of UK consumers make smarter, more informed decisions, and truly transforming industries.

Inspired by RVU's success? Whether you need persistent clusters with Dataproc or the agility of Serverless Spark, Google Cloud has a managed solution to help you focus on value, not infrastructure. Discover the right Spark on Google Cloud for your use case.

Introducing Looker self-service Explores for faster ad-hoc analysis

Mon, 06 Apr 2026 16:00:00 +0000

By design, Looker is the enterprise semantic platform which ensures that every data set meets a high standard of accuracy by acting as a single source of truth and providing long-term consistency of your metrics. Today, we are introducing a complement to this governed framework: self-service Explores, to accelerate high-velocity, ad-hoc analysis. Self-service Explores allows you to bring your own data directly into the Looker semantic layer, providing instant access to insights while maintaining the integrity of your existing governed data ecosystem.

Data teams often find themselves caught between two worlds. On one side, there’s the trusted, governed world of modern BI, where every metric is defined and every row is verified. Then there’s the agile, anything goes nature of spreadsheets and CSV files where you can get answers in seconds but run the risk of ending up in a siloed data vacuum.

Self-service Explores bring the value of modern, governed BI to the experimentation self-starting capability of spreadsheets, allowing anyone with the right permissions to turn a flat file into a fully functional Looker Explore in seconds. You can also import from Cloud from Google Drive and quickly transform Google Sheets data into conversational analytics. No code, no waiting — just insights.

You can drag and drop a comma separated or spreadsheet file (.csv, .xls, or .xlsx) or pull directly from Google Sheets, and Looker automatically creates an Explore. Behind the scenes, these files are securely stored in your own BigQuery instance, ensuring your data remains within your controlled infrastructure.

From importing a CSV to a self-service Explore in seconds

Key capabilities to power your analysis

With Looker self-service Explores, you gain:

Instant file uploads: Use a simple drag-and-drop interface to ingest local files for one-off analyses or testing a hypothesis before committing it to a permanent model.
Connect directly to Google Sheets: Easily import data via Google Sheets either via Oauth or by specifying the Google Sheets URL and sharing the document.
Merge queries (BigQuery): You can now combine your uploaded local files with standard, modeled Looker data, allowing you to enrich official company metrics with external data points to find new correlations. We’ve also added enhanced merge queries capabilities so you can perform merges on unlimited data as long as the data is on the same BigQuery connection.
Re-upload and refresh: You can easily re-import or update files within existing self-service Explores to keep your ad-hoc dashboards current.

Exploration through conversation, with red-tape free governance

Business intelligence in the agentic era has created new opportunities for business and data leaders throughout your organization to engage with their information faster and more intuitively, without significant technical demand. Self-service Explores bring support for conversational analytics, enabling you to ask questions in natural language about your uploaded data and get immediate answers, outlined visually, with grounding in Looker’s semantic layer. Users can chat with their data to drill down deeper, gaining more insights with follow-up questions and refinement.

Additionally, self-service Explores are built with admin controls at the forefront. Admins have full visibility and monitoring capabilities, with clear distinctions between ad-hoc data and modeled data. This gives users the freedom to explore, while maintaining the integrity of your core business logic. Self-service Explores give you the agility of a spreadsheet with the scale and security of BigQuery. It’s about spending less time waiting for a data model and more time actually using your data. Get started today.

Conversational Analytics now available for Looker Embedded environments

Mon, 06 Apr 2026 16:00:00 +0000

Looker Embedded analytics are at the heart of many next-generation data products, enabling monetization with live metrics and customizable user experiences. In the AI era, users expect apps to be highly interactive and conversational, and for data to be contextual, accessible and intuitive. Today, we are delivering conversational analytics in Looker Embedded environments with the Conversational Analytics API, now generally available, extending the natural language experience users have grown to expect from Looker to more surfaces, limited only by customers’ imagination.

Leading the shift to composable and agentic BI

Organizations are building integrated data experiences that meet users where they work. This is happening alongside the rise of agentic IDEs, where developers use AI agents to plan, code, and execute complex engineering tasks. Leveraging Looker’s embedded architecture within these environments incurs many benefits:

Differentiated agent experiences: Build high demand, unique conversational experiences that set your products apart.
Accelerated data monetization and developer extensibility: Turn raw data into sophisticated embedded products, transforming data assets into high-margin revenue streams.
AI-readiness in the IDE: Build with agentic IDEs with confidence, using Looker’s semantic layer to reduce hallucinations and maximize speed-to-market.

Conversational Analytics generally available for embedded users

Building on Looker’s existing embedded framework and governed semantic layer, you can now integrate Gemini-powered natural language querying and AI recommendations via a low-code iframe implementation or our extensible SDKs. This makes it easier to ship production-ready, conversational AI within any application.

With native support for querying multiple Looker Explores, a built-in code interpreter, and customizable theming, you can now provide a private-labeled, high-reasoning agent experience alongside your core features, bridging the gap between complex data models and intuitive user discovery.

Embed Conversational Analytics in Looker anywhere in your application using an iframe

Conversational Analytics now available from the Looker API

With our Conversational Analytics APIs, you can now build conversational experiences backed by Looker Explores directly within your customer-facing applications. By leveraging the Looker API, you can create multi-turn conversational workflows that offer AI-powered recommendations, while also verifying and explaining the underlying SQL query. The Looker API turns to the Looker SDK for user authentication and management, making it easy to integrate agentic conversations anywhere in your app.

Building a custom UI with Conversational Analytics in Looker API endpoints

Data exploration is a conversation away

The shift from static and inflexible dashboards to AI-driven exploration is transforming how businesses deliver value to their customers, empowering them to do more. Grounding these new conversational capabilities in Looker’s semantic layer means the insight accuracy you’ve always relied on from Looker is now available in third party applications, so you can know the insights are verifiable.

Whether you use the embedded iframe option or the Looker SDK, you can now build data experiences that don't just show information, but engage users in a dialogue. To start building conversational experiences in your own products with Looker Embedded, check out the documentation and best practices guide.

Envoy: A future-ready foundation for agentic AI networking

Fri, 03 Apr 2026 16:00:00 +0000

In today's agentic AI environments, the network has a new set of responsibilities.

In a traditional application stack, the network mainly moves requests between services. But as discussed in a recent white paper, Cloud Infrastructure in the Agent-Native Era, in an agentic system the network sits in the middle of model calls, tool invocations, agent-to-agent interactions, and policy decisions that can shape what an agent is allowed to do. The rapid proliferation of agents, often built on diverse frameworks, necessitates a consistent enforcement of governance and security across all agentic paths at scale. To achieve this, the enforcement layer must shift from the application level to the underlying infrastructure. That means the network can no longer operate as a blind transport layer. It has to understand more, enforce better, and adapt faster. This shift is precisely where Envoy comes in.

As a high-performance distributed proxy and universal data plane, Envoy is built for massive scale. Trusted by demanding enterprise environments, including Google Cloud, it supports everything from single-service deployments to complex service meshes using Ingress, Egress, and Sidecar patterns. Because of its deep extensibility, robust policy integration, and operational maturity, Envoy is uniquely suited for an era where protocols change quickly and the cost of weak control is steep. For teams building agentic AI, Envoy is more than a concept: it's a practical, production-ready foundation.

Agentic AI changes the networking problem

Agentic workloads still often use HTTP as a transport, but they break some of the assumptions that traditional HTTP intermediaries rely on. Protocols such as Model Context Protocol (MCP) and Agent2agent (A2A) use JSON-RPC or gRPC over HTTP, adding protocol-level phases such as MCP initialization, where client and server exchange their capabilities, on top of standard HTTP request/response semantics. The key aspects of agentic systems that require intermediaries to adapt include:

Diverse enterprise governance imperatives. The primary challenge is satisfying the wide spectrum of non-negotiable enterprise requirements for safety, security, data privacy, and regulatory compliance. These needs often go beyond standard network policies and require deep integration with internal systems, custom logic, and the ability to rapidly adapt to new organizational rules or external regulations. This demands a highly extensible framework where enterprises can plug in their specific governance models.
Policy attributes live inside message bodies, not headers. Unlike traditional web traffic where policy inputs like paths and headers are readily accessible, agentic protocols frequently bury critical attributes (e.g., model names, tool calls, resource IDs) deep within JSON-RPC or gRPC payloads. This shift requires intermediaries to possess the ability to parse and understand message contents to apply context-aware policies.
Handling diverse and evolving protocol characteristics. Agentic protocols are not uniform. Some, like MCP with Streamable HTTP, can introduce stateful interactions requiring session management across distributed proxies (e.g., using Mcp-Session-Id). The need to support such varied behaviors, along with future protocol innovations, reinforces the necessity of an inherently adaptable and extensible networking foundation.

These factors mean enterprises need more than just connectivity. The network must now serve as a central point for enforcing the crucial governance needs mentioned earlier. This includes providing capabilities like centralized security, comprehensive auditability, fine-grained policy enforcement, and dynamic guardrails, all while keeping pace with the rapid evolution of protocols and agent behaviors. Put simply, agentic AI transforms the network from a mere transit path into a critical control point.

Why Envoy fits this shift

Envoy is a strong fit for agentic AI networking for three reasons. Envoy is:

Battle-tested. Enterprises already rely on Envoy in high-scale, security-sensitive environments, making it a credible platform to anchor a new generation of traffic management and policy enforcement.
Extensible. Envoy can be extended through native filters, Rust modules, WebAssembly (Wasm) modules, and external processing patterns. That gives platform teams room to adopt new protocols without having to rebuild their networking layer every time the ecosystem changes.
Operationally useful today. Envoy already acts as a gateway, enforcement point, observability layer, and integration surface for control planes. That makes it a practical choice for organizations that need to move now, not after the standards settle.

Building on these core strengths, Envoy has introduced specific architectural advancements to meet the unique demands of agentic networking:

1. Envoy understands agent traffic

The first requirement for agentic networking is simple: The gateway needs to understand what the agent is actually trying to do.

That’s harder than it sounds. In protocols such as MCP, A2A, and OpenAI-style APIs, important policy signals may live inside the request body. Traditional HTTP proxies are optimized to treat bodies as opaque byte streams. That design is efficient, but it limits what the proxy can enforce. For protocols that use JSON messages, a proxy may need to buffer the entire request body to locate attribute values needed for policy application — especially when those attributes appear at the end of the JSON message. Business logic specific to gen AI protocols, such as rate limiting based on consumed tokens, may also require parsing server responses.

Envoy addresses this by deframing protocol messages carried over HTTP and exposing useful attributes to the rest of the filter chain. The extensibility model for gen AI protocols was guided by two goals:

Easy reuse of existing HTTP extensions that work with gen AI protocols out of the box, such as RBAC or tracers.
Easy access to deframed messages for gen-AI-specific extensions, so that developers can focus on gen AI business logic without needing to deal with HTTP or JSON envelopes.

Based on these goals, new extensions for gen AI protocols are still built as HTTP extensions and configured in the HTTP filter chain. This provides flexibility to mix HTTP-native business logic, such as OAuth or mTLS authorization, with gen AI protocol logic in a single chain. A deframing extension parses the protocol messages carried by HTTP and provides an ambient context with extracted attributes, or even the entirety of parsed messages, to downstream extensions via well-known filter state and metadata values.

Instead of forcing every policy component to parse JSON envelopes or protocol-specific message formats on its own, Envoy makes those attributes available as structured metadata. Once the gateway has deframed protocol messages, existing Envoy extensions such as ext_authz or RBAC can read protocol properties to evaluate policies using protocol-specific attributes such as tool names for MCP, message attributes for A2A, or model names for OpenAI.

Access logs can include message attributes for enhanced monitoring and auditing. The protocol attributes are also available to the Common Expression Language (CEL) runtime, simplifying creation of complex policy expressions in RBAC or composite extensions.

Buffering and memory management
Envoy is designed to use as little memory as possible when proxying HTTP requests. However, parsing agentic protocols may require an arbitrary amount of buffer space, especially when extensions require the entire message to be in memory. The flexibility of allowing extensions to use larger buffers needs to be balanced with adequate protection from memory exhaustion, especially in the presence of untrusted traffic.

To achieve this, Envoy now provides a per-request buffer size limit. Buffers that hold request data are also integrated with the overload manager, enabling a full range of protective actions under memory pressure, such as reducing idle timeouts or resetting requests that consume the most memory for an extended duration. These changes pave the way for Envoy to serve as a gateway and policy-enforcement point for gen AI protocols without compromising its resource efficiency.

2. Envoy enforces policy on things that matter

Understanding traffic is only useful if the gateway can act on it.

In agentic systems, policy is not just about which service an agent can reach. It’s about which tools an agent can call, which models it can use, what identity it presents, how much it can consume, and what kinds of outputs require additional controls. Those are higher-value decisions than simple layer-4 or path-based controls, and they are exactly the kinds of controls enterprises care about when agents are allowed to take action on their behalf.

Envoy is well-positioned here because it can combine transport-level security with application-aware policy enforcement. Teams can authenticate workloads with mTLS and SPIFFE identities, then enforce protocol-specific rules with RBAC, external authorization, external processing, access logging, and CEL-based policy expressions.

This capability is crucial because it lets platform teams decouple agent development from enforcement. Developers can focus on building useful agents, while operators enforce a consistent zero-trust posture at the network layer, even as tools, models, and protocols continue to change.A prime example of this zero-trust decoupling is the critical "user-behind-agent" scenario, where an AI agent must execute tasks on a human user's behalf. Traditionally, handing user credentials directly to an application introduces severe security risks — if the agent is compromised or manipulated via prompt injection, an attacker could exfiltrate or misuse those credentials. By offloading identity management to Envoy, the proxy can automatically insert user delegation tokens into outbound requests at the infrastructure layer. Because the agent never directly holds the sensitive credential, the risk of a compromised agent misusing or leaking the token is completely neutralized, ensuring actions remain strictly bound to the user's actual permissions.

Case study: Restricting an agent to specific GitHub MCP tools
Consider an agent that triages GitHub issues.

The GitHub MCP server may expose dozens of tools, but the agent may only need a small read-only subset, such as list_issues, get_issue, and get_issue_comments. In most enterprises, that difference matters. A useful agent should not automatically become an unrestricted one.

With Envoy in front of the MCP server, the gateway can verify the agent identity using SPIFFE during the mTLS handshake, parse the MCP message via the deframing filter, extract the requested method and tool name, and enforce a policy that allows only the approved tool calls for that specific agent identity. RBAC uses metadata created by the MCP deframing filter to check the method and tool name in the MCP message:

code_block: <ListValue: [StructValue([('code', 'envoy.filters.http.rbac:\r\n "@type": type.googleapis.com/envoy.extensions.filters.http.rbac.v3.RBACPerRoute\r\n rbac:\r\n rules:\r\n policies:\r\n github-issue-reader-policy:\r\n permissions:\r\n - and_rules:\r\n rules:\r\n - sourced_metadata:\r\n metadata_matcher:\r\n filter: envoy.http.filters.mcp\r\n path: [{ key: "method" }]\r\n value: { string_match: { exact: "tools/call" } }\r\n - sourced_metadata:\r\n metadata_matcher:\r\n filter: envoy.http.filters.mcp\r\n path: [{ key: "params" }, { key: "name" }]\r\n value:\r\n or_match:\r\n value_matchers:\r\n - string_match: { exact: "list_issues" }\r\n - string_match: { exact: "get_issue" }\r\n - string_match: { exact: "get_issue_comments" }\r\n principals:\r\n - authenticated:\r\n principal_name:\r\n exact: "spiffe://cluster.local/ns/github-agents/sa/issue-triage-agent"'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f507a0df790>)])]>

That’s the real value: Policy is enforced centrally, close to the traffic, and in terms that match the agent's actual behavior.

Beyond static rules: External authorization
A complex compliance policy that can’t be expressed using RBAC rules can be implemented in an external authorization service using the ext_authz protocol. Envoy provides MCP message attributes along with HTTP headers in the context of the ext_authz RPC. It can also forward the agent's SPIFFE identity from the peer certificate:

code_block: <ListValue: [StructValue([('code', 'http_filters:\r\n - name: envoy.filters.http.ext_authz\r\n typed_config:\r\n "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz\r\n grpc_service:\r\n envoy_grpc:\r\n cluster_name: auth_service_cluster\r\n include_peer_certificate: true\r\n metadata_context_namespaces:\r\n - envoy.http.filters.mcp'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f507a0df100>)])]>

This allows external services to make authorization decisions based on the full combination of agent identity, MCP method, tool name, and any other protocol attributes, without the agent or the MCP server needing to be aware of the policy layer.

Protocol-native error responses
When Envoy denies a request, the error should be meaningful to the calling agent. For MCP traffic, Envoy can use local_reply_config to map HTTP error codes to appropriate JSON-RPC error responses. For example, a 403 Forbidden can be mapped to a JSON-RPC response with isError: true and a human-readable message, ensuring the agent receives a protocol-appropriate denial rather than an opaque HTTP status code.

3. Envoy supports stateful agent interactions at scale

Not all agent traffic is stateless. Some protocols, including Streamable HTTP for MCP, can rely on session-oriented behavior. That creates a new challenge for intermediaries, especially when traffic flows through multiple gateway instances to achieve scale and resilience. An MCP session effectively binds the agent to the server that established it, and all intermediaries need to know this to direct incoming MCP connections to the correct server.

If a session is established on one backend, later requests in that conversation need to reach the right destination. That sounds straightforward for a single-proxy deployment, but it becomes more complicated in horizontally scaled systems, where multiple Envoy instances may handle different requests from the same agent.

Passthrough gateway
In the simpler passthrough mode, Envoy establishes one upstream connection for each downstream connection. Its primary use is enforcing centralized policies, such as client authorization, RBAC, rate limiting, and authentication, for external MCP servers. The session state transferred between intermediaries needs to include only the address of the server that established the session over the initial HTTP connection, so that all session-related requests are directed to that server.

Session state transfer between different Envoy instances is achieved by appending encoded session state to the MCP session ID provided by the MCP server. Envoy removes the session-state suffix from the session ID before forwarding the request to the destination MCP server. This session stickiness is enabled by configuring Envoy's envoy.http.stateful_session.envelope extension.

Aggregating gateway
In aggregating mode, Envoy acts as a single MCP server by aggregating the capabilities, tools, and resources of multiple backend MCP servers. In addition to enforcing policies, this simplifies agent configuration and unifies policy application for multiple MCP servers.

Session management in this mode is more complicated because the session state also needs to include mapping from tools and resources to the server addresses and session IDs that advertised them. The session ID that Envoy provides to the agent is created before tools or resources are known, and the mapping has to be established later, after the MCP initialization phases between Envoy and the backend MCP servers are complete.

One approach, currently implemented in Envoy, is to combine the name of a tool or resource with the identifier and session ID of its origin server. The exact tool or resource names are typically not meaningful to the agent and can carry this additional provenance information. If unmodified tool or resource names are desirable, another approach is to use an Envoy instance that does not have the mapping, and then recreate it by issuing a tools/list command before calling a specific tool. This trades latency for the complexity of deploying an external global store of MCP sessions, and is currently in planning based on user feedback.

This matters because it moves Envoy beyond simple traffic forwarding. It allows Envoy to serve as a reliable intermediary for real agent workflows, including those spanning multiple requests, tools, and backends.

4. Envoy supports agent discovery

Envoy is adding support for the A2A protocol and agent discovery via a well-known AgentCard endpoint. AgentCard, a JSON document with agent capabilities, enables discovery and multi-agent coordination by advertising skills, authentication requirements, and service endpoints. The AgentCard can be provisioned statically via direct response configuration or obtained from a centralized agent registry server via xDS or ext_proc APIs. A more detailed description of A2A implementation and agent discovery will be published in a forthcoming blog post.

5. Envoy is a complete solution for agentic networking challenges

Building on the same foundation that enabled policy application for MCP protocol in demanding deployments, Envoy is adding support for OpenAI and transcoding of agentic protocols into RESTful HTTP APIs. This transcoding capability simplifies the integration of gen AI agents with existing RESTful applications, with out-of-the-box support for OpenAPI-based applications and custom options via dynamic modules or Wasm extensions. In addition to transcoding, Envoy is being strengthened in critical areas for production readiness, such as advanced policy applications like quota management, comprehensive telemetry adhering to OpenTelemetry semantic conventions for generative AI systems, and integrated guardrails for secure agent operation.

Guardrails for safe agents
The next significant area of investment is centralized management and application of guardrails for all agentic traffic. Integrating policy enforcement points with external guardrails presently requires bespoke implementation and this problem area is ripe for standardization.

Control planes make this operational

The gateway is only part of the story. To achieve this policy management and rollout at scale, a separate control plane is required to dynamically configure the data plane using the xDS protocol, also known as the universal data plane API.

That is where control planes become important. Cloud Service Mesh, alongside open-source projects such as Envoy AI Gateway and kube-agentic-networking, uses Envoy as the data plane while giving operators higher-level ways to define and manage policy for agentic workloads.

This combination is powerful: Envoy provides the enforcement and extensibility in the traffic path, while control planes provide the operating model teams need to deploy that capability consistently.

Why this matters now

The shift towards agentic systems and gen AI protocols such as MCP, A2A, and OpenAI necessitates an evolution in network intermediaries. The primary complexities Envoy addresses include:

Deep protocol inspection. Protocol deframing extensions extract policy-relevant attributes (tool names, model names, resource paths) from the body of HTTP requests, enabling precise policy enforcement where traditional proxies would only see an opaque byte stream.
Fine-grained policy enforcement. By exposing these internal attributes, existing Envoy extensions like RBAC and ext_authz can evaluate policies based on protocol-specific criteria. This allows network operators to enforce a unified, zero-trust security posture, ensuring agents comply with access policies for specific tools or resources.
Stateful transport management. Envoy supports managing session state for the Streamable HTTP transport used by MCP, enabling robust deployments in both passthrough and aggregating gateway modes, even across a fleet of intermediaries.

Agentic AI protocols are still in their early stages, and the protocol landscape will continue to evolve. That’s exactly why the networking layer needs to be adaptable. Enterprises should not have to rebuild their security and traffic infrastructure every time a new agent framework, transport pattern, or tool protocol gains traction. They need a foundation that can absorb change without sacrificing control.

Envoy brings together three qualities that are hard to get in one place: proven production maturity, deep extensibility, and growing protocol awareness for agentic workloads. By leveraging Envoy as an agent gateway, organizations can decouple security and policy enforcement from agent development code.

That makes Envoy more than just a proxy that happens to handle AI traffic. It makes Envoy a future-ready foundation for agentic AI networking.

^{Special thanks to the additional co-authors of this blog: Boteng Yao, Software Engineer, Google and Tianyu Xia, Software Engineer, Google and Sisira Narayana, Sr Product Manager, Google.}

Introducing Veo 3.1 Lite and a new Veo upscaling capability on Vertex AI

Fri, 03 Apr 2026 16:00:00 +0000

We are introducing Veo 3.1 Lite, Google's most cost-effective video model on Vertex AI.

Alongside this new model, we are also launching a new, standalone Veo upscaling capability on Vertex AI to help you enhance your existing video assets.

Choosing the right Veo model for your workload

When integrating video generation into your applications, matching the model to your specific use case is critical. The Veo 3.1 family now includes three tiers, all of which feature native audio generation capabilities:

Veo 3.1: This model is designed for state-of-the-art video generation where visual fidelity is the top priority for final production cuts
Veo 3.1 Fast: This option delivers faster video generation while maintaining high quality, making it ideal for standard production workflows.
Veo 3.1 Lite: This is our most cost-effective model, empowering businesses to build high-volume video applications and rapidly iterate and scale.

For a pricing breakdown, visit the Vertex AI pricing page.

For a full guide, visit our ultimate prompting guide for Veo 3.1.

Upscale your assets with the new Veo upscaling capability

A highly requested feature from customers is the ability to upscale existing low-resolution videos to higher resolutions. Available in private preview, and coming soon to public preview, the new Veo upscaling capability allows you to enhance your videos up to 1080p and 4K, regardless of whether they were generated by Veo, another AI model, or a traditional camera.

Take a look at this demo to see how Veo brings your idea to life.

Get started with Veo 3.1 Lite today

Rolling out today, you can access the model via the Vertex AI API and Vertex AI Media Studio.

To dive deeper into the best practices, explore the following resources:

What’s new with Google Cloud

Fri, 03 Apr 2026 16:00:00 +0000

Want to know the latest from Google Cloud? Find it here in one handy location. Check back regularly for our newest updates, announcements, resources, events, learning opportunities, and more.

Tip: Not sure where to find what you’re looking for on the Google Cloud blog? Start here: Google Cloud blog 101: Full list of topics, links, and resources.

aside_block: <ListValue: []>

Mar 30 - Apr 3

ASEAN Webinar | April 30: Mastering Agentic Governance at Scale with GCP
As AI agents move from experimental pilots to core enterprise functions, governance is the critical next step. Join Google Cloud experts Shilpi Puri & Wely Lau for a webinar on April 30th at 11:00 AM SGT to learn how to architect a secure AI Management layer. We’ll explore developing governed MCP endpoints, managing tool access to enterprise data, and operationalizing AI with robust audit logs. The session includes a live demo of these frameworks in action on Google Cloud.

RSVP here.

Mar 23 - Mar 27

Turn your API sprawl into an agent-ready catalog
As organizations scale, APIs often become scattered across multiple gateways, creating "blind spots" that hinder AI adoption. To solve this, we’ve introduced two new capabilities for Apigee API hub: a new integration with API Gateway to automatically centralize API metadata into a single control plane, and a specification boost add-on (now in public preview). This add-on uses AI to enhance your API documentation with the precise examples and error codes that AI agents need to function reliably.

Read the full blog post to get started.
Webinar | April 16: AI Command & Control
As AI agents move from experimental pilots to core enterprise functions, governance is the critical next step. Join Google Cloud expert Satyam Maloo for a webinar on April 16th at 11:00 AM IST to learn how to architect a secure AI Management layer. We’ll explore developing governed MCP endpoints, managing tool access to enterprise data, and operationalizing AI with robust audit logs. The session includes a live demo of these frameworks in action on Google Cloud.

RSVP here.
Modernizing and Decoupling Event Ingestion with Apigee
In modern cloud-native architectures, decoupling producers from consumers is critical for building resilient systems. While Google Cloud Pub/Sub provides a scalable backbone, exposing it directly to external clients can introduce security and management overhead. This new guide explores how to leverage Apigee as an intelligent HTTP ingestion point. Learn how to handle security, mediation, and traffic control before messages reach your internal bus using the PublishMessage policy or Pub/Sub API.

Read the full guide.

Mar 16 - Mar 20

Gemini-powered Assistant in BigQuery Studio Gets Context-Aware Upgrades
The Gemini-powered assistant in BigQuery Studio has been transformed into a fully context-aware analytics partner, supporting your entire data lifecycle. The new capabilities include intelligent resource discovery, which uses Dataplex Universal Catalog search to find resources across projects and deep dive into metadata using natural language. You can now automate tasks, such as scheduling production-grade queries directly through the chat interface, and instantly troubleshoot long-running or failed jobs with root cause analysis and cost control auditing.

Explore the full range of what the assistant can do.

Mar 9 - Mar 13

Want to use Gemini to develop code and don't know where to start?
This article includes a couple of examples of developing code with Gemini prompts; it identified changes that were needed to be made to get the code working. The article also refers to other examples that are available on github.

Mar 2 - Mar 6

Introducing Gemini 3.1 Flash-Lite, our fastest and most cost-efficient Gemini 3 series model. Built for high-volume developer workloads at scale, 3.1 Flash-Lite delivers high quality for its price and model tier. Gemini 3.1 Flash-Lite can tackle tasks at scale, like high-volume translation and content moderation, where cost is a priority. And it can also handle more complex workloads where more in-depth reasoning is needed, like generating user interfaces and dashboards, creating simulations or following instructions.

Starting today, 3.1 Flash-Lite is rolling out in preview to enterprises via Vertex AI and developers via the Gemini API in Google AI Studio.
TechTalk: Implementing Device Authorization Grant (RFC 8628) for Apigee
Learn how to authorize "headless" devices like Smart TVs or AI agents that lack keyboards and browsers. Join our Community TechTalk on March 19 (5PM CET / 12PM EDT) to go under the hood of Apigee X/Hybrid. We’ll cover the real-world mechanics of state management, polling, and human-in-the-loop security patterns for devices and autonomous agents.

Register for the TechTalk

Feb 23 - Feb 27

Pro-level image generation gets faster and more accessible with Nano Banana 2
Nano Banana 2 is our state-of-the-art image generation and editing model. It delivers Pro-level image generation and editing at the speed you expect from Flash — making the quality, reasoning, and world knowledge you loved about Nano Banana Pro more accessible. Learn more about the model here.

The Intelligent Path to Compliance: Transforming Regulatory QC with Google Cloud
Reducing "Refuse to File" (RTF) risks and submission cycle times is critical for life sciences leaders. Google Cloud’s Regulatory Submission Semantic QC Auditor leverages Gemini and RAG architecture to transform Quality Control from a manual burden into an active, intelligent workflow.

By automating semantic cross-referencing, narrative coherence checks, and dynamic guidance-based auditing, this solution ensures rigorous accuracy and auditability. Operating within a secure GxP-ready environment, it empowers teams to detect subtle inconsistencies and generate remediation plans without sacrificing data privacy.

Learn more.
Stop typing, start interacting! The Gemini Live Agent Challenge is here. Build immersive agents that can help you see, hear, and speak using Gemini and Google Cloud. Compete for your share of $80,000+ in prizes and a trip to Google Cloud Next '26!

Submissions are open from February 16, 2026 to March 16, 2026. Learn more and register at geminiliveagentchallenge.devpost.com

Feb 9 - Feb 13

Introducing Gemini 3.1 Pro on Google Cloud.
3.1 Pro is a noticeably smarter, more capable baseline for complex problem-solving. We’re shipping 3.1 Pro at scale, building upon our goal to help you transform your business for the agentic future. Learn more about the model’s capabilities here. Gemini 3.1 Pro is available starting today in preview in Vertex AI and Gemini Enterprise. Developers can access the model in preview via the Gemini API in Google AI Studio, Android Studio, Google Antigravity, and Gemini CLI.
Automate Storage Compatibility with GKE Dynamic Default Storage Classes
Managing storage across mixed-generation VM clusters in GKE just got easier. With the new Dynamic Default Storage Class, Google Kubernetes Engine automatically selects between Persistent Disk (PD) and Hyperdisk based on a node's specific hardware compatibility. This abstraction eliminates the need for complex scheduling rules and manual pairing, ensuring your volumes "just work" regardless of the underlying infrastructure. By defining both variants in a single class, you reduce operational overhead while maintaining peak performance and cost-efficiency across your entire cluster.

Explore automated disk type selection
Community TechTalk: AI-Powered Apigee Development with strofa.io
Join the Apigee community on February 26 for a deep dive into strofa.io. Guest speaker Denis Kalitviansky will demonstrate how this new AI-powered tool automates and orchestrates Apigee development, from local emulators to large-scale hybrid environments. Discover how to scale your API management and streamline team collaboration using the latest in AI-driven automation.

Register now to reserve your spot.

Jan 26 - Jan 30

Simplify API Governance with Native OpenAPI v3 Support
Eliminate integration debt and accelerate deployment velocity with the General Availability of OpenAPI v3 (OASv3) support for API Gateway and Cloud Endpoints. You no longer need to downgrade modern specifications to OASv2. Instead, you can now define API contracts and enforce critical policies—including telemetry, quotas, and security—using native Google-specific extensions directly within your OASv3 files. This update ensures your APIs are secure by design while remaining fully compatible with the modern developer ecosystem and Google Cloud’s AI services.

Get started with OpenAPI v3 on API Gateway and Cloud Endpoints.

Accelerate API Testing with the New Open Source API Tester
Start validating your APIs with API Tester, a simple, YAML-based Test Driven Development (TDD) framework. Designed for the Apigee community, this tool allows you to write human-readable tests, run them instantly via a web client or CLI, and perform deep unit testing on Apigee proxies. With native support for JSONPath assertions and Apigee shared flows, you can verify everything from payload data to internal variables like proxy.basepath without leaving your terminal.

Explore the API Tester guide and start testing your proxies today.
Secure Sensitive Data with Kubernetes Secrets in Apigee hybrid
Enhance security in Apigee hybrid by accessing Kubernetes Secrets directly within your API proxies. This hybrid-exclusive feature keeps sensitive credentials within your cluster boundary and prevents replication to the management plane. It supports strict separation of duties: operators manage secrets via kubectl, while developers reference them as secure flow variables—ideal for high-compliance and GitOps workflows.

Implement Kubernetes Secrets in your hybrid proxies.
See the Console in a Whole New Light: Dark Mode is Now Generally Available in Google Cloud
Elevate your cloud management workflow with Dark Mode, now generally available in the Google Cloud console. We have delivered a modern, cohesive, and accessible experience reimagined for maximum comfort and productivity—especially during extended working hours and low-light environments. Dark Mode can be enabled automatically based on your operating system's preference, or manually through the Settings -> Appearance menu.

Switch to Dark Mode today to enjoy a modern, comfortable, and productive environment!
Apigee X Networking: PSC or VPC Peering?
Deciding how to connect Apigee X? Watch this video to compare Private Service Connect and VPC Peering. We break down northbound and southbound routing, IP consumption, and how to reach targets on-prem or in the cloud. Learn to simplify your architecture and avoid common networking "gotchas" for a smoother deployment.

Watch the video.

Jan 19 - Jan 23

Bridge the Gap: Excel-to-API Conversion in Apigee Portals
Give your customers more ways to connect! This new article by Tyler Ayers explores how to extend the Apigee Integrated Portal to support direct Excel file uploads. By leveraging SheetJS and custom portal scripts, you can enable users to upload spreadsheets, preview data, and submit it directly to your APIs, all without writing a single line of integration code themselves. It’s a powerful way to simplify onboarding for those who aren't yet API-ready.

Learn how to build it.
Elevate your applications with Firestore’s new advanced query engine
We have fundamentally reimagined Firestore with pipeline operations for Enterprise edition. Experience a powerful new engine featuring over a hundred new query features, index-less queries, new index types, and observability tooling to improve query performance. Seamlessly migrate using built-in tools and leverage Firestore’s existing differentiated serverless foundation, virtually unlimited scale, and industry-leading SLA. Join a community of 600K developers to craft expressive applications that maximize the benefits of rich queryability, real-time listen queries, robust offline caching, and cutting-edge AI-assistive coding integrations.

Learn more about Firestore pipeline operations.

Introducing Gemma 4 on Google Cloud: Our most capable open models yet

Thu, 02 Apr 2026 16:00:00 +0000

Today, we are releasing Gemma 4 on Google Cloud.

What’s new: It is, byte for byte, the most capable family of open models. Built from the same research as Gemini 3 and released under a commercially permissive Apache 2.0 license, these models move beyond chat. With context windows up to 256K, native vision and audio processing, and fluency in over 140 languages, they excel at complex logic, offline code generation, and agentic workflows. Learn more about the model here.

Why it matters for your business: Enterprise AI requires models that execute complex logic while keeping data within secure boundaries. Gemma 4 gives you this balance. Organizations can deploy these models across Google Cloud to meet strict compliance guarantees, including Sovereign Cloud solutions. This provides a foundation for digital sovereignty, granting teams complete control over their data, infrastructure, and models.

Where you can get started with Gemma 4

Vertex AI
Deploy Gemma 4 to your own Vertex AI endpoints. Select the model from Model Garden and provision the specific compute resources your application requires. This self-deployment model gives you direct control over your serving infrastructure and costs while keeping your data within your Google Cloud environment.

You can also fine-tune Gemma 4 using Vertex AI Training Clusters (VTC), which offer optimized SFT recipes and high-scale resiliency through NVIDIA NeMo Megatron. This ensures you can efficiently adapt any variant, from the effective 2B (E2B) model for edge tasks to the 31B dense model for complex enterprise orchestration. Here’s an end to end guide for efficient fine-tuning and serving of the Gemma 4 31B model on Vertex AI.

Additionally, we're committed to empowering customer choice and innovation through our curated collection of first-party, open, and third-party models available on Vertex AI. That’s why we're thrilled to announce that Gemma 4 26B MoE model will be available as fully managed and serverless on Model Garden over the coming days.

Agent Development Kit (ADK)
ADK is a flexible and modular open-source framework for developing and deploying AI agents. Gemma 4 offers advanced agentic capabilities, including reasoning, function calling, code generation, and structured output. ADK helps you build fully functional AI agents with Gemma 4. Start building AI agents with Gemma 4 and Google ADK today.

Cloud Run
You can now run demanding Gemma 4 inference workloads efficiently on Cloud Run, leveraging the power of NVIDIA RTX PRO 6000 (Blackwell) GPUs. With 96GB of vGPU memory, you can easily deploy models like Gemma-4-31B-it on serverless GPUs.

Cloud Run handles the underlying infrastructure, allowing you to focus on your applications. Your models scale to zero when inactive and dynamically adjust with demand, ensuring optimized costs as you only pay for what you use. Plus, you have the flexibility to tailor CPU and memory configurations for each inference workload. Try it out now, on demand with no reservations, in us-central1 or europe-west4.

Google Kubernetes Engine (GKE)
GKE provides a highly scalable and customizable environment for deploying Gemma 4, perfect for teams that require fine-grained control over their AI infrastructure. By managing your own infrastructure on GKE, you gain the flexibility to tailor compute resources, select specific GPU or TPU accelerators, and implement custom autoscaling metrics that match your exact traffic patterns. This level of control also ensures your AI workloads can seamlessly integrate with your existing microservices while adhering to your organization's strict security and data compliance requirements.

Starting today, you can efficiently serve Gemma 4 models on GKE using vLLM, a high-throughput and memory-efficient LLM serving engine. By leveraging GKE, you can seamlessly scale your inference workloads from zero to peak demand while optimizing your resource utilization and costs. To help you get started, check out our newly updated tutorial on how to serve Gemma 4 on GKE.

Looking ahead, Gemma 4 is uniquely positioned to power the next generation of agentic applications on Google Cloud. Pairing Gemma 4’s multi-step planning capabilities with the new GKE Agent Sandbox, developers can safely execute LLM-generated code and tool calls within highly isolated, Kubernetes-native environments that offer sub-second cold starts with up to 300 sandboxes per second for secure, efficient multi-step planning. Furthermore, by leveraging the GKE Inference Gateway and advanced distributed inference features in llm-d like predicted-latency-based scheduling, these complex workflows benefit from intelligent routing that dynamically balances cache reuse and server load. GKE Inference Gateway with Predictive Latency Boost can cut time-to-first-token (TTFT) latency by up to 70% by replacing heuristic guesswork with real-time capacity-aware routing, no manual tuning required.

Google Cloud TPUs
Gemma 4 will be available on TPUs across Google Cloud through GKE, GCE, and Vertex AI. Starting today, you can now use a number of popular open source TPU projects to serve, pretrain, and post-train Gemma-4-31B dense and Gemma-4-26B-A4B MoE.

For pretraining and post-training experimentation, you can leverage MaxText and perform post training to customize for text analysis and generation, reasoning and image analysis use cases.

For online serving and batch inference, you’ll be able to use vLLM TPU for your production workloads using our prebuilt docker containers, quickstart vision, and text demo tutorials.

Stay tuned for community-contributed SGLang-JAX tutorials.

Sovereign Cloud
Gemma 4 will be available across all our Sovereign Cloud offerings, including public cloud with Data Boundary, Google Cloud Dedicated (such as S3NS in France), and Google Distributed Cloud for air-gapped and on-premises deployments. This expansion reinforces our commitment to an open, sovereign digital world where organizations maintain total control over their data, encryption, and operational environment.

By providing open weights, Gemma 4 empowers developers to build specialized solutions for highly sensitive environments. Enterprise and government agencies can now deploy localized services that respect regional nuances and domain expertise while meeting strict data residency and sovereignty rules. This approach ensures that organizations can innovate rapidly with AI while remaining fully compliant with national and industry requirements.

Get started today

From Vertex AI to Sovereign Cloud, you can start building with Gemma 4 today. By choosing Gemma 4 on Google Cloud, enterprises and sovereign organizations gain a trusted, transparent foundation that delivers state-of-the-art capabilities while meeting the highest standards for security and reliability.

How Honeylove boosts product quality and service efficiency with BigQuery

Thu, 02 Apr 2026 16:00:00 +0000

Building the perfect bra takes thousands of data points.

That’s why Honeylove isn’t just another intimates brand. We’re a technology company that happens to make exceptional bras, tops, shapewear, and bodysuits. Technology shapes everything we do, from how we iterate garments based on customer feedback to how we optimize sizing across those thousands of data points.

When Honeylove was born in 2018, though, our data wasn’t consolidated. We were looking at analytics in Shopify, checking email campaign performance in one platform, and reviewing ad metrics in another. We weren’t connecting the dots as effectively as we could have.

Then we fell in love with BigQuery. In this post, we’ll cover how Honeylove uses BigQuery and Gemini to unify our data, automate key business insights, and leverage AI to boost product quality and service efficiency — as well as how other organizations looking to make the most of their data can follow our approach intimately.

Transforming insights with BigQuery and Gemini

The first step was getting all our data in one place. BigQuery gave us exactly what we needed: a performant, economical, unified data platform that integrates seamlessly with the tools our team already uses within the Google ecosystem, such as Google Ads and Google Sheets . This helped eliminate manual data silos and enabled us to quickly adopt AI and ML capabilities across the business.

The real transformation came when we started leveraging BigQuery ML functions for contribution analysis. We built models to analyze the key drivers behind some of our most critical metrics: conversion rate, customer satisfaction scores, website performance, and return rates.

What’s really powerful for us is that we can feed these contribution analysis results directly into Gemini to produce accessible reports and summaries. Before implementing this approach, 10 to 15 people would spend an hour before key meetings manually reviewing dashboards, trying to drill into the data and find meaningful insights. We’ve saved hundreds of hours per year just by automating this process with Gemini.

But the impact of BigQuery and Gemini goes beyond time savings. These tools help us find patterns and insights we would’ve missed entirely. Even if you have the best marketing analysts looking over dashboards, they just wouldn’t be able to slice it in the same way these reports allow us to do.

We’ve also been able to transform forecasting inventory and demand planning, another area where manual processes previously dominated. By deploying and training BigQuery ML’s ARIMA univariate forecasting models, we’ve used high-accuracy SKU-level demand forecasts that automatically adjust for seasonality and recent changes.

These automated forecasts consistently come within 5% of what we calculate manually — a huge improvement over third-party vendors that were sometimes off by 20% to 30%. Having that additional checkpoint gives us more confidence when making critical inventory decisions.

Unlocking value and creative with multimodal embeddings

Customer service tickets can be a treasure trove of valuable feedback and information for ecommerce brands. But only if you can extract insights from them, and with Google Cloud, we can.

We leverage Gemini embedding models and BigQuery vector search to transform the unstructured text of our tickets into actionable data. We generate vector embeddings for tickets already in our data warehouse using simple SQL commands, and then use those vectors for semantic searching through retrieval-augmented generation (RAG).

This allows us to ask precise, natural-language questions, such as “What do customers love about our bras?” or “What changes would you like to see to our bodysuits?” In response, Gemini instantly identifies similar use cases, enabling us to move beyond keyword matching and quickly find the root causes of any issues, which are often nuanced. This proactively guides product improvements and enhances service efficiency. We’re saving about 30 seconds per ticket, which might not sound dramatic until you multiply it across thousands of interactions.

We’re also experimenting with multimodal embeddings for video asset search across our ad and influencer content library. It’s been fun to test queries like “find me videos with dogs” or “find me a video with a red dress” and watch it actually work. The next step is to use those embeddings to compare new creative assets with existing ones and predict performance based on our historical data.

Growth creative has traditionally been driven by gut feelings rather than numerical analysis, but we hope to change that by using our huge library of existing ad creative to inform what we test and create in the future.

Building for the future with Google Cloud

Today, Google Cloud and BigQuery are a central pillar of our company. They allow us to spend less time on manual tasks and more time on high-value work that solves real-world problems, making us very efficient as a small team.

Working with the Google Cloud team is invaluable. They’ve been a true partner, and they continue to support our roadmap. We’re leaning further into BigQuery ML functionality, moving more of our data science work into automated, always-available models rather than offline analyses.

We’re also developing internal knowledge bots using the Vertex AI RAG Engine, connected directly to our internal documents hosted on Google Drive, to provide instant answers to internal policy and process questions. Additionally, we’re experimenting with Conversational Analytics API to provide a “BI in a box” experience so our teams can ask plain-text questions and get metrics and charts without needing an analyst.

As a technology-first company, this transformation continues to have a profound impact on what we do at Honeylove. It accelerated innovation in product quality, improved operational efficiency, and ensured that our customers receive a more intelligent and consistent service experience.

What’s new with Google Data Cloud

Thu, 02 Apr 2026 16:00:00 +0000

March 23 - March 27

We showed you how you can scale your reads with Cloud SQL autoscaling read pools. This feature allows you to provision multiple read replicas that are accessible via a single read endpoint and to dynamically adjust your read capability based on real-time application needs.
Our customers are leveraging the full power of Conversational Analytics and Looker to drive major business and technical breakthroughs in the AI era. Companies like Telenor, Pet Circle, Fluent Commerce, Lighthouse Intelligence, Wego, and ROLLER are turning data into insights and actions, grounded by Looker’s semantic layer.

March 16 - March 20

We introduced an enhanced Gemini assistant in BigQuery Studio, transforming the agent from a code assistant into a fully context-aware analytics partner.

February 23 - February 27

We introduced managed and remote MCP support for Google Cloud databases, including AlloyDB, Spanner, Cloud SQL, Bigtable and Firestore, to power the next generation of agents. This announcement extends the ability for AI models to plan, build, and solve complex problems, connecting to the database tools our customers leverage daily as the backbone of their work environment.
We outlined how you can build a conversational agent in BigQuery using the Conversational Analytics API to help you build context-aware agents that can understand natural language, query your BigQuery data, and deliver answers in text, tables, and visual charts.

February 16 - February 20

Our customers are leveraging the full power of Looker to drive major business and technical breakthroughs. Companies like Arrive, Audika, Carousell, Framebridge, GumGum, Intel, Overdose Digital, Ocean Network Express, Subskribe and Promevo are leveraging Looker’s newest AI-driven capabilities, including Conversational Analytics, to transform data to insights and actions, and empower their entire organization with a single source of truth, powered by Looker’s semantic layer.

February 2 - February 6

Join us on March 4 for our webinar, Win Your AI Strategy with Cloud SQL Enterprise Plus, to learn how to power your generative AI workloads with 3x higher performance and 99.99% availability. Register today to discover how to build a scalable, enterprise-grade foundation for your most demanding AI applications.

January 26 - January 30

We introduced Conversational Analytics in BigQuery, which allows users to analyze data using natural language. Conversational Analytics in BigQuery is an intelligent agent that generates, executes and visualizes answers grounded in your business context directly in BigQuery Studio, making data insights for data professionals more conversational.
We outlined how data products have become the foundation for AI agents, providing the context needed to make autonomous agents reliable and trusted for real business use, backed by organized business logic and semantic understanding.
We highlighted how you can supercharge data analytics workflows, and outlined Google Cloud’s AI agent offerings for data engineering, data science, and development tools, so you can integrate agentic workflows in your applications, empower your teams and speed discovery.

January 19 - January 23

We have fundamentally reimagined Firestore with pipeline operations for Enterprise edition. Experience a powerful new engine featuring over a hundred new query features, index-less queries, new index types, and observability tooling to improve query performance. Seamlessly migrate using built-in tools and leverage Firestore’s existing differentiated serverless foundation, virtually unlimited scale, and industry-leading SLA. Join a community of 600K developers to craft expressive applications that maximize the benefits of rich queryability, real-time listen queries, robust offline caching, and cutting-edge AI-assistive coding integrations.
Introducing Google Cloud SQL on MSSQLTips: We are highlighting a new technical guide published on MSSQLTips titled "Introducing Google Cloud SQL." This article serves as an essential resource for SQL Server administrators and developers exploring Google Cloud's fully managed database service. It provides a detailed overview of Cloud SQL capabilities, including high availability, security integration, and the seamless transition of on-premises SQL Server workloads to the cloud, making it an ideal resource for those planning their migration strategy.
We are excited to announce the Public Preview of Microsoft Entra ID (formerly Azure Active Directory) integration with Cloud SQL for SQL Server. Designed to tackle the challenge of identity sprawl in multi-cloud environments, this integration allows organizations to govern database access using their existing Microsoft identity infrastructure. Key benefits include centralized identity management, enhanced security features like Multi-Factor Authentication (MFA), and simplified user administration through direct group mapping. This feature is available for SQL Server 2022 and supports both public and private IP configurations.

January 12 - January 16

Google-built JDBC Driver for BigQuery is now available in Preview
We are excited to announce the launch of the new, Google-built JDBC driver for BigQuery. This new open-source driver provides a direct, high-performance connection for Java applications to BigQuery and is developed entirely in-house by Google. Download a new driver and connect your Java application to BigQuery.
Troubleshoot Airflow tasks instantly with Gemini Cloud Assist investigations: Cloud Composer just got smarter. We are excited to announce that Gemini Cloud Assist investigations are now available directly within Cloud Composer 3. Instead of manually sifting through raw logs, you can now simply click "Investigate" on a failed Airflow task. Gemini analyzes logs and task metadata to identify failure patterns—such as resource exhaustion or timeouts—and provides actionable recommendations driven by Gemini Cloud Assist to resolve the issue. This integration shifts the debugging experience from manual toil to automated root cause analysis, significantly reducing the time required to restore your pipelines. Learn more about AI-assisted troubleshooting.

vSphere and BRICKSTORM Malware: A Defender's Guide

Thu, 02 Apr 2026 14:00:00 +0000

Written by: Stuart Carrera

Introduction

Building on recent BRICKSTORM research from Google Threat Intelligence Group (GTIG), this post explores the evolving threats facing virtualized environments. These operations directly target the VMware vSphere ecosystem, specifically the vCenter Server Appliance (VCSA) and ESXi hypervisors. To help organizations stay ahead of these risks, we will focus on the essential hardening strategies and mitigating controls necessary to secure these critical assets.

By establishing persistence at the virtualization layer, threat actors operate beneath the guest operating system where traditional security protections are ineffective. This strategy takes advantage of a significant visibility gap, as these control planes do not support standard endpoint detection and response (EDR) agents and have historically received less security focus than traditional endpoints.

This activity is not the result of a security vulnerability in vendors' products or infrastructure. Instead, these intrusions rely on the effectiveness of exploiting weak security architecture and identity design, a lack of host-based configuration enforcement, and limited visibility within the virtualization layer. By operating within these unmonitored areas, attackers can establish long-term persistence and gain administrative control over the entire vSphere environment.

Figure 1: BRICKSTORM vSphere attack chain

This guide provides a framework for an infrastructure-centric defense. To help automate some of this guidance and secure the control plane against threats like BRICKSTORM, Mandiant released a vCenter Hardening Script that enforces these security configurations directly at the Photon Linux layer. By implementing these recommendations, organizations can transform the virtualization layer into a hardened environment capable of detecting and blocking persistent threats.

vCenter Server Appliance Risk Analysis

The vCenter Server Appliance (VCSA) is the central point of control and trust for the vSphere infrastructure. Running on a specialized Photon Linux operating system, the VCSA typically hosts critical Tier-0 workloads, such as domain controllers and privileged access management (PAM) solutions. This means the underlying virtualization platform inherits the same classification and risk profile as the highly sensitive assets it supports.

A compromise of the vCenter control plane grants an attacker administrative control over every managed ESXi host and virtual machine, effectively rendering traditional organizational tiering irrelevant. Because the VCSA is a purpose-built appliance, relying on out-of-the-box defaults is often insufficient; achieving a Tier-0 security standard requires intentional, custom security configurations at both the vSphere and the underlying Photon Linux layers.

For a threat actor, the VCSA provides:

Centralized Command: This provides the ability to power off, delete, or reconfigure any virtual machine combined with the ability to reset root credentials on any managed ESXi host providing full control of the hypervisor.
Total Data Access: Access to the underlying storage (VMDKs) of every application, bypassing operating system permissions and traditional file system security. This provides a direct path for data exfiltration of Tier-0 assets.
Command-Line Logging Gaps: If an attacker gains access to the underlying Photon OS shell via Secure Shell (SSH), there is no remote logging of the shell commands.

Management Plane Dependencies

Many organizations host their Active Directory domain controllers as virtual machines (VMs) within the same vSphere cluster managed by a vCenter that is itself AD-integrated. If an attacker disables the virtual network or encrypts the datastores, vCenter loses its ability to authenticate administrators. In a scenario where the VCSA is encrypted or wiped, the tools required for large-scale recovery are also lost. This forces organizations to rely on manual restores via individual ESXi hosts, extending the recovery timeline exponentially.

vSphere 7 End of Life

vSphere 7 reached End of Life (EoL) in October 2025. Organizations with this legacy technical debt will have vSphere software entering a window (until upgrade) where they will no longer receive critical security patches. This provides an opportunity for threat actors to exploit known vulnerabilities that will not be fixed.

The Strategic Advantage of Proactive Measures

To secure the control plane, organizations should adopt a strategy where the infrastructure itself acts as the primary line of defense.

A resilient defense relies on two strategies:

Technical Hardening: Defense-in-depth should be applied to the hypervisor layer to reduce the attack surface. Threat actors target insecure defaults. Hardening measures, such as enabling Secure Boot, strictly firewalling management interfaces, and disabling shell access, create “friction.” When a threat actor attempts to write a persistence script to /etc/rc.local.d or modify a startup file, a hardened configuration can block the action or force the actor to use methods that generate excessive log telemetry.
High-Fidelity Signal Analysis: Threat actors are adept at rotating infrastructure and recompiling tools to change their signatures. Relying on a blocklist of bad IPs or a database of known malware hashes is not an effective strategy as threat actors utilize command-and-control servers and native binaries. Instead, the focus should shift entirely to behavioral patterns.

Building on this strategic foundation where the infrastructure itself acts as the primary line of defense, this guide outlines four phases of technical enforcement:

Phase 1: Benchmarking and Base Controls – Establishing the foundation with Security Technical Implementation Guides (STIG) and patching.
Phase 2: Identity Management – Hardening administrative access to critical infrastructure via PAWs and PAM solutions.
Phase 3: vSphere Network Hardening – Eliminating lateral movement with Zero Trust networking.
Phase 4: Logging and Forensic Visibility – Transforming the appliance into a proactive security sensor.

Phase 1: Benchmarking and Base Controls

Organizations should use the hardening measures outlined in the Mandiant vSphere hardening blog post combined with a strict patching and upgrade strategy. This provides a standard foundation to develop a strong security posture. By implementing an enhanced security baseline centered on the Photon Linux DISA STIG and VMware security hardening guides, organizations can harden the OS-level components that actors target.

Key Frameworks:

STIG Control Mappings to Attacker TTPs

STIG ID	Control Title	TTP	Detail
V-258910	Require Multi-factor authentication (MFA)	Establish Foothold / Privilege Escalation	MFA on vCenter web login prevents compromised Active Directory credentials from granting full access.
V-256337	Real-time Alert on SSO Account Actions	Persistence / Anti-Forensics	Creates local accounts, deploys backdoors, and deletes the accounts within minutes. Real-time alerting on PrincipalManagement events is required to catch this activity.
V-258921	Verify User Roles (Least Privilege)	Data Exfiltration	Identifies and removes excessive permissions from standard user roles that are aggregated into non-admin roles.
V-258956	Limit membership to "BashShellAdministrators"	Escalate Privileges	Even if an attacker compromises a vSphere Admin account, they cannot access the Photon OS bash shell unless that account is in this specific single sign-on (SSO) group. It blocks the "VAMI-to-Shell" pivot used to deploy backdoors.
V-258968	Disable SSH Enablement	Initial Access	Actors often use the VAMI (Port 5480) to enable SSH before deploying the backdoor. This control ensures that SSH is "Disabled."

STIG controls mapping

vSphere Infrastructure-Level Data Exfiltration

Standard vSphere configurations typically mask high-risk permissions such as VM cloning and exporting within generalized administrative roles, allowing these actions to blend into the background noise of routine operations. This architecture provides a threat actor with the means to execute a silent exfiltration of a domain controller or credential repository. Organizations should transition from a model of permissive vSphere access control to a comprehensive cryptographic enforcement policy.

Security Control	What It Protects Against	Implementation Method
vSphere VM Encryption	Theft of VMDK files from the datastore; offline analysis and snapshot of memory	Enable in VM Policies (Requires a KMS)
In-Guest Encryption (BitLocker)	Mounting the VMDK to another VM; offline file system browsing	Enable inside Windows OS (Requires a vTPM)
vMotion Encryption	Capture of in-memory credentials (krbtgt hashes) during live migration	Set vMotion to "Required" in VM Options
Virtual TPM (vTPM) & Secure Boot	Bootkit persistence and tampering; strengthens in-guest features like Credential Guard	Enable in VM Options (Hardware & Boot sections)
Lock Boot Order & BIOS	Booting from a malicious ISO to reset passwords or bypass security controls	Set a VM BIOS password and configure boot options
Disable Copy/Paste	Silent data exfiltration of credentials or secrets via the VM console	Set VM Advanced Settings (`isolation.tools.* = true`)

Recommended controls for data exfiltration mitigation

Resilience against vSphere data exfiltration requires a shift in how high-value virtual assets are governed:

Mandatory Tier-0 Encryption: The enforcement of vSphere-native VM encryption is the primary and most essential control for all critical Tier-0 virtual machines. Organizations should mandate that every domain controller, certificate authority, and password vault be encrypted at the virtual machine level.
Cryptographic Isolation: Tier-0 assets should be subject to a unique key-locked encryption policy. By mandating a separate key management server (KMS) cluster for these workloads, organizations ensure that a threat actor cannot unlock a cloned disk without access to a secure, hardware-backed vault.
Entitlement De-coupling: The "Clone" and "Export" privileges should be stripped from standard administrative roles. These functions should be reassigned to a highly restricted, auditable "break-glass" identity, used exclusively for emergency recovery scenarios.

Phase 2: Identity Management

Best practices for Identity management in vSphere focuses on mandating all vSphere administrative sessions originate from dedicated privileged access workstations and utilize a PAM while also enforcing host-level hardening through the restriction of the vpxuser shell access.

Privileged Access Workstations (PAWs)

To prevent a threat actor from pivoting to the virtualization management plane from compromised user endpoints or appliances, administrative sessions should originate from a dedicated PAW. This is a dedicated hardened workstation only utilized when interfacing with vSphere administrative functions or interfaces.

Privileged Access Management (PAM)

PAM tools serve as an intermediary to mitigate specific threats such as the BRICKSTEAL credential harvester. By mandating credential injection, organizations ensure that passwords are never typed or exposed in memory on the target system where malware could intercept them. Automated secret rotation should be enforced to limit the lifespan of any compromised credentials, particularly for root passwords and service account keys.

Authentication and Platform Hardening

Accounts residing in the default vsphere.local single sign-on (SSO) domain, most notably the built-in administrator@vsphere.local superuser, pose a specific security risk because they do not support modern MFA integration. Due to this limitation, organizations should limit the use of vsphere.local accounts for daily administration; instead, they should be treated as emergency "break-glass" credentials that are secured with complex, vaulted passwords.

The vSphere VPXUSER

The vpxuser is a high-privilege system account provisioned by vCenter on each managed host to facilitate core infrastructure management operations.

A threat actor possessing administrative control over the VCSA effectively inherits the delegated authority of the vpxuser across the entire managed cluster. This entitlement enables a pivot from the management plane to the host-level shell.

The Primary Mitigation (vSphere ESXi 8.0+): Disabling Shell Access

To mitigate this lateral movement vector, vSphere 8.0 introduced a technical control allowing administrators to remove shell access from the vpxuser account. Enforce the following configuration on all ESXi 8.0+ hosts to restrict the vpxuser identity:

esxcli system account set -i vpxuser -s false

ESXi Host Identity Hardening Strategy

Additional hardening measures to prevent bypasses via alternative mechanisms, such as Host Profile manipulation, include:

Control Type	Strategic Requirement	Implementation Method
Pivot Mitigation	VPXUSER Shell Lock	Disable shell access for the management account to sever the vCenter-to-Host attack path.
Account Obfuscation	Rename root Account	Transition the default `root` identifier to a unique, non-predictable string to invalidate automated brute-force attempts.
Credential Entropy	15+ Character Baseline	Enforce a strict, system-wide password complexity policy using `Security.PasswordQualityControl`.
Vaulted Identity	Secure Credentials	Mandate the use of an enterprise password vault for all local host credentials to ensure auditable "break-glass" access.

ESXi host hardening

Phase 3: vSphere Network Hardening

Securing the Virtualization Network

Establishing a vSphere Zero Trust network posture is the foundational requirement for securing a resilient Tier-0 architecture. Because the vCenter Server Appliance (VCSA) and ESXi hypervisors lack native MFA support for local privileged accounts, identity-based validation is insufficient as a singular point of security enforcement. Once a threat actor harvests these credentials, the logical network architecture remains the only defensive layer capable of preventing the threat actor's access to the vSphere management plane.

A strictly segmented architecture integrating physical network isolation with host-based micro-segmentation serves as the definitive safeguard; by systematically eliminating all logical network paths from untrusted zones to the management zone, the underlying attack vector is neutralized, ensuring that a BRICKSTORM intrusion remains physically and logically incapable of compromising the vCenter control plane.

The architectural blueprint shown in Figure 2 is designed to eliminate these common internal attack vectors.

Figure 2: vSphere Zero Trust networking and detection

1. Immutable Virtual Local Area Network (VLAN) Segmentation

Organizations should enforce isolation through distinct 802.1Q VLAN IDs. Threat actors will exploit "flat" or poorly partitioned networks where a compromise in a low-security/low-trust zone (such as a demilitarized zone [DMZ] or edge appliance) can route directly to the Management VAMI (Port 5480) or shell access to the VCSA (Port 22) high-trust network segments.

VLAN	Description	Members	Strategic Security Policy
Host Management	ESXi Hypervisor Control Plane	ESXi vmk0 Management Interfaces	Restricted Access. Exclusively accepts traffic from the VCSA and authorized PAWs.
VCSA / Infrastructure	Cluster Management Applications	vCenter (VCSA), Backup Servers, NSX Managers	Tier-0 Restricted Zone. Should be logically and physically unreachable from all Guest VM segments.
vMotion	Live Memory Migration	ESXi vmk1 (vMotion Stack)	Non-Routable. Prevents interception of unencrypted RAM data during migration.
Storage	vSAN / iSCSI / NFS	ESXi vmk2 (Storage Stack)	Non-Routable. Critical for block-level data integrity; prevents out-of-band disk manipulation.
Virtual Machine	Production Workloads	Virtual Machine Port Groups	Untrusted Zone. Entirely isolated from all infrastructure management VLANs.

Layer 2 segmentation

2. Routing as a Security Barrier

The objective is to transform the Management Network into a secured zone. A threat actor residing on a standard corporate subnet or Wi-Fi network should be physically unable to communicate with the VCSA.

A. Virtual Routing and Forwarding (VRF) Segmentation

Action: Transition all Infrastructure VLANs into a dedicated VRF instance on the core routing layer.
Strategic Impact: This creates a defined routing table. Even in the event of a total compromise in the "User" or "Guest" VRF, the network hardware will have no route to the "Management" VRF, preventing lateral movement even if physical adjacency exists.

B. Privileged Admin Workstation (PAW Exclusive Access)

Action: Deconstruct all direct routes from the general corporate LAN to the Management Subnet(s).
Strategic Impact: Access to the Management Subnet should originate from a designated PAW IP range / subnet. All other internal subnets including standard user workstations, and guest VMs should have no route or be subject to an explicit Deny policy at the gateway. This forces the threat actor to attempt a compromise of the PAW, a significantly more hardened and monitored target, before they can connect to the VCSA.

3. Hardened Perimeter Ingress and Egress Filtering

These rules should be enforced at the hardware firewall or Layer 3 Core acting as the gateway for the Management Subnet. Because the VCSA's GUI-based native firewall is architecturally incapable of enforcing egress (outbound) policy, the upstream network gateway should enforce this policy. Organizations should implement a restrictive egress policy to ensure that if a VCSA is compromised, it cannot connect to malicious command-and-control infrastructure or exfiltrate Tier-0 data.

A. Ingress Filtering (Incoming to Management)

Source	Destination	Protocol / Port	Policy	Mitigation
PAW	Mgmt VLAN	TCP / 443	ALLOW	Authorized vSphere Client/API Access
PAW	ESXi VLAN	TCP / 902	ALLOW	Secure Remote Console (MKS) Access
ESXi	VCSA IP	TCP / 443	ALLOW	ESXi Host to vCenter communication
Backup	VCSA IP	TCP / 443	ALLOW	Backup API Access
Monitoring	Mgmt VLAN	ICMP Ping UDP / 161 (SNMP)	ALLOW	Verified Infrastructure Health Probes
ANY	Mgmt VLAN	TCP / 22	DENY	MANDATORY SSH BLOCK. Enforce shell access via PAW only.
ANY	Mgmt VLAN	TCP / 5480	DENY	MANDATORY VAMI BLOCK. Prevents unauthorized management enablement.
Guest VM	Mgmt VLAN	ANY	DENY	Eliminates all East-West lateral movement paths

Ingress filtering

B. Egress Filtering (Outbound from VCSA/Management)

Source	Destination	Protocol / Port	Policy	Mitigation
VCSA	Internal DNS	UDP/TCP 53	ALLOW	Restrict DNS to trusted internal resolvers only.
VCSA	Remote Syslog	TCP / 6514	ALLOW	TLS Encrypted Telemetry. Required for SIEM visibility
VCSA	Public IP for VMware Update Manager	TCP / 443	ALLOW	Strictly limit to "162.159.140.167" and "172.66.0.165" (VMware Update servers).
VCSA	Identity Provider	TCP / 443	ALLOW	Required for Federated Authentication (Okta/Entra)
VCSA	Internal Subnets	ANY	DENY	Block Internal Scanning. Prevents VCSA-to-Internal pivots.
VCSA	Internet (ANY)	ANY	DENY	Suppresses C2. Blocks DoH, SOCKS proxies, and data exfiltration.

Egress filtering

Note on Micro-Segmentation: While physical firewalls secure the management plane (North-South), VMware NSX Distributed Firewall (DFW) is the required standard for controlling guest-to-guest (East-West) traffic. Where applicable, NSX should be used to protect the data plane, while physical network hardware remains the control of the management plane.

Host-Based Firewalls for VCSA and ESXi

Host-based firewalls should be used in tandem with network-based firewalls to achieve a resilient defense-in-depth posture. While network firewalls effectively manage "North-South" traffic (entering/leaving the subnet), they are blind to "East-West" traffic within the same VLAN. Host-based firewalls are capable of blocking an attacker sitting on the same network segment. By enforcing security at the individual endpoint, organizations can ensure that the access path does not grant logical authority over the vSphere control plane.

The VCSA Host-Based Firewall (Photon OS)

Managed via the Virtual Appliance Management Interface (VAMI), the VCSA firewall is a native control to prevent lateral movement from compromised "trusted" entities such as backup servers or monitoring devices that share the management VLAN. The firewall should be used as a primary layer of defense to enforce the "principle of least privilege" at the host network level.

Strategic Implementation: The default policy should be transitioned to "Default Deny." You should explicitly define authorized IP addresses for every management service.

Recommended VCSA Host-Based Firewall Scoping

Port	Protocol	Source	Detail
UI / API (443)	TCP	PAW IP + Backup IP	Restricts vSphere Client access to hardened Admin stations.
VAMI (5480)	TCP	PAW IP Only	Prevents unauthorized SSH enablement or log tampering.
SSH (22)	TCP	PAW IP Only	Eliminates the primary shell residency path.
Heartbeat (902)	UDP	ESXi Management Subnet	Required for continuous Host-to-vCenter synchronization.
Internal (LADB)	TCP	Localhost (127.0.0.1)	Protects local inter-process communication.
ANY / ANY	ANY	DENY ALL	Blocks all unauthorized internal discovery.

VCSA host-based firewall

Limitations of the VAMI GUI Firewall

While the host-based firewall in the VCSA is a mandatory component of a defense-in-depth strategy, administrators should recognize that the standard VAMI GUI has the following operational limitations for defending against threat actors:

Lack of Port-Specific Granularity:The VAMI GUI lacks the precision required for a True Zero Trust model. In all versions, creating an IP-based rule for a specific server (e.g., a virtual backup server) forces an "all-or-nothing" approach. To grant that server legitimate access to the vSphere API on TCP 443, the administrator is often forced to trust that IP for all ports.

The Risk: This simultaneously grants the backup server unauthorized access to highly sensitive management interfaces like SSH (22) and the VAMI (5480). If an attacker compromises the backup server, they inherit an unobstructed management path to the VCSA shell.

Circular Administrative Dependency:A fundamental weakness of the native vCenter host-based firewall is its logical placement within the management plane it is intended to secure. The firewall is managed via the VAMI, which represents a secondary management entry point residing on TCP port 5480. This interface is logically adjacent to the standard vSphere Client (TCP port 443) and is frequently exposed across the same management network segments.

The Risk: Credentials captured via BRICKSTEAL grant a threat actor authority to reconfigure the appliance itself. By pivoting to the VAMI, the actor can use their compromised role to deactivate the firewall. This circular dependency ensures the firewall is managed by the very application it is intended to protect, allowing a threat actor to disable controls using the system's own management tools.

Forensic Visibility Gaps:The standard VAMI firewall is designed for connectivity management, not security monitoring. It does not generate remote logs for denied connection attempts or specific shell activity.

The Risk: This blinds security teams to active lateral movement. A threat actor can scan the VCSA from an unauthorized VM multiple times or use a VCSA shell unmonitored; because the firewall does not notify when it blocks a connection and shell commands are not logged, the SOC remains unaware of the intrusion attempt until the final stage of the attack.

Inbound-Only Policy Visibility Gaps:The GUI focuses primarily only on inbound traffic, leaving the Outbound (Egress) policy unmanaged.

The Risk: Modern malware, such as the BRICKSTORM backdoor, relies on outbound "Phone Home" (C2) traffic to receive commands. A firewall that does not restrict outbound traffic allows a compromised VCSA to communicate with external malicious infrastructure without restriction.

To overcome these limitations of the native VAMI firewall, organizations are recommended to consider the transition from native vSphere GUI-based management to OS-level hardening using the underlying Photon Linux iptables or nftables.

Tamper-Proof Integrity: By implementing granular firewall rules directly at the Photon Linux operating system level, the controls become independent of vCenter application permissions. Even a compromised vCenter Administrator cannot disable Photon OS-level rules via the VCSA GUI.
Granular Logic: OS-level rules allow for strict "Source IP + Destination Port" mapping, ensuring a backup server only sees port 443 and is rejected on all others.
Transformation into a Sensor: Unlike the VCSA GUI, Photon OS-level logging can be "bridged" to a security information and event management (SIEM) which transforms every denied connection attempt into a high-fidelity, early-warning alert.

The VAMI GUI firewall should be viewed as a basic security control, not a comprehensive Tier-0 security control. To effectively mitigate the attack vectors required for advanced campaigns, organizations should bypass the vulnerable GUI and enforce a strictly validated, granular, and logged firewall policy at the VCSA Photon Linux kernel level.

aside_block: <ListValue: [StructValue([('title', 'vCenter Hardening Script'), ('body', <wagtail.rich_text.RichText object at 0x7f50781e4880>), ('btn_text', 'Get the tool!'), ('href', 'https://github.com/mandiant/vcsa-hardening-tool'), ('image', None)])]>

The ESXi Hypervisor Firewall

The ESXi firewall is a stateful packet filter sitting between the VMkernel and the network. Restricting individual services to authorized management IPs is the only way to block an attacker on the same VLAN from reaching the host API or SSH port.

Strategic Implementation: Access should be restricted at the service level by deselecting "Allow connections from any IP address" and entering specific management IPs.

Recommended ESXi Host-Based Firewall Rules

Service Category	Service Name	Port / Protocol	Authorized Source	Strategic Defensive Value
Management Access	SSH Server, vSphere Web Client/Access	22, 443 / TCP	PAW Subnet / IPs only	Ensures shell and GUI access is restricted to hardened admin PAWs.
vCenter Control Plane	vCenter Agent (vpxa), Update Manager	902, 80 / TCP	VCSA IP Only	Prevents unauthorized entities from impersonating the VCSA.
Intra-Cluster	vMotion, HA, Fault Tolerance, DVSSync	8000, 8182 / TCP, 12345 / UDP	ESXi Mgmt Subnet / IPs	Prevents interception of unencrypted RAM data and heartbeat tampering.
Storage	NFC (File Copy), HBR (Replication)	902, 31031 / TCP	VCSA IP + Cluster IPs	Prevents unauthorized VMDK extraction or out-of-band data cloning.
Telemetry	Syslog, SNMP, NTP, DNS	514, 161, 123, 53 / UDP	SIEM & Infra Subnets	Ensures telemetry and core services are bound to verified internal providers.
Legacy / High Risk	CIM Server, SLP (Discovery)	5988, 5989 / TCP, 427 / UDP	EXPLICIT DENY / Monitoring IP	Neutralizes RCE vectors targeting the primary attack surface used for ESXi-specific ransomware (VMSA-2021-0002).

ESXi host-based firewall

Hardening as a Detection Enabler

When the infrastructure is configured with a "Default Deny" posture, it creates the friction necessary to expose a threat actor. In an unhardened environment, an attacker's port scan or lateral movement attempt is silent and successful; in a hardened environment, those same actions become indicators of compromise.

The Multi-Layered Signal Chain

Network-Level Visibility: Detection begins at the transit layer. Organizations should ensure that logging is enabled at the physical network and virtual switch (VDS) levels. This allows the SOC to track the "path" of a threat actor, identifying unauthorized scanning or connection attempts as they traverse subnets toward the vSphere management plane.
Host-Based Firewall Logging (IPtables): While the VCSA provides a management GUI for its firewall, it does not natively log denied access. To transform the appliance into a sensor, host-based firewall logging is strictly dependent on a custom OS-level IPtables configuration. By adding a logging target to the underlying Photon OS kernel, every rejected packet is recorded, providing the proof that an unauthorized threat actor is attempting to access the VCSA.
Immutable Logging: By enabling Remote Syslog Forwarding, these rejection logs are offloaded instantly. Even if an attacker eventually compromises the host, they cannot delete the local log sources.

Early Detection Signals

By correlating the denied access with identity-based events, organizations can identify a pattern of a BRICKSTORM lifecycle event in its earliest stages:

Failed Authentication Alerts: A log entry in the standard auth.log (for SSH) or a vCenter UserLoginSessionEvent showing a "Failed Login Attempt" from an unauthorized internal IP is a high-value alert.
Account Lockout Events: When an actor attempts to brute-force or use harvested credentials against local "break-glass" accounts (like administrator@vsphere.local), the resulting "Account Locked" event provides a high-priority signal that a targeted credential attack is in progress.
Behavioral Pattern Correlation: The most powerful signal occurs when the SIEM correlates these disparate sources. For example, a Firewall Drop (via IPtables) followed immediately by a Failed Login (via SSO) from the same source IP is a high-confidence indicator of an active intrusion attempt.

Network segmentation at the switch level is a prerequisite, but host-based firewalls are the primary enforcement point of a vSphere Zero Trust architecture. By complementing network-based firewalls with host-level filtering, organizations can eliminate the visibility gap on the management VLAN and transform the VCSA and ESXi hosts into sensors capable of exposing an adversary at the earliest stage of an intrusion.

Phase 4: Logging and Forensic Visibility

To facilitate the detection within the vSphere control plane, organizations should achieve comprehensive telemetry across the previously unmonitored layers of the underlying VCSA operating system.

The primary operational advantage exploited in this campaign is the lack of visibility inherent in the virtualization control plane. This monitoring visibility gap is driven by three critical factors:

The Logging Gap: By default, VCSA does not forward kernel-level audit logs. If an attacker wipes the local disk, the evidence of their residency is permanently erased.
The Restricted Logging Pipeline: Standard modern log forwarding agents such as Fluentd or Logstash are not supported for installation on the VCSA. To maintain appliance integrity, defenders are restricted to using the native rsyslog daemon. This prevents on-host log enrichment or advanced parsing, forcing the SIEM to process raw, legacy data streams. This technical complexity often leads to critical kernel-level signals being misclassified or ignored.
Operational Telemetry Fragmentation: Security indicators are frequently buried within standard cluster and application level events. As detailed in the vCenter Event Mapping, critical actions like VmNetworkAdapterAddedEvent or VmClonedEvent are logged as routine infrastructure management tasks. Because these signals are operational rather than security-focused, a threat actor's movements are easily disguised as routine tasks.

Securing the VCSA requires a transition from passive cluster monitoring to active OS-level hardening, utilizing a 'Default Deny' posture to eliminate the network path often exploited during advanced campaigns. This architectural shift transforms the appliance into a proactive security sensor, where the friction of blocked network activity and initial access serves as a high-fidelity indicator. By moving beyond complex vSphere application telemetry, organizations can generate the precise early warning signals needed to expose a BRICKSTORM intruder at the very moment they attempt unauthorized discovery.

What is auditd?

The Linux Audit Daemon (auditd) is the kernel's primary subsystem for tracking security-relevant events. Unlike standard "system logs" (which record application and management events), auditd records system calls. It sees exactly what commands were executed in the shell, which files were modified, and which users escalated privileges. The default Photon auditd rules cover Identity (useradd/del) and privilege escalation (sudo/privileged).

auditd Status: Verifying the Current Defensive Posture

auditd is the core forensic foundation for detecting low-level movements. While VCSA Photon logs provide visibility into management tasks, they are fundamentally blind to the "living-off-the-land" (LotL) techniques that define this campaign. This threat actor operates deep within the VCSA shell to execute binary injections, modify startup scripts using sed, and utilize sudo to fuel the BRICKSTEAL credential harvester. Only auditd, by recording the underlying system calls (syscalls), provides a granular record of these command-line maneuvers. In an environment where traditional EDR is absent, auditd captures the minute behavioral patterns that standard logs ignore.

The Default Configuration Gap

Modern VCSAs (vSphere 7 and 8) ship with a pre-configured set of STIG rules (located in /etc/audit/rules.d/audit.STIG.rules). However, there is a restriction in the default configuration:

Local Only: By default, auditd writes to a local file (/var/log/audit/audit.log).
Invisible to VAMI: The remote logging you configure in the VAMI (Port 5480) does not include these kernel logs by default.
The Attack Vector: Actors can gain root access, perform their actions, and simply run rm -rf /var/log/audit/* to delete the evidence. Unless these logs are streamed to your SIEM in real time, your forensic trail is non-existent.
Local Log Rotation: Since the local log location is /var/log/audit/audit.log, it is subject to rotation and deletion. If an attacker wipes this file, the remote syslog version is your only forensic record.

All auditd logs should be forwarded via the VCSA remote syslog. Remote forwarding of auditd is dependent on a "auditd bridge" configuration. If /etc/audisp/plugins.d/syslog.conf is set to active = yes, these logs will be tagged and forwarded. If set to no, they are stored locally only. To enable remote logging of auditd events and ensure forensic persistence, the following steps should be taken:

Step A: Check Service and Rule Status

Before activating the auditd remote logging bridge, you should determine if your VCSA is currently configured for auditd. Run these commands as root:

# 1. Check if the audit service is active
systemctl status auditd

# 2. List the rules currently enforced by the kernel memory
auditctl -l

If auditctl -l returns nothing, your rules have not been loaded, and the kernel is not "watching" for attacker behavior.

Step B: Check the "auditd Bridge" Status

Verify if kernel events are stored on the local disk or being forwarded to your remote SIEM.

# Check the active status of the syslog plugin
# Note: vSphere 8 still uses the /etc/audisp/ path for compatibility
grep "^active" /etc/audisp/plugins.d/syslog.conf

If this returns active = no, remote logging of auditd is not configured. The logs are sent only to the VCSA local disk where an attacker can easily wipe them.

Mapping Standard STIG Rules to Attacker TTPs

If your auditctl -l output shows the standard rules are now loaded, you have the following rules in place mapped to identified attacker tactics, techniques, and procedures (TTPs). These rules move you from periodic auditing or threat hunting to real-time behavioral detection.

Standard STIG Rule / Key	TTP Phase	Defensive Value
-k useradd / -k userdel	Establish Foothold	Creates local accounts, deploys backdoors, and deletes them within ~13 minutes. These rules log both ends of this rapid lifecycle.
-k execpriv (execve syscalls)	Binary Execution	Triggers when the actor executes unauthorized binaries (e.g., `pg_update`, `vmp`) with root privileges.
-k perm_mod (chmod, chown)	Weaponization	Actors use sed to inject code into startup scripts and then run `chmod +x`. This rule triggers the second the script is made executable.
-k privileged (sudo, su)	Credential Theft	`BRICKSTEAL` requires sudo to scrape memory and config files. This logs the original user ID even if they escalate to root.
-k modules (init_module)	Establish Persistence	Logs attempts to load malicious kernel modules or persistence drivers into the Photon OS.
-k shadow / -k passwd	Anti-Forensics	Logs any manual edits to the system's identity files used to create "trapdoor" root users.

Mapping of STIG rules

Activating Remote Logging for auditd

Step 1: Enable the Syslog Plugin

The Audit Dispatcher (audisp) should be configured to send events to the local syslog service so they can be forwarded via the VCSA remote syslog.

# Use sed to change the status from 'no' to 'yes'
sed -i 's/^active = no/active = yes/' /etc/audisp/plugins.d/syslog.conf

# Verify the change
grep "^active" /etc/audisp/plugins.d/syslog.conf

Step 2: Restart the Audit Daemon

You should reload the service to initialize the dispatcher and the syslog bridge:

kill -HUP $(pidof auditd)

Step 3: Verify the Bridge Is Operational

Check the local system messages to ensure the plugin has started successfully:

grep "audisp-syslog" /var/log/messages

You should see a message indicating the plugin has initialized or started.

Step 4: Confirm Logs Are Forwarded

journalctl -f | grep audit

You should see events with msg=audit prefix.

Syslog Tag (Key): In your SIEM, you should search for the field msg=audit followed by the key="XYZ" (e.g., key="execpriv"). This allows you to filter out of standard system logs and focus only on high-fidelity security events.

Additional Auditd Rules

Based on a default audit.STIG.rules output contained in the Photon OS 4.0 STIG auditd config, these three rules should be added.

Recommended Rule Addition	TTP	Detail
-w /usr/bin/rpm -p x -k software_mgmt	Malware Deployment	Detects SLAYSTYLE: Logs the execution of the RPM installer. Essential for spotting the deployment of unauthorized tools or malicious packages.
-w /etc/init.d/ -p wa -k startup_scripts	Establish Persistence	Detects Startup Injections: Directly identifies the sed-based modifications used by threat actors to ensure backdoors survive a reboot.
-w /root/.ssh/authorized_keys -p wa -k ssh_key_tamper	Establish Persistence	Persistence Sensor: Any write (`w`) to the root SSH directory is inherently suspicious and detects the "trapdoor" persistence TTP.

Additional STIG-based rules

Advanced Intrusion Detection Environment (AIDE)

While auditd provides low-level monitoring, AIDE serves as the source of digital validation for the VCSA. AIDE is a host-based file integrity monitoring (FIM) tool that is considered the industry standard for high-security Linux environments and is a requirement for DISA STIG compliance (PHTN-40-000237).

Note: Mandiant recommends organizations perform comprehensive testing and fine-tuning of these rules within a staging environment before production deployment to account for variations in specific vSphere configurations and operational workloads. Proper calibration of monitoring thresholds and file exclusion lists is essential to achieve an optimal signal-to-noise ratio and ensure high-fidelity alerting of unauthorized modifications.

Why AIDE Is Essential Alongside auditd

Relying on a single telemetry stream is insufficient to counter the sophisticated tactics of BRICKSTORM. By pairing AuditD's behavioral auditing with AIDE's cryptographic integrity checks, organizations establish a mutual defense that reduces an attacker's ability to operate undetected.

auditd (Behavioral Monitoring): Captures the action (e.g., "Root used sed to modify a script"). If an attacker achieves high-level privileges and "blinds" the audit service or wipes the local logs, the behavioral trail is lost.
AIDE (State Monitoring): Captures the result. AIDE creates a cryptographic baseline (DNA fingerprint) of every critical system file. It does not care how a file was changed or if the audit logs were wiped; it only cares that the file is no longer authentic.

Using AIDE Alongside auditd

The following steps walk through how to verify the current AIDE integrity foundation, add BRICKSTORM specific detections, and establish an immutable cryptographic baseline.

1: Diagnostic Assessment

Before modifying the environment, you should confirm the AIDE configuration status. Log in to the VCSA via SSH and run these commands as root:

Confirm AIDE is installed and compiled with the required config (WITH_AUDIT and SHA-512).

# Check version and compiled options
aide -v

2. Verify the AIDE Database

AIDE requires that a cryptographic baseline (snapshot) exists. Check the status of the database:

# Resolve the database directory (typically /var/lib/aide)
grep "@@define DBDIR" /etc/aide.conf
# Check for the active database
ls -lh /var/lib/aide/aide.db.gz

If aide.db.gz is missing, you have no baseline. If it exists but the timestamp is months old, your integrity foundation is stale and will produce high-noise alerts during a check.

3. Audit Current AIDE Coverage

Determine which parent directories are currently being monitored by the default rules:

# Filter for active file selection rules
grep -v "^#" /etc/aide.conf | grep "^/"

4. Editing AIDE Rule Set for BRICKSTORM Coverage

Open the configuration file.

vi /etc/aide.conf

Append these BRICKSTORM specific rules to the bottom. Use the STIG rule group to ensure SHA-512 enforcement.

# --- BRICKSTORM TARGETS ---
/root/.ssh              STIG    # Detects unauthorized SSH
/lib64                  STIG    # Detects system-level libraries
/etc/aide.conf          STIG    # Detects tampering with AIDE
/etc/audit/             STIG    # Detects attempts to edit config
/etc/audisp/            STIG    # Detects attempts to sever bridge

Append the file for log exclusions to reduce noise [the ! should come before the rules that tell AIDE to watch the parent folders (like /opt or /etc)].

# --- NOISE REDUCTION: EXCLUDE DYNAMIC LOGS ---
!/var/log/.*             # Ignore all standard logs
!/opt/vmware/var/log/.*  # Ignore vCenter-specific service logs
!/var/lib/.*             # Ignore dynamic database/state files

Note: Remove all # from append statements.

5. Initializing the AIDE Database

Once the rules are defined, you should generate a new cryptographic snapshot. This should only be performed when the VCSA is verified clean (e.g., immediately after patching).

# 1. Initialize the new fingerprint database
aide --init

# 2. Activate the database
mv /var/lib/aide/aide.db.new.gz /var/lib/aide/aide.db.gz

Copy the aide.db.gz to a read-only, off-box location. Comparing the VCSA against an off-box "Gold Image" ensures that even root-level attackers cannot hide their modifications by re-initializing the local database.

6. Enable the Remote Logging of AIDE Events via Logger Pipe

# Run a check and bridge the output to Syslog/SIEM
aide --check | logger -t AIDE_TRAP -p local6.crit

7. Enable Automation of AIDE Database Check

To move from manual oversight to automated alerting, you should establish a recurring scheduled task. This ensures that the VCSA programmatically verifies its own state and reports any discrepancies.

Open crontab:

crontab -e

Add the following edit to configure the task:

# Execute check every 6 hours and send results via VCSA remote syslog
0 */6 * * * /usr/bin/aide --check | logger -t AIDE_TRAP -p local6.crit

8. Conduct a Test Event

To confirm your defense is operational and your SIEM is successfully receiving AIDE alerts, perform a simulated breach.

Add a comment to a monitored area (e.g., /etc/rc.local):

echo "# Forensic Bridge Test" >> /etc/rc.local

Trigger a remote event trap:

aide --check | logger -t AIDE_TRAP -p local6.crit

Verify the Alert: Check the VCSA remote syslog target for the tag AIDE_TRAP:

AIDE found differences between database and filesystem!! followed by Changed files: /etc/rc.local.

VCSA Shell History

On a Photon-based VCSA, the /root/.bash_history file is not replicated to any other log file, nor is it sent to a remote syslog by default. This represents a major forensic visibility gap that threat actors take advantage of to maintain their unmonitored persistence.

The Buffer Issue: Commands typed into the shell are kept in a memory buffer. They are only written (appended) to the physical file on the disk when the user logs out of the session.
The Anti-Forensics Risk: If a threat actor gains shell access, their first move is often to run unset HISTFILE or history -c. This prevents the memory buffer from ever being written to the disk. Even if the file is written, an attacker can simply run rm /root/.bash_history before exiting.
No Remote Transmission: Standard VCSA syslog configurations monitor directories like /var/log/. They do not monitor hidden user files like .bash_history.

The reason the auditd remote syslog discussed in the previous steps is so critical is that it bypasses the need for .bash_history entirely. auditd intercepts system calls (syscalls) at the kernel level and exfiltrates detailed forensic data including the original User ID (AUID) and command outcomes to a remote SIEM as the command is executed. This bridge ensures that even if a threat actor purges local logs or crashes the session, an immutable, real-time audit trail remains securely preserved off-appliance.

Logging Design Principles

Recent CISA reporting and GTIG analysis describe threat actors abusing management interfaces (including enabling SSH), making persistence-related configuration changes, and using vCenter capabilities to access high-value virtual machines. An organization's logging strategy should therefore prioritize management-plane audit trails, service-state changes, identity events, hypervisor telemetry, and centralized forwarding.

Centralize first, then tune. Forward logs off-host in near real time so an attacker cannot tamper with them by wiping local disks. Configure both VCSA and ESXi to forward to a central syslog/SIEM target.
Treat logs as Tier-0 data. If vCenter is Tier-0, then vCenter/ESXi logs are also Tier-0. Restrict who can read them, who can change forwarding settings, and who can stop logging services.
Make timestamps defensible. Ensure consistent Network Time Protocol (NTP) across VCSA, ESXi hosts, jump boxes, and log collectors so correlation is reliable during an incident.
Log the actions that matter, not everything. For threat actor activity, you care less about generic "system is running" noise and more about: who accessed management, what changed, what was cloned/exported, what services were enabled, what binaries/configs were modified, and where the appliance/host talked to on the network.

Organizations should establish a "vSphere logging fundamentals" previously described by Mandiant by offloading all infrastructure logs to a centralized, remote SIEM.

The vSphere Unified Logging Architecture

The following summary table provides a definitive map of the vSphere telemetry streams described. By implementing these steps, organizations can move from a single localized log to a multilayered remote detection architecture that covers the entire BRICKSTORM malware lifecycle.

Type	Forensic Layer	Signal Observed	TTP Phase	Detail
vCenter Application Events	Management Plane (API/UI)	Programmatic Event IDs: VmClonedEvent, VibInstalledEvent, HostSshEnabledEvent	Initial Access / Exfiltration	Tells you "What" high-level action was performed (e.g., a domain controller was cloned) and the Admin IP responsible.
Identity (SSO) Events	Identity Layer	Principal Events: com.vmware.sso.PrincipalManagement	Establish Persistence	Detects "Who" was created. Specifically catches the transient accounts used as deployment vehicles for backdoors.
AuditD Kernel Logs	OS Kernel (Photon OS)	Syscall Keys: key="execpriv", key="useradd", key="privileged"	Establish Persistence	Tells you "How" the shell was used. Captures commands typed by an intruder (e.g., sudo, sed, rpm) even if they delete their bash history.
AIDE Integrity	Filesystem	Syslog Tag: AIDE_TRAP stating: "differences found between database and filesystem"	Establish Persistence	Tells you "What was modified" to ensure residency. Detects physical changes to binaries and startup scripts that standard logs miss.
IPtables OS Firewall	Network Layer (Host-Based)	Kernel Message: VCSA_FW_DROP + Source IP + Destination Port	Initial Access / Lateral Movement	Tells you "Who is probing?". Identifies compromised internal VMs attempting to scan or brute-force VCSA management ports (SSH/VAMI).

vSphere VCSA logging

Implementation Best Practices

For both the VCSA and ESXi hosts, the implementation of remote syslog should move beyond legacy, unencrypted protocols. The following standards are required to ensure the integrity and survivability of the forensic trail:

Encryption via TLS (TCP Port 6514): Sending logs over UDP/514 is insecure and unreliable. Threat actors can access management traffic or spoof log entries. Organizations should enforce TCP with TLS encryption for all syslog traffic. This ensures that logs are encrypted in transit and guarantees delivery through the TCP handshake.
Certificate Validation: To prevent man-in-the-middle (MitM) attacks on the logging pipeline, the VCSA and ESXi hosts should be configured to validate the SSL certificate of the remote syslog server. This ensures that telemetry is being sent to a verified security authority and not a rogue listener controlled by the attacker.
VCSA Custom Shell Bridging: Because the VCSA does not forward shell activity or denied firewall connections by default, administrators should consider implementing an agentless bridge at the Photon OS level. By configuring the audisp (Audit Dispatcher) and piping iptables logs into the native rsyslog service, the VCSA is transformed from a passive appliance into an active sensor, capable of streaming real-time kernel-level alerts directly into the encrypted TLS pipeline.
Standardized Retention: Given this threat actor's dwell time averages 393 days, the remote syslog repository should be configured with a minimum retention period of 400 days. This allows investigators to correlate the programmatic eventTypeId of a year-old initial compromise with the low-level auditd signals of a current breach.

Summary of Logging Detections

Attack Phase	TTP	Key Forensic Log Source(s)	Technical Detail
Initial Access	Edge Appliance Exploitation	Tomcat Audit Logs: /home/kos/auditlog/fapi_cl_audit_log.log	Detects requests to /manager/text/deploy (CVE-2026-22769) to deploy malicious WAR files like SLAYSTYLE.
	Reconnaissance & Scanning	VCSA firewall_audit: SSH_BLOCKED_NEW, WEB_BLOCKED_NEW, VAMI_BLOCKED_NEW	Identifies attempts to probe management ports (22, 443, 5480) from unauthorized, non-whitelisted IPs.
Lateral Movement	Credential Abuse	Windows Event 4624 (Type 3); VCSA firewall_audit: ALLOWED SSH	Detects network logins from appliance IPs using stolen service account credentials.
	Stealth Pivoting (Ghost NICs)	vCenter Events: VmNetworkAdapterAddedEvent (8.0u3+) or VmReconfiguredEvent	VmNetworkAdapterAddedEvent is a high-fidelity "Critical" signal for bridging VMs into restricted networks. Legacy builds use VmReconfiguredEvent to track unauthorized NIC additions.
Takeover	Management Interface Access	VAMI Logs: /var/log/vmware/vami/vami-httpd.log	Records POST requests to /rest/com/vmware/cis/session followed by SSH enablement via PUT requests on port 5480.
	Interactive Shell Escape	SSO Audit (PrincipalManagement); VCSA SHELL_COMMAND	Monitors membership changes to BashShellAdministrators to escape VAMI to bash; tracks interactive commands like whoami or netstat.
Persistence	Startup Script Injections	AuditD Key -k startup_scripts; VCSA init files	Detects sed commands modifying /etc/sysconfig/init or /opt/vmware/etc/init.d/vami-lighttp.
	Transient SSO Accounts	SSO Audit (audit_events.log)	Rapid creation and deletion of local accounts (e.g., in vsphere.local) used solely for malware deployment.
	Filesystem Integrity / Binary	AIDE Monitor (AIDE_TRAP); AuditD Key -k execpriv	Detects physical changes to binaries in /lib64 or /root/.ssh and execution of unauthorized binaries like vmsrc.
	Rogue "Ghost VMs"	AUDIT log “vmx -x” /var/log/shell.log “/bin/vmx” /var/log/shell.log	Detection of unregistered virtual machine files (.vmx) hidden from standard management consoles.
Credential Theft	Tomcat Memory Scraping	vCenter Web Logs; AuditD Key -k privileged	Monitors HTTP requests to /web/saml2/sso/* (BRICKSTEAL); tracks sudo usage for scraping memory or DB credentials.
	Offline NTDS.dit Theft	vCenter Events; vCenter VPXD Logs; ESXi hostd.log	VmClonedEvent or VmBeingClonedEvent targeting domain controllers followed by VmDiskHotPlugEvent to mount disks offline to extract the ntds.dit database.
Exfiltration	C2 & Data Tunnelling	VCSA firewall_audit: INTERNET_BLOCKED, ZT_OUTBOUND_DENIED	Captures VCSA attempting unauthorized outbound calls to external C2 nodes via SOCKS proxies or DoH.

Mapping of logging and detections

Conclusion

It is critical for organizations to recognize that the vCenter Server control plane is a primary target for state-sponsored espionage and global ransomware operations. Technical hardening is essential to create the friction required to generate high-fidelity signals. By enforcing barriers such as VCSA OS-level firewalls, phishing-resistant MFA, and restricted management interfaces, organizations force a threat actor to attempt actions that are inherently suspicious.

Addressing forensic visibility gaps through the implementation of auditd, AIDE, and centralized remote logging ensures that evidence of persistence is preserved for incident response activities. Organizations should leverage this enhanced telemetry to build pattern-based behavioral detections rather than relying on static Indicators of Compromise (IoCs). As adversaries increasingly leverage AI across the entire attack lifecycle, the hardening and logging controls outlined in this guide should become the universal vSphere security baseline to ensure every unauthorized movement results in an immediate and immutable forensic response.