Managed Service for Apache Spark staging buckets

This document provides information about Managed Service for Apache Spark staging buckets. Managed Service for Apache Spark creates a Cloud Storage staging bucket in your project or reuses an existing staging bucket from previous batch creation requests. This is the default bucket created by Managed Service for Apache Spark clusters. For more information, see Managed Service for Apache Spark staging and temp buckets.

Managed Service for Apache Spark stores workload dependencies, config files, and job driver console output in the staging bucket.

Managed Service for Apache Spark sets regional staging buckets in Cloud Storage locations according to the Compute Engine zone where your workload is deployed, and then creates and manages these project-level, per-location buckets. Managed Service for Apache Spark-created staging buckets are shared among workloads in the same region, and are created with a Cloud Storage soft delete retention duration set to 0 seconds.