Encrypt memory with Confidential VM clusters

You can use the Confidential Computing feature to create a Managed Service for Apache Spark cluster with Confidential VMs to provide inline memory encryption.

Requirements

A cluster with Confidential VMs must meet following requirements:

  • Master and worker instances must use a machine type supported by the selected Confidential Computing technology. Supported machines associated with each technology include:
    • SEV: c4d, c3d, c2d, n2d
    • SEV_SNP: n2d
    • TDX: c3-standard-*, a3-highgpu-1g
  • The cluster must use one of the supported Ubuntu images.
  • The cluster must be created in a Compute Engine zone that supports the CPU platform required by the selected Confidential Computing technology—see View supported zones.
    • You can run the following command to list the CPUs supported in a Compute Engine zone: 
      gcloud compute zones describe ZONE --format="value(availableCpuPlatforms)"

Create a cluster with Confidential VMs

You can use the Google Cloud console, Google Cloud CLI, or the Managed Service for Apache Spark API to create a cluster that uses Confidential VMs.

Google Cloud console

To create a cluster that uses Confidential VMs, complete the following steps:

  1. Open the Create an Apache Spark cluster on Compute Engine page in the Google Cloud console.
  2. Under Define your cluster, select a supported zone.
  3. Under Advanced configurations - Infrastructure, select a supported master and worker node machine type.
  4. Under Advanced configurations - Security, select "Enable confidential computing".
  5. Confirm or specify other cluster settings, then click Create.

Google Cloud CLI

To create a cluster that uses Confidential VMs, use the gcloud dataproc clusters create command with the --confidential-compute-type flag.

gcloud dataproc clusters create CLUSTER_NAME \
    --confidential-compute-type=CONFIDENTIAL_COMPUTING_TECHNOLOGY (such as SEV, SEV_SNP, or TDX) \
    --image-version=UBUNTU_IMAGE_VERSION \
    --zone=ZONE \
    --master-machine-type=MACHINE_TYPE \
    --worker-machine-type=MACHINE_TYPE \
    other args ...

REST API

To create a cluster that uses Confidential VMs, include the ConfidentialInstanceConfig as part of a clusters.create request. Set confidentialInstanceType to the selected technology, such as SEV, SEV_SNP, or TDX.