Skip to content

beninanutshell/gdg-ace-demo

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GCP Associate Cloud Engineer (ACE) Demo Repository

Google Developers Group (GDG) Montreal
This repository is dedicated to participants of the GDG Montreal training program preparing for the Associate Cloud Engineer (ACE) certification exam.

This repository contains multiple hands-on demonstrations for learning how to deploy resources and applications on Google Cloud Platform. All demos assume a Shared VPC architecture where resources are provisioned in a service project and network resources are managed in a host project.

Repository: github.com/beninanutshell/gdg-ace-demo.git


Repository Structure

Cloud Run Functions (2nd gen) + Pub/Sub + BigQuery

Demo of a serverless receipt processing pipeline:

  • Trigger: Pub/Sub topic receives JSON receipts
  • Compute: Cloud Run Function (Python 3.12) processes messages
  • Storage: BigQuery table stores transformed data (partitioned and clustered)
  • IAM: Service account with least-privilege permissions

Key concepts: Event-driven architecture, Pub/Sub messaging, BigQuery data loading, Cloud Run Functions configuration.


Compute Engine + Managed Instance Group + Regional Load Balancer

Demo of a scalable web tier using GCE:

  • Compute: Regional Managed Instance Group (auto-scaling based on CPU)
  • Base image: Debian 12 + Apache2 + PHP
  • Load Balancing: Regional External Application Load Balancer (Layer 7)
  • Network: Shared VPC with separate subnets for instances and proxy-only
  • Monitoring: HTTP health checks + autoscaling policies

Key concepts: Instance templates, regional MIGs, autoscaling, external load balancers, Shared VPC, startup scripts.


Google Kubernetes Engine (GKE) Autopilot Cluster

Demo of a GKE cluster deployment with Shared VPC:

  • Cluster mode: Autopilot (fully managed nodes)
  • Network: Shared VPC with separate IP ranges for nodes, pods, and services
  • Security: Workload Identity, Binary Authorization, GKE Security Posture
  • IAM: Custom service account for node pools

Key concepts: GKE Autopilot, Shared VPC for GKE, secondary IP ranges, Workload Identity, cluster security.


Kubernetes Manifests for GKE Deployment

Kubernetes YAML manifests for deploying an application on GKE:

  • Deployment: Multi-replica Nginx deployment with pod anti-affinity and topology spread
  • Service: ClusterIP service for internal communication
  • Gateway API: HTTPRoute and Gateway for external access (modern Ingress alternative)
  • Autoscaling: Horizontal Pod Autoscaler (HPA) based on CPU/memory
  • Security: Pod security context, resource limits, health checks
  • Load testing: k6 script to test HPA behavior

Key concepts: Kubernetes deployments, Gateway API, HPA, pod topology spread, security best practices, load testing.


Cloud Run + Container Registry

Demo of a fully managed serverless container deployment:

  • Compute: Cloud Run service (scale-to-zero enabled)
  • Container: Nginx serving a static web page
  • IAM: Public access via allUsers invoker role
  • Scaling: Min 0, max 10 instances with concurrency=1 for fast scale-up

Key concepts: Cloud Run Gen2, containerization, scale-to-zero, public service exposure, Artifact Registry.


Python Source Code for Cloud Run Function

Python application code for the serverless function demo:

  • Framework: Cloud Run Functions runtime (Python 3.12)
  • Dependencies: Managed via requirements.txt
  • Purpose: Receipt processing and BigQuery data insertion

Prerequisites

All demos require:

  1. GCP Organization with billing enabled
  2. Two GCP projects:
    • Host project: Contains the Shared VPC network
    • Service project: Where compute resources are provisioned (attached to Shared VPC)
  3. Terraform >= 1.6
  4. gcloud CLI configured and authenticated
  5. Docker (for building container images)

Container Image Sources

Container images are built from Dockerfiles located in:

  • run/: Dockerfile and index.html for Cloud Run demo
  • gke/: Dockerfile and index.html for GKE demo

Both demos use the same base image (nginx:alpine) serving a static Cymbal Superstore page.


Setting Up Artifact Registry (One-Time Setup)

Before deploying Cloud Run or GKE demos, create an Artifact Registry repository to store your Docker images.

Step 1: Set Environment Variables

export PROJECT_ID="your-service-project-id"
export REGION="us-central1"
export REPO_NAME="docker-repo"

Step 2: Enable Artifact Registry API

gcloud services enable artifactregistry.googleapis.com \
  --project=$PROJECT_ID

Step 3: Create Docker Repository

gcloud artifacts repositories create $REPO_NAME \
  --project=$PROJECT_ID \
  --repository-format=docker \
  --location=$REGION \
  --description="Docker repository for GDG ACE demo applications"

Step 4: Configure Docker Authentication

gcloud auth configure-docker ${REGION}-docker.pkg.dev

Step 5: Verify Repository Creation

gcloud artifacts repositories list \
  --project=$PROJECT_ID \
  --location=$REGION

Your Artifact Registry is now ready to receive Docker images. The image path format will be:

us-central1-docker.pkg.dev/your-service-project-id/docker-repo/IMAGE_NAME:TAG

Shared VPC Architecture

All demos in this repository assume the following network topology:

┌─────────────────────────────────────────────────────────────┐
│                     Host Project                             │
│  ┌───────────────────────────────────────────────────────┐  │
│  │              Shared VPC Network                        │  │
│  │                                                         │  │
│  │  ┌──────────────────────────────────────────────────┐ │  │
│  │  │ Subnet: shared-vpc-subnet-vms                     │ │  │
│  │  │ Purpose: PRIVATE                                  │ │  │
│  │  │ IP Range: 10.0.0.0/20                             │ │  │
│  │  │ Use: GCE instances, GKE nodes                     │ │  │
│  │  └──────────────────────────────────────────────────┘ │  │
│  │                                                         │  │
│  │  ┌──────────────────────────────────────────────────┐ │  │
│  │  │ Subnet: shared-vpc-subnet-proxy                   │ │  │
│  │  │ Purpose: REGIONAL_MANAGED_PROXY                   │ │  │
│  │  │ IP Range: 10.1.0.0/26                             │ │  │
│  │  │ Use: Internal/Regional LB proxy instances         │ │  │
│  │  └──────────────────────────────────────────────────┘ │  │
│  │                                                         │  │
│  │  Firewall Rules (network tag-based):                  │  │
│  │   • iap-ssh: Allow IAP SSH (35.235.240.0/20)          │  │
│  │   • allow-health-checks: GCP health check IPs         │  │
│  │   • allow-proxy: Regional Managed Proxy subnet        │  │
│  │   • webserver: Allow HTTP traffic (0.0.0.0/0:80)      │  │
│  └───────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘
                         ▲
                         │ Attached to Shared VPC
                         │
┌────────────────────────┴────────────────────────────────────┐
│                  Service Project                             │
│  • Compute Engine instances                                  │
│  • GKE clusters                                              │
│  • Cloud Run services (VPC Connector for private access)     │
│  • Cloud Functions                                           │
└──────────────────────────────────────────────────────────────┘

Setting Up Shared VPC (One-Time Setup)

If you don't already have a Shared VPC configured, follow these commands to create the network infrastructure.

Step 1: Set Environment Variables

export HOST_PROJECT_ID="your-host-project-id"
export SERVICE_PROJECT_ID="your-service-project-id"
export REGION="us-central1"
export VPC_NAME="shared-vpc"
export SUBNET_VMS="shared-vpc-subnet-vms"
export SUBNET_PROXY="shared-vpc-subnet-proxy"

Step 2: Enable Shared VPC on Host Project

# Enable Shared VPC on the host project
gcloud compute shared-vpc enable $HOST_PROJECT_ID

# Attach service project to the Shared VPC
gcloud compute shared-vpc associated-projects add $SERVICE_PROJECT_ID \
  --host-project=$HOST_PROJECT_ID

Step 3: Create VPC Network

# Create custom VPC network
gcloud compute networks create $VPC_NAME \
  --project=$HOST_PROJECT_ID \
  --subnet-mode=custom \
  --bgp-routing-mode=regional

Step 4: Create Subnets

# Subnet for VM instances and GKE nodes
gcloud compute networks subnets create $SUBNET_VMS \
  --project=$HOST_PROJECT_ID \
  --network=$VPC_NAME \
  --region=$REGION \
  --range=10.0.0.0/20 \
  --enable-private-ip-google-access \
  --enable-flow-logs

# Proxy-only subnet for Regional Managed Proxy (required for Internal/Regional LB)
gcloud compute networks subnets create $SUBNET_PROXY \
  --project=$HOST_PROJECT_ID \
  --network=$VPC_NAME \
  --region=$REGION \
  --range=10.1.0.0/26 \
  --purpose=REGIONAL_MANAGED_PROXY \
  --role=ACTIVE

Step 5: Create Firewall Rules

# Allow IAP SSH access (tag: iap-ssh)
gcloud compute firewall-rules create allow-iap-ssh \
  --project=$HOST_PROJECT_ID \
  --network=$VPC_NAME \
  --action=allow \
  --rules=tcp:22 \
  --source-ranges=35.235.240.0/20 \
  --target-tags=iap-ssh \
  --description="Allow SSH via Identity-Aware Proxy"

# Allow health checks from GCP probe IPs (tag: allow-health-checks)
gcloud compute firewall-rules create allow-health-checks \
  --project=$HOST_PROJECT_ID \
  --network=$VPC_NAME \
  --action=allow \
  --rules=tcp \
  --source-ranges=35.191.0.0/16,130.211.0.0/22 \
  --target-tags=allow-health-checks \
  --description="Allow health checks from GCP load balancer probe IPs"

# Allow traffic from Regional Managed Proxy subnet
gcloud compute firewall-rules create allow-proxy-subnet \
  --project=$HOST_PROJECT_ID \
  --network=$VPC_NAME \
  --action=allow \
  --rules=tcp:80,tcp:443,tcp:8080 \
  --source-ranges=10.1.0.0/26 \
  --description="Allow traffic from Regional Managed Proxy subnet"

# Allow HTTP traffic from internet (tag: webserver)
gcloud compute firewall-rules create allow-http \
  --project=$HOST_PROJECT_ID \
  --network=$VPC_NAME \
  --action=allow \
  --rules=tcp:80 \
  --source-ranges=0.0.0.0/0 \
  --target-tags=webserver \
  --description="Allow HTTP traffic from internet"

Step 6: Grant IAM Permissions

Service project users need permissions to use subnets from the host project:

# Grant Network User role on the VPC subnets
gcloud projects add-iam-policy-binding $HOST_PROJECT_ID \
  --member="serviceAccount:service-${SERVICE_PROJECT_NUMBER}@compute-system.iam.gserviceaccount.com" \
  --role="roles/compute.networkUser"

# For GKE clusters, also grant to the GKE service account
gcloud projects add-iam-policy-binding $HOST_PROJECT_ID \
  --member="serviceAccount:service-${SERVICE_PROJECT_NUMBER}@container-engine-robot.iam.gserviceaccount.com" \
  --role="roles/compute.networkUser"

Note: Replace ${SERVICE_PROJECT_NUMBER} with your actual service project number. Find it with:

gcloud projects describe $SERVICE_PROJECT_ID --format="value(projectNumber)"

Getting Started with Demos

Each demo directory contains:

  • terraform.tfvars.example: Configuration template with placeholder values
  • README.md: Detailed deployment instructions
  • main.tf: Terraform provider and resource definitions
  • variables.tf: Input variable declarations
  • outputs.tf: Output values after deployment

Important: For Cloud Run (run/) and GKE (gke/, manifests/) demos, create an Artifact Registry repository first (see Setting Up Artifact Registry).

General Deployment Steps

  1. Navigate to the demo directory:

    cd function/  # or gce/, gke/, run/
  2. Copy the example configuration and edit it with your values:

    cp terraform.tfvars.example terraform.tfvars
    # Edit terraform.tfvars and fill in your project IDs and network details
    vim terraform.tfvars
  3. Initialize and apply Terraform:

    terraform init
    terraform plan
    terraform apply
  4. Clean up resources:

    terraform destroy

Load Testing with k6

Several demos include load test scripts (load_test.js) to simulate traffic and demonstrate autoscaling behavior. These scripts are written for k6, a modern open-source load testing tool.

Install k6

macOS (Homebrew):

brew install k6

Linux (Debian/Ubuntu):

sudo gpg --no-default-keyring \
  --keyring /usr/share/keyrings/k6-archive-keyring.gpg \
  --keyserver hkp://keyserver.ubuntu.com:80 \
  --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69

echo "deb [signed-by=/usr/share/keyrings/k6-archive-keyring.gpg] https://dl.k6.io/deb stable main" \
  | sudo tee /etc/apt/sources.list.d/k6.list

sudo apt-get update && sudo apt-get install k6

Docker (any platform):

docker run --rm -i grafana/k6 run - < load_test.js

Windows (Chocolatey):

choco install k6

Running Load Tests

Each demo with a load_test.js file includes instructions to:

  1. Update the target URL/IP in the script (replace placeholders with actual values from Terraform output)
  2. Run the test:
    k6 run load_test.js

k6 will display real-time metrics:

  • VUs (Virtual Users): Simulated concurrent users
  • http_reqs: Total HTTP requests sent
  • http_req_duration: Response time metrics (p95, p99)
  • http_req_failed: Failed request rate

Demos with Load Tests

Demo Script Purpose
gce/ load_test.js Test MIG autoscaling based on CPU utilization
run/ load_test.js Demonstrate Cloud Run scale-to-zero and rapid scale-up
manifests/ load_test.js Test GKE HPA (Horizontal Pod Autoscaler)

Tip: Start with low traffic and gradually increase load to observe autoscaling behavior in the GCP Console.


Security Notes

  • ⚠️ Never commit terraform.tfvars with real project IDs to public repositories
  • ⚠️ Use .gitignore to exclude sensitive files (.tfstate, .tfvars, credentials)
  • ✅ Use service accounts with least-privilege permissions
  • ✅ Enable VPC Flow Logs and Cloud Audit Logs for compliance

Additional Resources


License

This repository is provided as-is for educational purposes.

About

GDG - ACE Demo

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors