Skip to content

feat(integrations): add Google Cloud SQL to @deepnote/database-integrations#403

Merged
tkislan merged 3 commits into
mainfrom
tk/cloud-sql-database-integrations
Jun 16, 2026
Merged

feat(integrations): add Google Cloud SQL to @deepnote/database-integrations#403
tkislan merged 3 commits into
mainfrom
tk/cloud-sql-database-integrations

Conversation

@tkislan

@tkislan tkislan commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

What / Why

Adds cloud-sql as a supported DatabaseIntegrationType in the @deepnote/database-integrations package.

This is the upstream schema extraction for the Cloud SQL integration — vscode-deepnote#406 added a local shim with the comment "Pending addition to @deepnote/database-integrations; remove once the package includes 'cloud-sql'." This PR fulfils that.

Changes

  • database-integration-types.ts'cloud-sql' added to sqlIntegrationTypes
  • database-integration-metadata-schemas.tscloudSqlMetadataSchema (service_account: z.string(), consistent with Spanner/BigQuery convention)
  • database-integration-config.ts — config union member added
  • secret-field-paths.ts'cloud-sql': ['service_account'] (service account JSON is a secret)
  • database-integration-env-vars.tscase 'cloud-sql': return null (Cloud SQL connects via the cloud-sql-python-connector, not a plain SQLAlchemy URL — satisfies the exhaustiveness check)
  • integrations-prompts/cloud-sql.ts — new CLI prompt (single service_account field, modelled on Spanner)
  • add-integration.ts + edit-integration.ts — import + switch case wired in
  • database-integration-metadata-schemas.test.ts — 2 new test cases

Why cloud-sql produces no SQLAlchemy input

How integrations normally reach the runtime. For a SQL integration, getEnvVarForSqlCells (database-integration-env-vars.ts) calls getSqlAlchemyInput and, only if it returns a non-null { url, params, param_style }, emits an env var. The deepnote-toolkit runtime reads that env var and feeds the triple straight into SQLAlchemy: execute_sql_query_data_sourcecreate_engine(url, **params, pool_pre_ping=True) (deepnote_toolkit/sql/sql_execution.py:506). So url has to be a DSN that create_engine can dial directly, e.g. postgresql://user:pass@host:port/db.

Cloud SQL has no such DSN. A Cloud SQL instance isn't addressable by a static host:port URL — you connect through the Cloud SQL Python Connector (from google.cloud.sql.connector import Connector), which resolves the instance (project:region:instance), performs the IAM/TLS handshake using the service account, and hands SQLAlchemy a creator callable instead of a URL (see docs/google-cloud-sql.md). There is no url + params pair that captures that flow, so there is nothing valid for getSqlAlchemyInput to emit — hence return null.

Why null specifically (not a placeholder URL). Returning null makes getEnvVarForSqlCells skip the env var entirely (the new test asserts getSqlAlchemyInputVar(envVars, 'my-cloud-sql') is undefined). Emitting a half-formed URL would instead make the toolkit call create_engine on a DSN it can't dial and fail at query time. null cleanly means "no platform-injected engine; the connection is established in-notebook via the connector," which matches the documented Cloud SQL workflow today.

The toolkit gates this too. The toolkit bundles sqlalchemy-spanner and sqlalchemy-bigquery (pyproject.toml), so those can build engines from a Google-specific URL — Spanner emits spanner+spanner:///projects/<id>/instances/<instance>/databases/<db>, BigQuery emits bigquery:// with credentials_info in params. The toolkit does not ship cloud-sql-python-connector and has no google.cloud.sql import, so even a correct URL would have no driver to consume it. The closest existing precedent in this same file is BigQuery federated auth, which already return nulls when the platform can't construct a URL (database-integration-env-vars.ts:611, plus the federated-auth guard at :140). cloud-sql follows that shape; a full connector-backed implementation (toolkit + env-var) is a follow-up.

How to review

Start with database-integration-types.ts (one-liner), then database-integration-metadata-schemas.ts, then secret-field-paths.ts, then database-integration-env-vars.ts. The CLI files (add-integration.ts, edit-integration.ts, cloud-sql.ts) follow the exact same pattern as Spanner.

Testing

  • Typecheck: clean (tsc --noEmit across all packages)
  • Unit tests: 227/227 pass in @deepnote/database-integrations; all pass in @deepnote/cli when run in isolation
  • Build: succeeds (ESM + CJS + types)
  • CLI prompt (promptForFieldsCloudSql): not manually run, but covered by typecheck — follows the Spanner pattern exactly
  • Note: some pre-existing add-integration tests are intermittently flaky under parallel local execution (unrelated to this diff); CI is unaffected

Risks

  • Safe: purely additive, no existing behaviour changed
  • The return null in env-vars is intentional — Cloud SQL connects via the Cloud SQL Python Connector rather than a SQLAlchemy URL, mirroring how BigQuery federated auth already returns null (see Why cloud-sql produces no SQLAlchemy input above). A full connector-backed implementation (toolkit + env-var) is a follow-up.

Next step

Once this merges and @deepnote/database-integrations publishes a new version, vscode-deepnote#406 can be updated to remove the local shim and consume the package directly.

Summary by CodeRabbit

  • New Features
    • Added Cloud SQL database integration support, allowing users to create and manage Cloud SQL connections through the CLI with service account-based authentication.
    • CLI now supports adding and editing Cloud SQL integrations with streamlined configuration workflows for service account credentials.
    • Added comprehensive validation and environment variable handling for Cloud SQL integration configurations.

Marko Paleka and others added 3 commits June 11, 2026 16:22
…ations

Registers 'cloud-sql' as a new DatabaseIntegrationType. Adds:
- metadata schema (service_account: string)
- config union member in databaseIntegrationConfigSchema
- secret-field-paths entry (service_account is a secret)
- env-vars case returning null (Cloud SQL uses the cloud-sql-python-connector, not a plain SQLAlchemy URL)
- CLI prompts for add/edit commands
- metadata schema unit tests

Preceded by vscode-deepnote#406 which includes a local shim for E2E verification;
that shim can be removed once this package publishes a new version.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- env-vars: assert cloud-sql generates no SQLAlchemy var and no errors
- CLI add-integration: cloud-sql happy path (name + service account)
- CLI edit-integration: cloud-sql keep-defaults and update flows

Covers the new lines flagged by codecov/patch on #398.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR adds cloud-sql as a new database integration type. Changes span the type system (registering cloud-sql as a valid SQL integration), schema validation (requiring service_account metadata), environment variable handling (returning null for SQLAlchemy since Cloud SQL uses a Python connector), CLI prompts (collecting service account JSON interactively), and command wiring (integrating cloud-sql into add and edit flows). Comprehensive tests verify schema validation, end-to-end creation, and editing workflows.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Updates Docs ⚠️ Warning PR adds cloud-sql integration to @deepnote/database-integrations but modifies zero documentation files. docs/google-cloud-sql.md exists but was not updated. Update docs/google-cloud-sql.md or other relevant documentation to reflect the new cloud-sql integration type addition to the package.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: adding Google Cloud SQL support to the database-integrations package.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.95%. Comparing base (d921be7) to head (46c74d2).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #403      +/-   ##
==========================================
+ Coverage   83.93%   83.95%   +0.02%     
==========================================
  Files         145      146       +1     
  Lines        8018     8029      +11     
  Branches     2165     2230      +65     
==========================================
+ Hits         6730     6741      +11     
  Misses       1287     1287              
  Partials        1        1              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@tkislan tkislan marked this pull request as ready for review June 11, 2026 16:45
@tkislan tkislan requested a review from a team as a code owner June 11, 2026 16:45
@tkislan tkislan merged commit 119b50a into main Jun 16, 2026
21 checks passed
@tkislan tkislan deleted the tk/cloud-sql-database-integrations branch June 16, 2026 07:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants