feat(integrations): add Google Cloud SQL to @deepnote/database-integrations#403
Conversation
…ations Registers 'cloud-sql' as a new DatabaseIntegrationType. Adds: - metadata schema (service_account: string) - config union member in databaseIntegrationConfigSchema - secret-field-paths entry (service_account is a secret) - env-vars case returning null (Cloud SQL uses the cloud-sql-python-connector, not a plain SQLAlchemy URL) - CLI prompts for add/edit commands - metadata schema unit tests Preceded by vscode-deepnote#406 which includes a local shim for E2E verification; that shim can be removed once this package publishes a new version. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- env-vars: assert cloud-sql generates no SQLAlchemy var and no errors - CLI add-integration: cloud-sql happy path (name + service account) - CLI edit-integration: cloud-sql keep-defaults and update flows Covers the new lines flagged by codecov/patch on #398. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
📝 WalkthroughWalkthroughThis PR adds cloud-sql as a new database integration type. Changes span the type system (registering cloud-sql as a valid SQL integration), schema validation (requiring service_account metadata), environment variable handling (returning null for SQLAlchemy since Cloud SQL uses a Python connector), CLI prompts (collecting service account JSON interactively), and command wiring (integrating cloud-sql into add and edit flows). Comprehensive tests verify schema validation, end-to-end creation, and editing workflows. Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #403 +/- ##
==========================================
+ Coverage 83.93% 83.95% +0.02%
==========================================
Files 145 146 +1
Lines 8018 8029 +11
Branches 2165 2230 +65
==========================================
+ Hits 6730 6741 +11
Misses 1287 1287
Partials 1 1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
What / Why
Adds
cloud-sqlas a supportedDatabaseIntegrationTypein the@deepnote/database-integrationspackage.This is the upstream schema extraction for the Cloud SQL integration —
vscode-deepnote#406added a local shim with the comment "Pending addition to @deepnote/database-integrations; remove once the package includes 'cloud-sql'." This PR fulfils that.Changes
database-integration-types.ts—'cloud-sql'added tosqlIntegrationTypesdatabase-integration-metadata-schemas.ts—cloudSqlMetadataSchema(service_account: z.string(), consistent with Spanner/BigQuery convention)database-integration-config.ts— config union member addedsecret-field-paths.ts—'cloud-sql': ['service_account'](service account JSON is a secret)database-integration-env-vars.ts—case 'cloud-sql': return null(Cloud SQL connects via the cloud-sql-python-connector, not a plain SQLAlchemy URL — satisfies the exhaustiveness check)integrations-prompts/cloud-sql.ts— new CLI prompt (singleservice_accountfield, modelled on Spanner)add-integration.ts+edit-integration.ts— import + switch case wired indatabase-integration-metadata-schemas.test.ts— 2 new test casesWhy
cloud-sqlproduces no SQLAlchemy inputHow integrations normally reach the runtime. For a SQL integration,
getEnvVarForSqlCells(database-integration-env-vars.ts) callsgetSqlAlchemyInputand, only if it returns a non-null{ url, params, param_style }, emits an env var. The deepnote-toolkit runtime reads that env var and feeds the triple straight into SQLAlchemy:execute_sql→_query_data_source→create_engine(url, **params, pool_pre_ping=True)(deepnote_toolkit/sql/sql_execution.py:506). Sourlhas to be a DSN thatcreate_enginecan dial directly, e.g.postgresql://user:pass@host:port/db.Cloud SQL has no such DSN. A Cloud SQL instance isn't addressable by a static host:port URL — you connect through the Cloud SQL Python Connector (
from google.cloud.sql.connector import Connector), which resolves the instance (project:region:instance), performs the IAM/TLS handshake using the service account, and hands SQLAlchemy acreatorcallable instead of a URL (seedocs/google-cloud-sql.md). There is nourl+paramspair that captures that flow, so there is nothing valid forgetSqlAlchemyInputto emit — hencereturn null.Why
nullspecifically (not a placeholder URL). ReturningnullmakesgetEnvVarForSqlCellsskip the env var entirely (the new test assertsgetSqlAlchemyInputVar(envVars, 'my-cloud-sql')isundefined). Emitting a half-formed URL would instead make the toolkit callcreate_engineon a DSN it can't dial and fail at query time.nullcleanly means "no platform-injected engine; the connection is established in-notebook via the connector," which matches the documented Cloud SQL workflow today.The toolkit gates this too. The toolkit bundles
sqlalchemy-spannerandsqlalchemy-bigquery(pyproject.toml), so those can build engines from a Google-specific URL — Spanner emitsspanner+spanner:///projects/<id>/instances/<instance>/databases/<db>, BigQuery emitsbigquery://withcredentials_infoinparams. The toolkit does not shipcloud-sql-python-connectorand has nogoogle.cloud.sqlimport, so even a correct URL would have no driver to consume it. The closest existing precedent in this same file is BigQuery federated auth, which alreadyreturn nulls when the platform can't construct a URL (database-integration-env-vars.ts:611, plus the federated-auth guard at:140).cloud-sqlfollows that shape; a full connector-backed implementation (toolkit + env-var) is a follow-up.How to review
Start with
database-integration-types.ts(one-liner), thendatabase-integration-metadata-schemas.ts, thensecret-field-paths.ts, thendatabase-integration-env-vars.ts. The CLI files (add-integration.ts,edit-integration.ts,cloud-sql.ts) follow the exact same pattern as Spanner.Testing
tsc --noEmitacross all packages)@deepnote/database-integrations; all pass in@deepnote/cliwhen run in isolationpromptForFieldsCloudSql): not manually run, but covered by typecheck — follows the Spanner pattern exactlyadd-integrationtests are intermittently flaky under parallel local execution (unrelated to this diff); CI is unaffectedRisks
return nullin env-vars is intentional — Cloud SQL connects via the Cloud SQL Python Connector rather than a SQLAlchemy URL, mirroring how BigQuery federated auth already returnsnull(see Whycloud-sqlproduces no SQLAlchemy input above). A full connector-backed implementation (toolkit + env-var) is a follow-up.Next step
Once this merges and
@deepnote/database-integrationspublishes a new version,vscode-deepnote#406can be updated to remove the local shim and consume the package directly.Summary by CodeRabbit