Add error telemetry by williammartin · Pull Request #13251 · cli/cli

williammartin · 2026-04-21T12:40:23Z

Motivation

Today, when a command fails we have no visibility into what kinds of errors users are hitting. This makes it hard to prioritize reliability work or catch regressions early.

What this does

Records error information alongside the existing command_invocation telemetry event so we can understand failure patterns at scale. Each error event includes a classification (outcome) and the chain of error types (errTypes) - no user data, just type names.

To support this cleanly, the PR also:

Centralizes telemetry recording in ghcmd.Main() instead of wiring it per-command through Cobra, reducing boilerplate and making it harder to accidentally skip.
Introduces internal/gherrs to give exit codes and silencing behavior a proper home, replacing scattered sentinel checks across the codebase.
Simplifies pkg/cmdutil/telemetry.go down to just disable/check helpers now that recording lives in one place.

Validating the data

The new telemetry dimensions are covered by end-to-end tests that capture real Main() output, so we will catch regressions if the shape of the data changes.

Copilot

Pull request overview

This PR centralizes command-invocation telemetry recording in ghcmd.Main() and augments it with error-related dimensions, while introducing a dedicated internal error package to standardize exit codes and “silenced” behavior.

Changes:

Move command_invocation telemetry recording from per-command Cobra instrumentation into internal/ghcmd.Main() (including flag collection and error dimensions).
Add internal/gherrs to represent exit-coded/silenced errors and map existing error cases onto these types for consistent telemetry + exit status behavior.
Replace cmdutil telemetry recording helpers with telemetry-disable helpers (including recursive disabling) and update tests accordingly.

Show a summary per file

File	Description
pkg/cmdutil/telemetry.go	Removes command telemetry instrumentation; adds recursive disable + exported disabled check.
pkg/cmdutil/telemetry_test.go	Updates tests to cover disable/disabled-check behavior instead of recording.
pkg/cmd/root/root.go	Removes per-subcommand telemetry wiring from root command construction.
pkg/cmd/auth/auth.go	Switches to recursive telemetry disabling for the auth command subtree.
internal/gherrs/errors.go	Introduces standardized error types/interfaces for exit codes + silencing.
internal/ghcmd/cmd.go	Records command telemetry in a deferred block and adds error-to-dimensions logic.
internal/ghcmd/cmd_test.go	Adds tests for error-type extraction, error dimensions, and end-to-end telemetry capture.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 7/7 changed files
Comments generated: 3

Copilot

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 7/7 changed files
Comments generated: 3

Copilot

Copilot's findings

Files reviewed: 7/7 changed files
Comments generated: 4

- Preserve original error in DNS GeneralError wrapper for telemetry - Rename depth-bound test to match its actual behavior - Add e2e test for flags dimension coverage - Add e2e test for telemetry-disabled commands Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…te SSO - Pass original error to newErrDims so errTypes reflects the real error chain instead of replacement sentinels like gherrs.SilentError - Rewrite grabAllUnwrappableNestedErrorTypes to use BFS, handling errors.Join (Unwrap() []error) in addition to single-wrapped errors - Gate SSO URL and scopes suggestion behind errors.As(err, &httpErr) to prevent misleading messages on non-HTTP errors Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot's findings

Comments suppressed due to low confidence (4)

internal/gherrs/errors.go:15

This newly exported interface needs a godoc comment to satisfy the repository guideline for exported functions, types, and constants (AGENTS.md:135).

type ExitCoder interface {
	error
	ExitCode() ExitCode
}

internal/gherrs/errors.go:20

This newly exported interface needs a godoc comment to satisfy the repository guideline for exported functions, types, and constants (AGENTS.md:135).

type Silenced interface {
	error
	silent()
}

internal/gherrs/errors.go:80

This newly exported error type needs a godoc comment to satisfy the repository guideline for exported functions, types, and constants (AGENTS.md:135).

type ExtensionExecError struct {
	Code int
}

internal/gherrs/errors.go:95

This newly exported error type needs a godoc comment to satisfy the repository guideline for exported functions, types, and constants (AGENTS.md:135).

type GeneralError struct {
	WrappedErr error
	Message    string
}

Files reviewed: 7/7 changed files
Comments generated: 6

+		telemetryService.Record(ghtelemetry.Event{
+			Type:       "command_invocation",
+			Dimensions: dimensions,
+		})


 }

-func isTelemetryDisabled(cmd *cobra.Command) bool {
+func IsTelemetryDisabled(cmd *cobra.Command) bool {


@@ -0,0 +1,114 @@
+package gherrs
+
+type ExitCode int


Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

williammartin · 2026-05-13T18:19:05Z

@babakks I've requested your review on this, but I haven't made it ready for review. That is because honestly, this got a bit out of hand and maybe it just needs a different approach to get the main juice (errors in telemetry). So please give it some eyes and let me know whether you think it's worth pursuing this any further.

- Replace reflect with fmt.Sprintf for error type names - Filter noise stdlib types from error telemetry - Explicitly type sentinel error vars as error - Move IsExtensionCommand to root package - Remove alias telemetry (ExpandedCommandPath) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

williammartin force-pushed the wm/record-error-telemetry branch 2 times, most recently from 812ff32 to 3dff863 Compare April 21, 2026 14:50

williammartin requested a review from Copilot April 21, 2026 14:50

Copilot started reviewing on behalf of williammartin April 21, 2026 14:51 View session

Copilot AI reviewed Apr 21, 2026

View reviewed changes

Comment thread internal/ghcmd/cmd.go

Comment thread internal/ghcmd/cmd.go Outdated

Comment thread internal/gherrs/errors.go

williammartin force-pushed the wm/record-error-telemetry branch 2 times, most recently from 9623666 to 12e60b4 Compare April 21, 2026 15:00

williammartin requested a review from Copilot April 21, 2026 15:14

Copilot started reviewing on behalf of williammartin April 21, 2026 15:15 View session

Copilot AI reviewed Apr 21, 2026

View reviewed changes

Comment thread internal/ghcmd/cmd.go

Comment thread internal/ghcmd/cmd_test.go

Comment thread internal/ghcmd/cmd_test.go Outdated

williammartin force-pushed the wm/record-error-telemetry branch 2 times, most recently from 32538c9 to 6eeb828 Compare May 13, 2026 15:39

Add error telemetry

e608b06

williammartin force-pushed the wm/record-error-telemetry branch from 6eeb828 to e608b06 Compare May 13, 2026 15:42

williammartin requested a review from Copilot May 13, 2026 15:56

Copilot started reviewing on behalf of williammartin May 13, 2026 15:56 View session

Copilot AI reviewed May 13, 2026

View reviewed changes

Comment thread internal/ghcmd/cmd.go

Comment thread internal/ghcmd/cmd_test.go Outdated

Comment thread internal/ghcmd/cmd.go

Comment thread internal/ghcmd/cmd.go

williammartin and others added 2 commits May 13, 2026 18:13

williammartin requested a review from Copilot May 13, 2026 17:38

Copilot started reviewing on behalf of williammartin May 13, 2026 17:39 View session

Copilot AI reviewed May 13, 2026

View reviewed changes

Address PR review feedback

a10ddc3

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

williammartin requested review from babakks May 13, 2026 18:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add error telemetry#13251

Add error telemetry#13251
williammartin wants to merge 5 commits into
trunkfrom
wm/record-error-telemetry

williammartin commented Apr 21, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

williammartin commented May 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

williammartin commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

What this does

Validating the data

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Uh oh!

williammartin commented May 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

williammartin commented Apr 21, 2026 •

edited

Loading