Skip to content

feat(cli): stream block outputs in real-time during deepnote run#304

Open
jamesbhobbs wants to merge 17 commits into
mainfrom
feat/cli-streaming-output
Open

feat(cli): stream block outputs in real-time during deepnote run#304
jamesbhobbs wants to merge 17 commits into
mainfrom
feat/cli-streaming-output

Conversation

@jamesbhobbs

@jamesbhobbs jamesbhobbs commented Feb 19, 2026

Copy link
Copy Markdown
Contributor

You can test streaming using this project: cli_streaming_output.deepnote

metadata:
  createdAt: '2026-04-03T15:55:00.000Z'
  modifiedAt: '2026-04-03T15:55:05.150Z'
project:
  id: cli-streaming-output-example
  name: CLI streaming output example
  notebooks:
    - blocks:
        - blockGroup: streaming-intro-group
          content: |-
            # CLI Streaming Output Example

            Run the code block below with `deepnote run examples/7_cli_streaming_output.deepnote --block streaming-code-block` to verify that stdout is rendered as each update arrives.

            You should see one line per second for five seconds, followed by a completion message.
          id: streaming-intro-block
          metadata: {}
          sortingKey: a0
          type: markdown
          contentHash: sha256:3a8996d39f741cce9803d293d45789eccaacdf5399e3f299e41bfbec9ed68838
        - blockGroup: streaming-code-group
          content: |-
            import time

            for step in range(5):
                print(f"Streaming step {step + 1}/5", flush=True)
                time.sleep(1)

            print("Done streaming.", flush=True)
          id: streaming-code-block
          metadata: {}
          sortingKey: a1
          type: code
          executionCount: 1
          outputs:
            - name: stdout
              output_type: stream
              text: |
                Streaming step 1/5
                Streaming step 2/5
                Streaming step 3/5
                Streaming step 4/5
                Streaming step 5/5
                Done streaming.
          contentHash: sha256:052585c3cdddd024b89a10bac4fada91a1485dea32de8a042c561a4425352009
          executionStartedAt: '2026-04-03T15:55:00.010Z'
          executionFinishedAt: '2026-04-03T15:55:05.020Z'
      executionMode: block
      id: cli-streaming-output-notebook
      isModule: false
      name: 7. CLI streaming output
  settings: {}
version: 1.0.0
execution:
  finishedAt: '2026-04-03T15:55:05.150Z'
  startedAt: '2026-04-03T15:55:00.000Z'
  summary:
    blocksExecuted: 1
    blocksFailed: 0
    blocksSucceeded: 1
    totalDurationMs: 5150
  triggeredBy: user

Summary

  • Wire the existing onOutput callback from the execution engine to render block outputs immediately as they arrive from the Jupyter kernel, instead of batching them after block completion
  • For long-running blocks (e.g., training loops with print() updates), users now see output in real-time instead of waiting until the block finishes
  • Machine output modes (-o json, -o toon) are unchanged — they still collect outputs via result.outputs in onBlockDone

Test plan

  • All 1918 existing tests pass
  • TypeScript typecheck passes
  • Biome lint passes
  • -o json output includes outputs in blocks array (tested: does not render output via onOutput in machine output mode)
  • Blocks with no output show [1/3] label ✓ (50ms) on one line (tested: keeps status on same line as label for blocks with no output)
  • Manual test: run a notebook with a long-running block that prints incrementally and verify output streams in real-time

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes

    • Improved interactive CLI streaming: adds per-block newline handling, ensures completion markers align, avoids duplicating rendered outputs when blocks stream, and conditionally omits extra block output in interactive mode. Machine/JSON mode unchanged—streamed data stays in payloads and is not printed.
  • Tests

    • New tests for streaming behavior, output ordering, newline/spacing rules, inline success marks, and JSON vs interactive modes.

jamesbhobbs and others added 2 commits February 19, 2026 16:54
…ution

CLI --input values were being ignored because input blocks unconditionally
assigned from their saved metadata during execution, overwriting the
CLI-provided values. This patches the in-memory DeepnoteFile to inject
CLI input values into input block metadata before execution begins.

Also adds a test documenting that empty projects return exit code 0.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously, `deepnote run` waited until each block finished executing
before rendering all outputs at once. For long-running blocks (e.g.,
training loops with print() updates), users saw nothing until completion.

Wire the existing `onOutput` callback from the execution engine to
render outputs immediately as they arrive from the Jupyter kernel.
Machine output modes (-o json, -o toon) are unchanged.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@codecov

codecov Bot commented Feb 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.95%. Comparing base (99881fe) to head (91b2793).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #304      +/-   ##
==========================================
+ Coverage   83.93%   83.95%   +0.02%     
==========================================
  Files         145      145              
  Lines        8018     8028      +10     
  Branches     2227     2230       +3     
==========================================
+ Hits         6730     6740      +10     
  Misses       1287     1287              
  Partials        1        1              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jamesbhobbs jamesbhobbs changed the base branch from main to fix/cli-input-overrides February 19, 2026 17:27
Base automatically changed from fix/cli-input-overrides to main February 20, 2026 08:27
@coderabbitai

coderabbitai Bot commented Feb 20, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

Adds test-only harnesses to simulate streamed block outputs and a streaming-focused test suite. Runtime changes add per-block streamed-output tracking to RunExecutionState, an onOutput(blockId, blockOutput) callback that prints a single blank line before the first streamed output for that block in human mode, renders streamed outputs immediately, and records blocks that streamed. onBlockDone now detects whether a block streamed output and, if so, suppresses re-rendering of result.outputs and adjusts newline/completion-line placement; otherwise it renders result.outputs as before.

Sequence Diagram

sequenceDiagram
    participant CLI as CLI Run
    participant Engine as Engine
    participant Renderer as Output Renderer
    participant State as Execution State

    Engine->>CLI: onBlockStart(blockId)
    CLI->>State: record block start

    loop streamed outputs for block
        Engine->>CLI: onOutput(blockId, output)
        CLI->>State: is first output?
        alt first streamed output for block
            CLI->>Renderer: print blank line
            CLI->>State: mark block as streamed
        end
        CLI->>Renderer: render streamed output
    end

    Engine->>CLI: onBlockDone(blockId, result)
    CLI->>State: remove block from streamed set (hadStreamedOutput?)
    alt had streamed output
        CLI->>Renderer: print newline / finalize completion line (do not re-render result.outputs)
    else no streamed output
        CLI->>Renderer: render result.outputs (and trailing newline if present)
    end
    CLI->>Renderer: print completion indicator (✓ / failure)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Updates Docs ⚠️ Warning PR implements streaming output feature for deepnote run command but CLI documentation not updated. Update packages/cli/README.md run command section to document streaming behavior and machine output batching.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly describes the main change: streaming block outputs in real-time during the deepnote run command. It matches the PR's core objective of wiring the onOutput callback for immediate output rendering.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Review rate limit: 4/5 reviews remaining, refill in 12 minutes.

Comment @coderabbitai help to get the list of available commands and usage tips.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Feb 20, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.gitignore:
- Around line 1-34: The .gitignore contains duplicated patterns; remove the
redundant entries so each ignore pattern appears only once (e.g. duplicate lines
for .DS_Store, .env, .npmrc, .pnpm-store, .qlty/* entries, *.png, *.tsbuildinfo,
coverage, dist, examples/report.md, examples/snapshots/*_T*.snapshot.deepnote,
and node_modules) ensuring the file only lists unique patterns; update the
.gitignore by deleting the duplicate lines and keep one canonical occurrence of
each pattern.

In `@packages/cli/src/commands/run.test.ts`:
- Around line 568-595: The test "renders output immediately via onOutput in
non-machine mode" asserts only for a checkmark but not the streamed text; update
the test to assert that the streamed output "Hello World" was written to stdout
by checking the spy used for process.stdout.write (stdoutWriteSpy) rather than
consoleLogSpy, e.g., verify that stdoutWriteSpy was called with or contains
"Hello World" after invoking action(HELLO_WORLD_FILE, {}); locate the
mockRunProject stub and the test block around it and add the assertion
referencing stdoutWriteSpy and the action/HELLO_WORLD_FILE test flow.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: e921e22f-7913-46cf-8c89-02ee618c98cf

📥 Commits

Reviewing files that changed from the base of the PR and between 9949ada and 75d2636.

📒 Files selected for processing (3)
  • .gitignore
  • packages/cli/src/commands/run.test.ts
  • packages/cli/src/commands/run.ts

Comment thread .gitignore Outdated
Comment thread packages/cli/src/commands/run.test.ts
@dinohamzic

Copy link
Copy Markdown
Contributor

@coderabbitai what's pending? can you approve?

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/cli/src/commands/run.ts (1)

1040-1094: ⚠️ Potential issue | 🟡 Minor

Track line termination, not just “had streamed output”.

The main print() path emits stream.text with a trailing newline. Here, once hadStreamedOutput is true, prefix always adds another \n, so normal line-oriented stdout ends up with an extra blank line before ✓/✗. This needs per-block “last chunk ended with newline” state instead of a boolean that only says something streamed.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/commands/run.ts` around lines 1040 - 1094, The boolean
blocksWithStreamedOutput is too coarse — change streaming state to track whether
the last streamed chunk for a block ended with a newline and use that in
onBlockDone to avoid inserting an extra blank line; when onOutput(streaming)
runs for a blockId set both state.blocksWithStreamedOutput.add(blockId) and
state.blocksLastChunkEndedWithNewline.set(blockId,
blockOutput.stream?.text?.endsWith('\n') ?? false) (use the same symbols
onOutput, blockOutput), and in onBlockDone compute hadStreamedOutput =
state.blocksWithStreamedOutput.delete(result.blockId) and lastEndedNewline =
state.blocksLastChunkEndedWithNewline.get(result.blockId) then remove the map
entry; change prefix to insert '\n' only when state.agentStreamed is true OR
(hadStreamedOutput is true and lastEndedNewline is false) so you don't add an
extra newline after output that already ended with '\n'.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/cli/src/commands/run.test.ts`:
- Around line 192-199: The mock implementation for mockRunProject calls
lifecycle callbacks fire-and-forget causing race conditions; change the
implementation in mockRunProject.mockImplementation to await the async callbacks
on options (await options?.onBlockStart?.(TEST_BLOCK, 0, 1), await
options?.onOutput?.(TEST_BLOCK.id, blockOutput) for each streamedOutputs
iteration, and await options?.onBlockDone?.(result)) so the mock resolves only
after all lifecycle handlers complete, ensuring tests wait for the same async
contract used in run.ts.
- Around line 609-653: The tests don't actually exercise the onOutput path
because they include the same chunk in result.outputs as well as streamedOutputs
and never assert that interactive rendering stayed silent in JSON mode; update
the two tests that call setupStreamingRun/action (the human-mode and
machine-mode cases) so that: for the human-mode test (the one calling
action(HELLO_WORLD_FILE, {})) provide the chunk only via streamedOutputs and
ensure result.outputs is empty so onOutput must do the rendering, then assert
stdoutWriteSpy (or getOutput(consoleLogSpy)) contains "Hello World" and the
success marker; for the machine-mode test (action(HELLO_WORLD_FILE, { output:
'json' })), keep the chunk in streamedOutputs but assert that no interactive
writes occurred (e.g., stdoutWriteSpy has no calls containing "Hello World") and
still parse getOutput(consoleLogSpy) to verify parsed.blocks[0].outputs contains
the chunk; change setupStreamingRun, test inputs, and assertions accordingly to
reference setupStreamingRun, action, HELLO_WORLD_FILE, stdoutWriteSpy,
consoleLogSpy, and getOutput.

---

Outside diff comments:
In `@packages/cli/src/commands/run.ts`:
- Around line 1040-1094: The boolean blocksWithStreamedOutput is too coarse —
change streaming state to track whether the last streamed chunk for a block
ended with a newline and use that in onBlockDone to avoid inserting an extra
blank line; when onOutput(streaming) runs for a blockId set both
state.blocksWithStreamedOutput.add(blockId) and
state.blocksLastChunkEndedWithNewline.set(blockId,
blockOutput.stream?.text?.endsWith('\n') ?? false) (use the same symbols
onOutput, blockOutput), and in onBlockDone compute hadStreamedOutput =
state.blocksWithStreamedOutput.delete(result.blockId) and lastEndedNewline =
state.blocksLastChunkEndedWithNewline.get(result.blockId) then remove the map
entry; change prefix to insert '\n' only when state.agentStreamed is true OR
(hadStreamedOutput is true and lastEndedNewline is false) so you don't add an
extra newline after output that already ended with '\n'.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 1119c44b-984a-442b-b48c-e609e8cd72bc

📥 Commits

Reviewing files that changed from the base of the PR and between 45da19e and 86ac7d5.

📒 Files selected for processing (2)
  • packages/cli/src/commands/run.test.ts
  • packages/cli/src/commands/run.ts

Comment thread packages/cli/src/commands/run.test.ts
Comment thread packages/cli/src/commands/run.test.ts
@coderabbitai

coderabbitai Bot commented Apr 3, 2026

Copy link
Copy Markdown
Contributor
✅ Actions performed

Comments resolved and changes approved.

@deepnote deepnote deleted a comment from coderabbitai Bot Apr 3, 2026
@deepnote deepnote deleted a comment from coderabbitai Bot Apr 3, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/cli/src/commands/run.test.ts`:
- Around line 658-673: The test currently leaves payloads in result.outputs
which allows non-stream fallback rendering to satisfy assertions; update the
tests (those invoking setupStreamingRun and any other cases around lines
referencing streamedOutputs/result) to remove result.outputs so the test
exercises stream-only paths — ensure setupStreamingRun calls supply
streamedOutputs but do not include outputs in the result object (and do the same
for the other case around lines 709–721) so only the streaming path (not
onBlockDone fallback rendering) can pass the assertions.
- Around line 595-755: Add unit tests in the existing "streaming output via
onOutput" suite to cover streamed stderr and special-character outputs: call
setupStreamingRun with streamedOutputs containing an output object with
output_type: 'stream', name: 'stderr' and text including unicode and newline
characters (e.g., "error ✓ 🚀\nsecond line"), then invoke
action(HELLO_WORLD_FILE, {}) and assert using stdoutWriteSpy (or a
stderrWriteSpy if present) and getLogCalls(consoleLogSpy) that stderr text is
handled distinctly from stdout (not rendered in machine JSON mode, included in
blocks outputs in JSON), that special characters and newlines are preserved in
rendered output, and that newline/blank-line sequencing behavior (the tests
using getLogCalls, serverReadyIdx, and checkIdx) still holds for stderr streams;
reuse existing helpers setupStreamingRun, action, getOutput, getLogCalls,
mockRunProject identifiers to locate where to add these cases.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9c0e47a0-ffbb-420b-84d7-bc2acdc6085e

📥 Commits

Reviewing files that changed from the base of the PR and between 86ac7d5 and 19f3367.

📒 Files selected for processing (1)
  • packages/cli/src/commands/run.test.ts

Comment thread packages/cli/src/commands/run.test.ts
Comment thread packages/cli/src/commands/run.test.ts
@dinohamzic dinohamzic marked this pull request as ready for review April 3, 2026 14:33
@dinohamzic dinohamzic requested a review from a team as a code owner April 3, 2026 14:33
@dinohamzic dinohamzic requested a review from tkislan April 3, 2026 14:33

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
packages/cli/src/commands/run.test.ts (1)

649-659: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Make these two assertions stream-only to prevent fallback false positives.

Line 655 and Line 703 keep payloads in result.outputs, so both tests can still pass if streamed-state tracking regresses and rendering falls back to onBlockDone.

Suggested fix
         setupStreamingRun({
           streamedOutputs: [...outputs],
           result: {
             blockId: TEST_BLOCK.id,
             blockType: TEST_BLOCK.type,
             success: true,
-            outputs: [...outputs],
+            outputs: [],
             executionCount: 1,
             durationMs: 50,
           },
         })
@@
         setupStreamingRun({
           streamedOutputs: [output],
           result: {
             blockId: TEST_BLOCK.id,
             blockType: TEST_BLOCK.type,
             success: true,
-            outputs: [output],
+            outputs: [],
             executionCount: 1,
             durationMs: 50,
           },
         })

Also applies to: 697-707

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/commands/run.test.ts` around lines 649 - 659, The tests
currently include payloads in both streamedOutputs and result.outputs, allowing
a fallback onBlockDone path to satisfy assertions; update the two failing tests
to be stream-only by removing payloads from the result.outputs in the
setupStreamingRun call (i.e., leave streamedOutputs populated but set
result.outputs to an empty array or undefined) and then assert only against
streamedOutputs/streaming state (not result.outputs). Target the
setupStreamingRun usage and the assertions that reference streamed outputs in
run.test.ts so the expectations rely solely on the streaming path rather than
onBlockDone fallback.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@packages/cli/src/commands/run.test.ts`:
- Around line 649-659: The tests currently include payloads in both
streamedOutputs and result.outputs, allowing a fallback onBlockDone path to
satisfy assertions; update the two failing tests to be stream-only by removing
payloads from the result.outputs in the setupStreamingRun call (i.e., leave
streamedOutputs populated but set result.outputs to an empty array or undefined)
and then assert only against streamedOutputs/streaming state (not
result.outputs). Target the setupStreamingRun usage and the assertions that
reference streamed outputs in run.test.ts so the expectations rely solely on the
streaming path rather than onBlockDone fallback.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 65a45c22-94a4-4084-844c-92a4ba30ff04

📥 Commits

Reviewing files that changed from the base of the PR and between 19f3367 and a44acf2.

📒 Files selected for processing (1)
  • packages/cli/src/commands/run.test.ts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants