feat(cli): stream block outputs in real-time during deepnote run#304
feat(cli): stream block outputs in real-time during deepnote run#304jamesbhobbs wants to merge 17 commits into
deepnote run#304Conversation
…ution CLI --input values were being ignored because input blocks unconditionally assigned from their saved metadata during execution, overwriting the CLI-provided values. This patches the in-memory DeepnoteFile to inject CLI input values into input block metadata before execution begins. Also adds a test documenting that empty projects return exit code 0. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously, `deepnote run` waited until each block finished executing before rendering all outputs at once. For long-running blocks (e.g., training loops with print() updates), users saw nothing until completion. Wire the existing `onOutput` callback from the execution engine to render outputs immediately as they arrive from the Jupyter kernel. Machine output modes (-o json, -o toon) are unchanged. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #304 +/- ##
==========================================
+ Coverage 83.93% 83.95% +0.02%
==========================================
Files 145 145
Lines 8018 8028 +10
Branches 2227 2230 +3
==========================================
+ Hits 6730 6740 +10
Misses 1287 1287
Partials 1 1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
📝 WalkthroughWalkthroughAdds test-only harnesses to simulate streamed block outputs and a streaming-focused test suite. Runtime changes add per-block streamed-output tracking to RunExecutionState, an onOutput(blockId, blockOutput) callback that prints a single blank line before the first streamed output for that block in human mode, renders streamed outputs immediately, and records blocks that streamed. onBlockDone now detects whether a block streamed output and, if so, suppresses re-rendering of result.outputs and adjusts newline/completion-line placement; otherwise it renders result.outputs as before. Sequence DiagramsequenceDiagram
participant CLI as CLI Run
participant Engine as Engine
participant Renderer as Output Renderer
participant State as Execution State
Engine->>CLI: onBlockStart(blockId)
CLI->>State: record block start
loop streamed outputs for block
Engine->>CLI: onOutput(blockId, output)
CLI->>State: is first output?
alt first streamed output for block
CLI->>Renderer: print blank line
CLI->>State: mark block as streamed
end
CLI->>Renderer: render streamed output
end
Engine->>CLI: onBlockDone(blockId, result)
CLI->>State: remove block from streamed set (hadStreamedOutput?)
alt had streamed output
CLI->>Renderer: print newline / finalize completion line (do not re-render result.outputs)
else no streamed output
CLI->>Renderer: render result.outputs (and trailing newline if present)
end
CLI->>Renderer: print completion indicator (✓ / failure)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Review rate limit: 4/5 reviews remaining, refill in 12 minutes. Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.gitignore:
- Around line 1-34: The .gitignore contains duplicated patterns; remove the
redundant entries so each ignore pattern appears only once (e.g. duplicate lines
for .DS_Store, .env, .npmrc, .pnpm-store, .qlty/* entries, *.png, *.tsbuildinfo,
coverage, dist, examples/report.md, examples/snapshots/*_T*.snapshot.deepnote,
and node_modules) ensuring the file only lists unique patterns; update the
.gitignore by deleting the duplicate lines and keep one canonical occurrence of
each pattern.
In `@packages/cli/src/commands/run.test.ts`:
- Around line 568-595: The test "renders output immediately via onOutput in
non-machine mode" asserts only for a checkmark but not the streamed text; update
the test to assert that the streamed output "Hello World" was written to stdout
by checking the spy used for process.stdout.write (stdoutWriteSpy) rather than
consoleLogSpy, e.g., verify that stdoutWriteSpy was called with or contains
"Hello World" after invoking action(HELLO_WORLD_FILE, {}); locate the
mockRunProject stub and the test block around it and add the assertion
referencing stdoutWriteSpy and the action/HELLO_WORLD_FILE test flow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: e921e22f-7913-46cf-8c89-02ee618c98cf
📒 Files selected for processing (3)
.gitignorepackages/cli/src/commands/run.test.tspackages/cli/src/commands/run.ts
Remove exclusion for PNG files in packages directory.
|
@coderabbitai what's pending? can you approve? |
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/cli/src/commands/run.ts (1)
1040-1094:⚠️ Potential issue | 🟡 MinorTrack line termination, not just “had streamed output”.
The main
print()path emitsstream.textwith a trailing newline. Here, oncehadStreamedOutputis true,prefixalways adds another\n, so normal line-oriented stdout ends up with an extra blank line before✓/✗. This needs per-block “last chunk ended with newline” state instead of a boolean that only says something streamed.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/cli/src/commands/run.ts` around lines 1040 - 1094, The boolean blocksWithStreamedOutput is too coarse — change streaming state to track whether the last streamed chunk for a block ended with a newline and use that in onBlockDone to avoid inserting an extra blank line; when onOutput(streaming) runs for a blockId set both state.blocksWithStreamedOutput.add(blockId) and state.blocksLastChunkEndedWithNewline.set(blockId, blockOutput.stream?.text?.endsWith('\n') ?? false) (use the same symbols onOutput, blockOutput), and in onBlockDone compute hadStreamedOutput = state.blocksWithStreamedOutput.delete(result.blockId) and lastEndedNewline = state.blocksLastChunkEndedWithNewline.get(result.blockId) then remove the map entry; change prefix to insert '\n' only when state.agentStreamed is true OR (hadStreamedOutput is true and lastEndedNewline is false) so you don't add an extra newline after output that already ended with '\n'.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/cli/src/commands/run.test.ts`:
- Around line 192-199: The mock implementation for mockRunProject calls
lifecycle callbacks fire-and-forget causing race conditions; change the
implementation in mockRunProject.mockImplementation to await the async callbacks
on options (await options?.onBlockStart?.(TEST_BLOCK, 0, 1), await
options?.onOutput?.(TEST_BLOCK.id, blockOutput) for each streamedOutputs
iteration, and await options?.onBlockDone?.(result)) so the mock resolves only
after all lifecycle handlers complete, ensuring tests wait for the same async
contract used in run.ts.
- Around line 609-653: The tests don't actually exercise the onOutput path
because they include the same chunk in result.outputs as well as streamedOutputs
and never assert that interactive rendering stayed silent in JSON mode; update
the two tests that call setupStreamingRun/action (the human-mode and
machine-mode cases) so that: for the human-mode test (the one calling
action(HELLO_WORLD_FILE, {})) provide the chunk only via streamedOutputs and
ensure result.outputs is empty so onOutput must do the rendering, then assert
stdoutWriteSpy (or getOutput(consoleLogSpy)) contains "Hello World" and the
success marker; for the machine-mode test (action(HELLO_WORLD_FILE, { output:
'json' })), keep the chunk in streamedOutputs but assert that no interactive
writes occurred (e.g., stdoutWriteSpy has no calls containing "Hello World") and
still parse getOutput(consoleLogSpy) to verify parsed.blocks[0].outputs contains
the chunk; change setupStreamingRun, test inputs, and assertions accordingly to
reference setupStreamingRun, action, HELLO_WORLD_FILE, stdoutWriteSpy,
consoleLogSpy, and getOutput.
---
Outside diff comments:
In `@packages/cli/src/commands/run.ts`:
- Around line 1040-1094: The boolean blocksWithStreamedOutput is too coarse —
change streaming state to track whether the last streamed chunk for a block
ended with a newline and use that in onBlockDone to avoid inserting an extra
blank line; when onOutput(streaming) runs for a blockId set both
state.blocksWithStreamedOutput.add(blockId) and
state.blocksLastChunkEndedWithNewline.set(blockId,
blockOutput.stream?.text?.endsWith('\n') ?? false) (use the same symbols
onOutput, blockOutput), and in onBlockDone compute hadStreamedOutput =
state.blocksWithStreamedOutput.delete(result.blockId) and lastEndedNewline =
state.blocksLastChunkEndedWithNewline.get(result.blockId) then remove the map
entry; change prefix to insert '\n' only when state.agentStreamed is true OR
(hadStreamedOutput is true and lastEndedNewline is false) so you don't add an
extra newline after output that already ended with '\n'.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 1119c44b-984a-442b-b48c-e609e8cd72bc
📒 Files selected for processing (2)
packages/cli/src/commands/run.test.tspackages/cli/src/commands/run.ts
…ote into feat/cli-streaming-output
✅ Actions performedComments resolved and changes approved. |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/cli/src/commands/run.test.ts`:
- Around line 658-673: The test currently leaves payloads in result.outputs
which allows non-stream fallback rendering to satisfy assertions; update the
tests (those invoking setupStreamingRun and any other cases around lines
referencing streamedOutputs/result) to remove result.outputs so the test
exercises stream-only paths — ensure setupStreamingRun calls supply
streamedOutputs but do not include outputs in the result object (and do the same
for the other case around lines 709–721) so only the streaming path (not
onBlockDone fallback rendering) can pass the assertions.
- Around line 595-755: Add unit tests in the existing "streaming output via
onOutput" suite to cover streamed stderr and special-character outputs: call
setupStreamingRun with streamedOutputs containing an output object with
output_type: 'stream', name: 'stderr' and text including unicode and newline
characters (e.g., "error ✓ 🚀\nsecond line"), then invoke
action(HELLO_WORLD_FILE, {}) and assert using stdoutWriteSpy (or a
stderrWriteSpy if present) and getLogCalls(consoleLogSpy) that stderr text is
handled distinctly from stdout (not rendered in machine JSON mode, included in
blocks outputs in JSON), that special characters and newlines are preserved in
rendered output, and that newline/blank-line sequencing behavior (the tests
using getLogCalls, serverReadyIdx, and checkIdx) still holds for stderr streams;
reuse existing helpers setupStreamingRun, action, getOutput, getLogCalls,
mockRunProject identifiers to locate where to add these cases.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 9c0e47a0-ffbb-420b-84d7-bc2acdc6085e
📒 Files selected for processing (1)
packages/cli/src/commands/run.test.ts
There was a problem hiding this comment.
♻️ Duplicate comments (1)
packages/cli/src/commands/run.test.ts (1)
649-659:⚠️ Potential issue | 🟠 Major | ⚡ Quick winMake these two assertions stream-only to prevent fallback false positives.
Line 655 and Line 703 keep payloads in
result.outputs, so both tests can still pass if streamed-state tracking regresses and rendering falls back toonBlockDone.Suggested fix
setupStreamingRun({ streamedOutputs: [...outputs], result: { blockId: TEST_BLOCK.id, blockType: TEST_BLOCK.type, success: true, - outputs: [...outputs], + outputs: [], executionCount: 1, durationMs: 50, }, }) @@ setupStreamingRun({ streamedOutputs: [output], result: { blockId: TEST_BLOCK.id, blockType: TEST_BLOCK.type, success: true, - outputs: [output], + outputs: [], executionCount: 1, durationMs: 50, }, })Also applies to: 697-707
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/cli/src/commands/run.test.ts` around lines 649 - 659, The tests currently include payloads in both streamedOutputs and result.outputs, allowing a fallback onBlockDone path to satisfy assertions; update the two failing tests to be stream-only by removing payloads from the result.outputs in the setupStreamingRun call (i.e., leave streamedOutputs populated but set result.outputs to an empty array or undefined) and then assert only against streamedOutputs/streaming state (not result.outputs). Target the setupStreamingRun usage and the assertions that reference streamed outputs in run.test.ts so the expectations rely solely on the streaming path rather than onBlockDone fallback.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@packages/cli/src/commands/run.test.ts`:
- Around line 649-659: The tests currently include payloads in both
streamedOutputs and result.outputs, allowing a fallback onBlockDone path to
satisfy assertions; update the two failing tests to be stream-only by removing
payloads from the result.outputs in the setupStreamingRun call (i.e., leave
streamedOutputs populated but set result.outputs to an empty array or undefined)
and then assert only against streamedOutputs/streaming state (not
result.outputs). Target the setupStreamingRun usage and the assertions that
reference streamed outputs in run.test.ts so the expectations rely solely on the
streaming path rather than onBlockDone fallback.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 65a45c22-94a4-4084-844c-92a4ba30ff04
📒 Files selected for processing (1)
packages/cli/src/commands/run.test.ts
You can test streaming using this project:
cli_streaming_output.deepnoteSummary
onOutputcallback from the execution engine to render block outputs immediately as they arrive from the Jupyter kernel, instead of batching them after block completionprint()updates), users now see output in real-time instead of waiting until the block finishes-o json,-o toon) are unchanged — they still collect outputs viaresult.outputsinonBlockDoneTest plan
-o jsonoutput includes outputs in blocks array (tested:does not render output via onOutput in machine output mode)[1/3] label ✓ (50ms)on one line (tested:keeps status on same line as label for blocks with no output)🤖 Generated with Claude Code
Summary by CodeRabbit
Bug Fixes
Tests