End-to-end tests for the Freebuff CLI binary. Tests verify that the compiled binary works correctly by interacting with it via tmux.
Two testing approaches are supported:
Use the FreebuffSession class to start the binary in tmux, send commands, capture output, and assert directly.
import { describe, test, expect, afterEach } from 'bun:test'
import { FreebuffSession, requireFreebuffBinary } from '../utils'
describe('My Feature', () => {
let session: FreebuffSession | null = null
afterEach(async () => {
if (session) await session.stop()
session = null
})
test('works correctly', async () => {
const binary = requireFreebuffBinary()
session = await FreebuffSession.start(binary)
await session.send('/help')
const output = await session.capture(2)
expect(output).toContain('Shortcuts')
}, 60_000)
})Use the Codebuff SDK to run a testing agent that interacts with Freebuff via custom tmux tools. The agent reasons about the CLI output and verifies complex behaviors.
import { describe, test, expect, afterEach } from 'bun:test'
import { CodebuffClient } from '@codebuff/sdk'
import { freebuffTesterAgent } from '../agent/freebuff-tester'
import { createFreebuffTmuxTools, requireFreebuffBinary } from '../utils'
describe('Agent Test', () => {
let cleanup: (() => Promise<void>) | null = null
afterEach(async () => {
if (cleanup) await cleanup()
cleanup = null
})
test('verifies startup', async () => {
const apiKey = process.env.CODEBUFF_API_KEY
if (!apiKey) return // Skip if no API key
const binary = requireFreebuffBinary()
const tmuxTools = createFreebuffTmuxTools(binary)
cleanup = tmuxTools.cleanup
const client = new CodebuffClient({ apiKey })
const result = await client.run({
agent: freebuffTesterAgent.id,
prompt: 'Start Freebuff and verify the branding is correct.',
agentDefinitions: [freebuffTesterAgent],
customToolDefinitions: tmuxTools.tools,
handleEvent: () => {},
})
expect(result.output.type).not.toBe('error')
}, 180_000)
})- tmux must be installed:
brew install tmux(macOS) orsudo apt-get install tmux(Ubuntu) - Freebuff binary must be built:
bun freebuff/cli/build.ts 0.0.0-dev - SDK built (for agent tests):
cd sdk && bun run build - CODEBUFF_API_KEY (for agent tests only): Set this environment variable
bun freebuff/cli/build.ts 0.0.0-devbun test freebuff/e2e/tests/bun test freebuff/e2e/tests/version.e2e.test.ts
bun test freebuff/e2e/tests/startup.e2e.test.ts
bun test freebuff/e2e/tests/help-command.e2e.test.ts
bun test freebuff/e2e/tests/agent-startup.e2e.test.tsFREEBUFF_BINARY=/path/to/freebuff bun test freebuff/e2e/tests/- Create a new file in
freebuff/e2e/tests/with the naming convention<feature>.e2e.test.ts - Add the test name to
.github/workflows/freebuff-e2e.ymlmatrix:
matrix:
test:
- version
- startup
- help-command
- agent-startup
- your-new-test # <-- add here- The test will automatically run in parallel with other tests in CI.
The .github/workflows/freebuff-e2e.yml workflow:
- Builds the Freebuff binary once (linux-x64)
- Runs each test file in parallel via GitHub Actions matrix strategy
- Uploads tmux session logs on failure for debugging
Triggers:
- Nightly at 6:00 AM PT
- Manual via workflow_dispatch
| Method | Description |
|---|---|
FreebuffSession.start(binaryPath) |
Start binary in tmux, returns session |
session.send(text) |
Send text input (presses Enter) |
session.sendKey(key) |
Send special key (e.g. 'C-c', 'Escape') |
session.capture(waitSec?) |
Capture terminal output |
session.captureLabeled(label, waitSec?) |
Capture and save to session logs |
session.waitForText(pattern, timeoutMs?) |
Poll until text appears |
session.stop() |
Stop session and clean up |
Creates SDK custom tools for agent-driven testing:
start_freebuff- Launch the CLIsend_to_freebuff- Send text inputcapture_freebuff_output- Capture terminal outputstop_freebuff- Stop and clean up
| Function | Description |
|---|---|
requireFreebuffBinary() |
Get binary path, throws if not found |
getFreebuffBinaryPath() |
Get binary path (may not exist) |