Skip to content

feat(ci): add code coverage pipeline and E2E mocking with aimock#4

Open
edelauna wants to merge 6 commits intomainfrom
feat-ci-code-coverage
Open

feat(ci): add code coverage pipeline and E2E mocking with aimock#4
edelauna wants to merge 6 commits intomainfrom
feat-ci-code-coverage

Conversation

@edelauna
Copy link
Copy Markdown

@edelauna edelauna commented Apr 27, 2026

Summary

  • Unit test coverage (code-qa.yml): Adds a matrix unit-test job (ubuntu + windows) running pnpm test:coverage across 6 workspaces: src, webview-ui, packages/core, packages/cloud, packages/telemetry, apps/cli. Each workspace gets @vitest/coverage-v8 and a test:coverage script producing coverage/lcov.info. A test:coverage Turbo task is added with outputs: ["coverage/**"] so unchanged packages are served from cache. CI caches .turbo/cache via actions/cache to skip vitest on unchanged packages on subsequent runs.

  • Codecov (codecov.yml): Uploads lcov files from both platforms using CODECOV_TOKEN. Project check uses target: auto (never regress below current baseline, 1% threshold). Patch check enforces 80% coverage on new lines (5% threshold). Requires CODECOV_TOKEN secret to be added to repo settings.

  • E2E mocking (e2e.yml): New workflow triggered on PRs touching src/**, webview-ui/**, apps/vscode-e2e/**, packages/core/**. Runs test:ci:mock via xvfb-run (headless VS Code, 30-minute timeout). Uses @copilotkit/aimock — a local HTTP server that replays committed fixture files. No OpenRouter API key needed. Fixtures are matched by substring on the last user message; three fixtures are committed covering the existing test suite.

  • approveCurrentAsk() API: Adds approveCurrentAsk() to RooCodeAPI (types + implementation). The E2E harness calls this to unblock completion_result asks without going through webview button routing. The primary button for completion_result routes to "Start New Task" by design — correct UX for users, not useful in a test harness.

  • packages/types CJS fix: ai-sdk-provider-poe is ESM-only; its ./code subpath has no require export condition. tsup externalises it by default, leaving a runtime require("ai-sdk-provider-poe/code") in the CJS bundle that Node can't resolve. Added noExternal: ["ai-sdk-provider-poe"] to packages/types/tsup.config.ts so esbuild bundles it inline. No-op in repos that don't import poe.ts.

  • runTest.ts hardening: Four fixes to the E2E test runner: (1) fail-fast guard when AIMOCK_RECORD=true but OPENROUTER_API_KEY is missing; (2) process.exitCode = 1 instead of process.exit(1) so the finally cleanup block runs on failure; (3) fs.mkdtemp() moved inside the try block so cleanup always runs even if workspace creation fails; (4) testWorkspace guarded with if (testWorkspace) in finally.

  • Test timeout bump: custom-tool-registry.spec.ts has two tests that dynamically import TypeScript files at runtime. The first cold compilation exceeds 120s on Windows under V8 coverage instrumentation. Both timeouts bumped from 120s to 300s to match observed worst-case CI runner performance.

Key implementation details

  • aimock matching: Fixtures are matched by substring on incoming_last_user_message. The <environment_details> block (which includes the current time) is always present — match strings must never include timestamps or paths. The modes test triggers a second API call whose last user message starts with <environment_details> directly; a programmatic regex fixture handles this in runTest.ts.
  • Record mode: OPENROUTER_API_KEY=<key> pnpm --filter @roo-code/vscode-e2e test:record proxies unmatched requests to OpenRouter and writes fixtures/openai-*.json. Copy the relevant block into a named fixture file, then delete the auto-recorded files (they are gitignored).
  • Turbo cache: actions/cache persists .turbo/cache between runs keyed by lockfile hash + SHA. On a rerun with no source changes, all packages hit Turbo cache and vitest never starts.

Test procedure

Coverage (local):

pnpm test:coverage

E2E mock replay (no API key):

pnpm --filter @roo-code/vscode-e2e test:ci:mock

Record new fixtures (requires OpenRouter key):

OPENROUTER_API_KEY=<key> pnpm --filter @roo-code/vscode-e2e test:record

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 27, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds V8 coverage collection and root test:coverage orchestration with Codecov upload; introduces LLMock-backed VS Code E2E record/replay harness with fixtures and a new E2E workflow; implements approveCurrentAsk() API and refactors test-runner cleanup/timeouts.

Changes

Cohort / File(s) Summary
CI Workflows
\.github/workflows/code-qa.yml, \.github/workflows/e2e.yml
Unit job now runs pnpm test:coverage, uploads multiple lcov.info files to Codecov with per-matrix flags and Turbo cache; new E2E workflow runs mocked headless tests under xvfb-run (30m timeout).
Root coverage orchestration
package.json, turbo.json, codecov.yml
Adds root test:coverage script, Turbo test:coverage task (outputs coverage/**), and Codecov config (precision/status/comment layout).
Vitest coverage scripts & configs
src/package.json, src/vitest.config.ts, webview-ui/package.json, webview-ui/vitest.config.ts, apps/cli/package.json, apps/cli/vitest.config.ts, packages/*/package.json (core, cloud, telemetry), packages/*/vitest.config.ts
Adds test:coverage scripts and @vitest/coverage-v8 devDependency where applicable; enables v8 coverage provider, reporters (text, lcov), and excludes tests/config/mocks from coverage.
E2E harness & runner changes
apps/vscode-e2e/src/runTest.ts, apps/vscode-e2e/src/suite/index.ts, apps/vscode-e2e/package.json
Adds LLMock-backed record/replay harness (random port), fixture loading/augmentation, AIMOCK_URL injection, OpenRouter switching for mock/record modes, test:ci:mock and test:record scripts, moves cleanup into finally, and replaces process.exit(1) with process.exitCode = 1.
E2E fixtures & docs
apps/vscode-e2e/AGENTS.md, apps/vscode-e2e/fixtures/.gitignore, apps/vscode-e2e/fixtures/markdown-lists.json, apps/vscode-e2e/fixtures/modes.json, apps/vscode-e2e/fixtures/task-hello-world.json
Adds AGENTS guide explaining Aimock record/replay, ignores auto-recorded openai-*.json, and introduces new fixtures (lists, modes, hello-world).
Public API & bundling
packages/types/src/api.ts, src/extension/api.ts, packages/types/tsup.config.ts
Adds approveCurrentAsk(): Promise<void> to RooCodeAPI and implements it in the extension API; updates tsup to bundle ai-sdk-provider-poe inline.
Tests: timeouts
packages/core/src/custom-tools/__tests__/custom-tool-registry.spec.ts
Increases two per-test timeouts from 120_000ms to 300_000ms.
Minor test runner outputs
apps/vscode-e2e/src/runTest.ts
Registers an additional regex-anchored fixture for a post-switch_mode call during replay mode.

Sequence Diagram(s)

sequenceDiagram
    participant Runner as Test Runner
    participant LLMock as LLMock Server
    participant VSCode as VS Code Extension
    participant Fixtures as Fixture Store

    Runner->>Runner: Read env (AIMOCK_RECORD, USE_MOCK, OPENROUTER_API_KEY, AIMOCK_URL)
    alt Record Mode
        Runner->>LLMock: Start LLMock (record/proxy real API)
    else Replay/Mock Mode
        Runner->>Fixtures: Load fixtures/
        Runner->>LLMock: Start LLMock (replay mode)
        LLMock->>Fixtures: Load fixture mappings
    end
    Runner->>VSCode: Launch extension with AIMOCK_URL & API key override
    VSCode->>LLMock: Send LLM request
    alt Fixture matched
        LLMock->>Fixtures: Match by substring/regex
        Fixtures-->>LLMock: Return fixture response (toolCalls)
    else No match & Record Mode
        LLMock->>LLMock: Proxy to real OpenRouter and persist openai-*.json
    end
    LLMock-->>VSCode: Return mocked/recorded response
    VSCode->>VSCode: Emit ask/completion events
    Runner->>VSCode: Call approveCurrentAsk() to auto-approve ask
    Runner->>LLMock: Shutdown and cleanup
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Poem

🐰 I hopped through fixtures, JSON and cheer,
Mocked LLMs whispering answers near,
Coverage counters tapped each tiny quest,
Headless tests ran steady, timeouts blessed,
I clicked "approve" — the ask may now rest.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main changes: adding code coverage pipeline and E2E mocking infrastructure.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed PR description is comprehensive, detailed, and well-structured with clear sections covering summary, implementation details, and test procedures that align with the description template sections.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat-ci-code-coverage

Comment @coderabbitai help to get the list of available commands and usage tips.

@edelauna edelauna changed the title Feat ci code coverage feat(ci): add code coverage pipeline and E2E mocking with aimock Apr 27, 2026
@edelauna edelauna force-pushed the feat-ci-code-coverage branch from 8866a06 to dea2378 Compare April 27, 2026 02:02
@edelauna edelauna marked this pull request as ready for review April 27, 2026 02:06
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (2)
codecov.yml (1)

11-12: Patch gate is not strictly 80% with current threshold.

With target: 80% and threshold: 5%, PRs can pass around ~75% patch coverage. If you want strict enforcement, set threshold to 0%.

Suggested tweak
 patch:
   default:
     target: 80%
-    threshold: 5%
+    threshold: 0%
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@codecov.yml` around lines 11 - 12, The codecov patch gate allows ~75% patch
coverage because target: 80% combined with threshold: 5% permits a 5% drop;
change the threshold key in the codecov.yml from 5% to 0% so the patch coverage
requirement is strictly enforced at the configured target (update the threshold:
value to 0% next to the existing target: 80% entry).
.github/workflows/e2e.yml (1)

8-13: Path filter is a bit narrow for E2E reliability.

Dependency/infrastructure changes outside these folders can still break mocked E2E. Consider including root dependency/build files (package.json, pnpm-lock.yaml, turbo.json, and possibly .github/actions/setup-node-pnpm/**) in trigger paths.

Suggested path expansion
         paths:
             - "src/**"
             - "webview-ui/**"
             - "apps/vscode-e2e/**"
             - "packages/core/**"
+            - "package.json"
+            - "pnpm-lock.yaml"
+            - "turbo.json"
+            - ".github/actions/setup-node-pnpm/**"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/e2e.yml around lines 8 - 13, Update the e2e workflow path
filters to include root dependency/build files so infra changes trigger E2E:
modify the paths array in .github/workflows/e2e.yml (the paths: block) to add
entries for "package.json", "pnpm-lock.yaml", "turbo.json", and the actions
directory ".github/actions/setup-node-pnpm/**" (in addition to the existing
"src/**", "webview-ui/**", "apps/vscode-e2e/**", "packages/core/**").
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@apps/vscode-e2e/AGENTS.md`:
- Around line 20-21: The command examples in AGENTS.md use inconsistent
working-directory conventions (e.g., the git clean command with
apps/vscode-e2e/fixtures/ vs later examples using fixtures/... as if cwd is
apps/vscode-e2e/); pick one convention and make all examples consistent — either
prefix every path with apps/vscode-e2e/ (so commands like git clean -fx
apps/vscode-e2e/fixtures/) or change all to relative to apps/vscode-e2e/ (cd
apps/vscode-e2e && git clean -fx fixtures/); update the affected lines (the git
clean example and the later fixtures/... examples) so copy/paste onboarding
steps always work.

In `@apps/vscode-e2e/src/runTest.ts`:
- Around line 93-99: The catch in runTest.ts currently calls process.exit(1)
which aborts the event loop and prevents the awaited cleanup in the finally
block (await fs.rm and await mock?.stop()) from running; change the catch to set
process.exitCode = 1 instead of calling process.exit(1) so the process can
finish pending async teardown in the finally block (inside the same
try/catch/finally around the test-run logic and cleanup).

In `@apps/vscode-e2e/src/suite/index.ts`:
- Around line 20-28: The code may call api.setConfiguration with
openRouterApiKey set to process.env.OPENROUTER_API_KEY! which can be undefined
in record mode; before calling setConfiguration, add an explicit guard that
checks isRecord (the isRecord constant) and verifies
process.env.OPENROUTER_API_KEY is present, and if missing throw a clear error or
exit (with a message mentioning OPENROUTER_API_KEY and record mode) so the
failure is immediate and actionable; keep the rest of the setConfiguration logic
(including aimockUrl handling and openRouterBaseUrl) unchanged.

In `@packages/types/tsup.config.ts`:
- Around line 11-13: Remove the stray noExternal entry referencing
"ai-sdk-provider-poe" from the tsup config in packages/types (the noExternal
array), because `@roo-code/types` does not declare or import that package; either
delete that array element or remove the entire noExternal key if unused, and if
the package is actually required add it to this package's dependencies instead
of keeping it in noExternal.

---

Nitpick comments:
In @.github/workflows/e2e.yml:
- Around line 8-13: Update the e2e workflow path filters to include root
dependency/build files so infra changes trigger E2E: modify the paths array in
.github/workflows/e2e.yml (the paths: block) to add entries for "package.json",
"pnpm-lock.yaml", "turbo.json", and the actions directory
".github/actions/setup-node-pnpm/**" (in addition to the existing "src/**",
"webview-ui/**", "apps/vscode-e2e/**", "packages/core/**").

In `@codecov.yml`:
- Around line 11-12: The codecov patch gate allows ~75% patch coverage because
target: 80% combined with threshold: 5% permits a 5% drop; change the threshold
key in the codecov.yml from 5% to 0% so the patch coverage requirement is
strictly enforced at the configured target (update the threshold: value to 0%
next to the existing target: 80% entry).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: f6facfb6-7de1-477d-afdc-242ba3b61259

📥 Commits

Reviewing files that changed from the base of the PR and between 96d6e43 and dea2378.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (29)
  • .github/workflows/code-qa.yml
  • .github/workflows/e2e.yml
  • apps/cli/package.json
  • apps/cli/vitest.config.ts
  • apps/vscode-e2e/AGENTS.md
  • apps/vscode-e2e/fixtures/.gitignore
  • apps/vscode-e2e/fixtures/.gitkeep
  • apps/vscode-e2e/fixtures/markdown-lists.json
  • apps/vscode-e2e/fixtures/modes.json
  • apps/vscode-e2e/fixtures/task-hello-world.json
  • apps/vscode-e2e/package.json
  • apps/vscode-e2e/src/runTest.ts
  • apps/vscode-e2e/src/suite/index.ts
  • codecov.yml
  • package.json
  • packages/cloud/package.json
  • packages/cloud/vitest.config.ts
  • packages/core/package.json
  • packages/core/vitest.config.ts
  • packages/telemetry/package.json
  • packages/telemetry/vitest.config.ts
  • packages/types/src/api.ts
  • packages/types/tsup.config.ts
  • src/extension/api.ts
  • src/package.json
  • src/vitest.config.ts
  • turbo.json
  • webview-ui/package.json
  • webview-ui/vitest.config.ts

Comment thread apps/vscode-e2e/AGENTS.md
Comment thread apps/vscode-e2e/src/runTest.ts
Comment thread apps/vscode-e2e/src/suite/index.ts
Comment on lines +11 to +13
// ai-sdk-provider-poe is ESM-only (./code subpath has no "require" condition)
// so tsup must bundle it inline rather than emit a runtime require() call.
noExternal: ["ai-sdk-provider-poe"],
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify whether `@roo-code/types` actually depends on or imports ai-sdk-provider-poe.

set -euo pipefail

echo "== packages/types/package.json dependency check =="
cat packages/types/package.json | jq '{dependencies, devDependencies, peerDependencies}'

echo
echo "== references to ai-sdk-provider-poe inside packages/types =="
rg -n -C2 --type=ts --type=js --type=json 'ai-sdk-provider-poe' packages/types || true

echo
echo "Expected:"
echo "- If no dependency/import exists in packages/types, remove noExternal from this file."
echo "- If imports exist, ensure dependency is declared in packages/types/package.json."

Repository: Zoo-Code-Org/Zoo-Code

Length of output: 1086


🏁 Script executed:

# Search entire workspace for ai-sdk-provider-poe
rg -n 'ai-sdk-provider-poe' --type=ts --type=js --type=json

Repository: Zoo-Code-Org/Zoo-Code

Length of output: 249


🏁 Script executed:

# Check all tsup.config files to see if this noExternal appears elsewhere
fd -e tsup.config.ts -e tsup.config.js

Repository: Zoo-Code-Org/Zoo-Code

Length of output: 47


🏁 Script executed:

# Check if ai-sdk-provider-poe is in workspace root package.json or any root dependencies
cat package.json | jq '{dependencies, devDependencies, peerDependencies}' || true

Repository: Zoo-Code-Org/Zoo-Code

Length of output: 701


🏁 Script executed:

# Check all source files (excluding node_modules and config files) for ai-sdk-provider-poe
find packages/types -name "*.ts" -o -name "*.tsx" -o -name "*.js" -o -name "*.jsx" | grep -v node_modules | xargs rg 'ai-sdk-provider-poe' || echo "No matches found"

Repository: Zoo-Code-Org/Zoo-Code

Length of output: 243


🏁 Script executed:

# Let's also look at the entire packages/types structure to understand what this package actually exports
ls -la packages/types/src/ 2>/dev/null || echo "No src directory"
head -50 packages/types/tsup.config.ts

Repository: Zoo-Code-Org/Zoo-Code

Length of output: 2817


Remove the noExternal entry for ai-sdk-provider-poe—this package is not a dependency of @roo-code/types.

The noExternal configuration on line 13 references a package that @roo-code/types neither declares as a dependency nor imports anywhere in its codebase. This config should be removed unless ai-sdk-provider-poe is actually used by this package and needs to be added as a dependency.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/types/tsup.config.ts` around lines 11 - 13, Remove the stray
noExternal entry referencing "ai-sdk-provider-poe" from the tsup config in
packages/types (the noExternal array), because `@roo-code/types` does not declare
or import that package; either delete that array element or remove the entire
noExternal key if unused, and if the package is actually required add it to this
package's dependencies instead of keeping it in noExternal.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The entry is needed. CodeRabbit's search ran against the PR diff only and didn't see packages/types/src/providers/poe.ts, which lives in zoo/main (this repo's Poe provider addition). That file re-exports poeDefaultModelId, POE_DEFAULT_BASE_URL, and getPoeDefaultModelInfo directly from ai-sdk-provider-poe/code. The package is declared as a dependency in packages/types/package.json (pinned at 2.0.18) and its ./code subpath is ESM-only — no require export condition. Without noExternal, tsup externalises it and the built dist/index.cjs emits a runtime require("ai-sdk-provider-poe/code") that Node can't resolve (confirmed by the previous CI failure: Package subpath './code' is not defined by "exports"). Keeping the entry as-is.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only users with a collaborator, contributor, member, or owner role can interact with CodeRabbit.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@apps/vscode-e2e/src/runTest.ts`:
- Around line 9-12: When AIMOCK_RECORD is set (isRecord true) but
process.env.OPENROUTER_API_KEY is missing, add an early fail-fast check in
runTest.ts that throws or exits with a clear error message; locate the block
that defines isRecord and useMock and if isRecord &&
!process.env.OPENROUTER_API_KEY then call process.exit(1) or throw a descriptive
Error indicating OPENROUTER_API_KEY is required for recording (mention
AIMOCK_RECORD) so recording mode fails immediately rather than producing
downstream errors.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 12360661-eeaa-42af-8668-1c188d7ba9a5

📥 Commits

Reviewing files that changed from the base of the PR and between dea2378 and 64396e9.

📒 Files selected for processing (3)
  • apps/vscode-e2e/AGENTS.md
  • apps/vscode-e2e/src/runTest.ts
  • apps/vscode-e2e/src/suite/index.ts
✅ Files skipped from review due to trivial changes (1)
  • apps/vscode-e2e/AGENTS.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • apps/vscode-e2e/src/suite/index.ts

Comment thread apps/vscode-e2e/src/runTest.ts
@edelauna edelauna force-pushed the feat-ci-code-coverage branch from 64396e9 to f8d0666 Compare April 27, 2026 02:53
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@apps/vscode-e2e/src/runTest.ts`:
- Around line 59-73: Move resource acquisition into the existing try/finally so
cleanup always runs: call mock.start() and create the temporary workspace
(fs.mkdtemp(...) that assigns testWorkspace) inside the same try block that
already ensures mock?.stop() and workspace cleanup in the finally; ensure
references to mock.start(), mock?.stop(), and the testWorkspace variable (used
later for cleanup) are updated so both startup failures and normal runs trigger
the finally cleanup.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 334a27dd-dc8d-4fd4-9e85-508ee6de55d8

📥 Commits

Reviewing files that changed from the base of the PR and between 64396e9 and f8d0666.

📒 Files selected for processing (4)
  • apps/vscode-e2e/AGENTS.md
  • apps/vscode-e2e/src/runTest.ts
  • apps/vscode-e2e/src/suite/index.ts
  • packages/core/src/custom-tools/__tests__/custom-tool-registry.spec.ts
✅ Files skipped from review due to trivial changes (2)
  • packages/core/src/custom-tools/tests/custom-tool-registry.spec.ts
  • apps/vscode-e2e/AGENTS.md

Comment thread apps/vscode-e2e/src/runTest.ts
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
apps/vscode-e2e/src/runTest.ts (1)

21-60: ⚠️ Potential issue | 🟡 Minor

Move LLMock startup under the existing try/finally.

mock.start() still runs before Line 72, so a startup failure skips both the catch logging and the finally cleanup path. That reopens the teardown gap for mock initialization errors.

♻️ Proposed fix
 let mock: InstanceType<typeof LLMock> | undefined
+let testWorkspace: string | undefined

- if (useMock) {
- 	const fixturesDir = path.resolve(__dirname, "../fixtures")
-
- 	mock = new LLMock({
- 		port: 0, // random free port
- 		...(isRecord && {
- 			record: {
- 				providers: { openai: "https://openrouter.ai/api" },
- 				fixturePath: fixturesDir,
- 			},
- 		}),
- 	})
-
- 	mock.loadFixtureDir(fixturesDir)
-
- 	if (!isRecord) {
- 		mock.addFixture({
- 			match: { userMessage: /^<environment_details>/ },
- 			response: {
- 				toolCalls: [
- 					{
- 						name: "attempt_completion",
- 						arguments: JSON.stringify({ result: "Switched to ❓ Ask mode as requested." }),
- 						id: "call_modes_post_switch_001",
- 					},
- 				],
- 			},
- 		})
- 	}
-
- 	await mock.start()
- }
-
- let testWorkspace: string | undefined
-
 try {
+	if (useMock) {
+		const fixturesDir = path.resolve(__dirname, "../fixtures")
+
+		mock = new LLMock({
+			port: 0, // random free port
+			...(isRecord && {
+				record: {
+					providers: { openai: "https://openrouter.ai/api" },
+					fixturePath: fixturesDir,
+				},
+			}),
+		})
+
+		mock.loadFixtureDir(fixturesDir)
+
+		if (!isRecord) {
+			mock.addFixture({
+				match: { userMessage: /^<environment_details>/ },
+				response: {
+					toolCalls: [
+						{
+							name: "attempt_completion",
+							arguments: JSON.stringify({ result: "Switched to ❓ Ask mode as requested." }),
+							id: "call_modes_post_switch_001",
+						},
+					],
+				},
+			})
+		}
+
+		await mock.start()
+	}
+
 	// Create a temporary workspace folder for tests
 	testWorkspace = await fs.mkdtemp(path.join(os.tmpdir(), "roo-test-workspace-"))
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/vscode-e2e/src/runTest.ts` around lines 21 - 60, The LLMock is started
before entering the existing try/finally so failures in mock.start() bypass the
catch and finally cleanup; move the mock.start() call into the same try block
that wraps the test run (so startup errors are handled by the existing catch and
ensure mock.stop()/cleanup in finally), keeping mock.loadFixtureDir(...) and
mock.addFixture(...) where they are but calling mock.start() after those within
the try; reference LLMock instance "mock" and its start/stop methods
(mock.start, mock.stop) to locate and move the call.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@apps/vscode-e2e/src/runTest.ts`:
- Around line 21-60: The LLMock is started before entering the existing
try/finally so failures in mock.start() bypass the catch and finally cleanup;
move the mock.start() call into the same try block that wraps the test run (so
startup errors are handled by the existing catch and ensure mock.stop()/cleanup
in finally), keeping mock.loadFixtureDir(...) and mock.addFixture(...) where
they are but calling mock.start() after those within the try; reference LLMock
instance "mock" and its start/stop methods (mock.start, mock.stop) to locate and
move the call.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 1f2c7e9e-ac8f-40c8-8fe0-b7cd79a6fab6

📥 Commits

Reviewing files that changed from the base of the PR and between f8d0666 and 84e16e9.

📒 Files selected for processing (1)
  • apps/vscode-e2e/src/runTest.ts

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
.github/workflows/code-qa.yml (1)

61-68: Cache key at Line 65 is too granular and reduces cache reuse.

Including ${{ github.sha }} creates a new cache on every commit. Restore keys help, but this still increases churn and cache pressure unnecessarily.

♻️ Suggested cache-key adjustment
-                  key: ${{ runner.os }}-turbo-${{ hashFiles('**/pnpm-lock.yaml') }}-${{ github.sha }}
+                  key: ${{ runner.os }}-turbo-${{ hashFiles('**/pnpm-lock.yaml') }}
                   restore-keys: |
                       ${{ runner.os }}-turbo-${{ hashFiles('**/pnpm-lock.yaml') }}-
                       ${{ runner.os }}-turbo-
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/code-qa.yml around lines 61 - 68, The cache key for the
"Cache Turbo" step is too granular because it includes `${{ github.sha }}`,
causing a new cache per commit; update the key used by the actions/cache step
(the `key:` under the "Cache Turbo" step that references `runner.os` and
`hashFiles('**/pnpm-lock.yaml')`) to remove `${{ github.sha }}` and rely on
`runner.os` plus the lockfile hash (e.g., `${{ runner.os }}-turbo-${{
hashFiles('**/pnpm-lock.yaml') }}`) so caches are reused across commits, and
adjust the `restore-keys` to match the less specific prefixes already present
for `.turbo/cache`.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In @.github/workflows/code-qa.yml:
- Around line 61-68: The cache key for the "Cache Turbo" step is too granular
because it includes `${{ github.sha }}`, causing a new cache per commit; update
the key used by the actions/cache step (the `key:` under the "Cache Turbo" step
that references `runner.os` and `hashFiles('**/pnpm-lock.yaml')`) to remove `${{
github.sha }}` and rely on `runner.os` plus the lockfile hash (e.g., `${{
runner.os }}-turbo-${{ hashFiles('**/pnpm-lock.yaml') }}`) so caches are reused
across commits, and adjust the `restore-keys` to match the less specific
prefixes already present for `.turbo/cache`.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 03caa32e-ff21-4369-aeb0-a8d30a04ed5d

📥 Commits

Reviewing files that changed from the base of the PR and between 84e16e9 and 791ae33.

📒 Files selected for processing (1)
  • .github/workflows/code-qa.yml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant