Skip to content

feat(cli): show eval progress durations in ms#1187

Draft
christso wants to merge 4 commits intomainfrom
feat/1186-progress-durations-ms
Draft

feat(cli): show eval progress durations in ms#1187
christso wants to merge 4 commits intomainfrom
feat/1186-progress-durations-ms

Conversation

@christso
Copy link
Copy Markdown
Collaborator

@christso christso commented Apr 28, 2026

Summary

  • show compact timing on completed/failed eval progress lines as τ agent/totalms when both metrics are available
  • thread durationMs and evalRun.durationMs through core progress events into the CLI progress display
  • normalize the displayed total so it never renders lower than the displayed agent duration
  • add focused progress display tests for compact formatting, normalization, and missing-metrics behavior

Closes #1186.

Root Cause

The eval runner already computed both per-agent duration (result.durationMs) and full eval duration (result.evalRun.durationMs), but the progress event shape and CLI formatter dropped those values before rendering the completed result line. Once surfaced, the raw numbers could still be misleading because provider-reported agent duration can exceed local wall-clock eval timing for mock or remote targets.

Validation

  • bun test apps/cli/test/commands/eval/progress-display.test.ts
  • bun test apps/cli/test/commands/eval/statistics-inconclusive.test.ts
  • bun --filter @agentv/core typecheck
  • bun --filter agentv typecheck
  • git push pre-push hook passed: build, typecheck, lint, test, validate example evals

Red/Green UAT

Command used in both cases:

NO_COLOR=1 bun apps/cli/dist/cli.js eval examples/features/execution-metrics/evals/dataset.eval.yaml --target mock_metrics_agent --test-id simple-thresholds-pass --workers 1

Before (main):

0/1   🔄 simple-thresholds-pass | mock_metrics_agent
1/1   ✅ simple-thresholds-pass | mock_metrics_agent | 100% PASS

After (feat/1186-progress-durations-ms):

0/1   🔄 simple-thresholds-pass | mock_metrics_agent
1/1   ✅ simple-thresholds-pass | mock_metrics_agent | 100% PASS | τ 245/245ms

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented Apr 28, 2026

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 020c420
Status: ✅  Deploy successful!
Preview URL: https://1f089c28.agentv.pages.dev
Branch Preview URL: https://feat-1186-progress-durations.agentv.pages.dev

View logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: show agent and total durations in eval progress output

1 participant