Add skill tracking + silent-run feedback for workflow_dispatch#778
Merged
Add skill tracking + silent-run feedback for workflow_dispatch#778
Conversation
claude-code-action@v1 rejects track_progress: true on workflow_dispatch events, so manual retries and bootstrap dispatches leave no real-time signal on the PR that the skill is running -- reviewers had to hunt through the Actions tab to see progress. On pull_request runs the action posts its own tracking comment, so this gap only affects workflow_dispatch. Two workflow_dispatch-only comments: - Before skill_gen: a "Claude Opus is generating docs updates for <project> <tag>. Follow progress: <run URL>" placeholder. Static but enough to signal "yes it started". - After Augment PR body: a run summary with skill_gen / skill_review / autofix conclusions and a link to the run for the Claude Code Report. Runs even if earlier steps failed so reviewers can see which step died. Both are gated on `github.event_name == 'workflow_dispatch'` to avoid duplicating the action's own tracking on Renovate-opened PRs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves reviewer visibility for manually triggered (workflow_dispatch) runs of the upstream-release-docs workflow by adding explicit PR timeline comments, compensating for claude-code-action@v1 not supporting track_progress on that event type.
Changes:
- Post a “skill started” placeholder PR comment at the beginning of
workflow_dispatchruns. - Post an “always()” completion summary PR comment at the end of
workflow_dispatchruns with step conclusions and a run link.
Companion to the tracking comments: when the skill concludes
successfully but produces zero commits, the PR body previously
read as if content was added. Reviewers had no way to distinguish
"skill ran silently" from "skill produced stuff".
Adds:
- A new step that rev-counts commits between pre_skill SHA and
HEAD. Output exposed as steps.skill_commits.outputs.count.
- A [!NOTE] block in the PR body when SKILL_COMMIT_COUNT=0 and
no NO_CHANGES.md was written. Lists the three likely causes
(docs already ahead of pin; no doc-relevant release; skill's
verification matched existing prose).
- A row in the workflow_dispatch summary comment showing the
commit count.
Surfaced by the e2e test on PR #777: the skill ran to success but
produced nothing because main was already ahead of the pinned
scratch branch. PR body looked like a normal augmented PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
rdimitrov
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
Improves reviewer-facing feedback for the Upstream Release Docs GitHub Actions workflow, especially for workflow_dispatch runs where claude-code-action@v1 can’t use track_progress, and for “silent” successful runs that produce no content commits.
Changes:
- Add a pre-skill placeholder PR comment for
workflow_dispatchruns to surface that generation is in progress. - Add a post-run PR comment (always runs) summarizing step conclusions and the number of skill commits.
- Detect “silent” runs (0 skill commits + no
NO_CHANGES.md) and inject a[!NOTE]block into the PR body explaining likely causes.
Two fixes from Copilot's line-level comments: 1. Pre-skill placeholder comment step is best-effort. Was previously unguarded -- a transient gh failure (rate limit, API hiccup, permission edge case) would abort the workflow before skill_gen ran. Matches the `|| true` pattern already used by the post-run summary comment and the augmentation-failure comment. 2. Silent-run [!NOTE] is gated on BOTH skill steps having succeeded. The note claims "ran to success"; previously the condition only checked SKILL_COMMIT_COUNT=0 and NOTE_BLOCK empty, which could trigger the note even if skill_review had failed. Now requires steps.skill_gen.conclusion == 'success' AND steps.skill_review.conclusion == 'success'. Partial failures are already covered by the separate augmentation-failure comment step. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
JAORMX
approved these changes
Apr 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three feedback-visibility improvements, all surfaced by the e2e test on PR #777.
1. Pre-skill placeholder comment (workflow_dispatch only)
`claude-code-action@v1` rejects `track_progress: true` on `workflow_dispatch` events with a hard error ("track_progress is only supported for events: pull_request, issue_comment, ..."). On manual retries and bootstrap dispatches, the PR gets no real-time "Claude is working" comment -- reviewers had to open the Actions tab to see whether the skill was actually running.
Posts a simple placeholder before `skill_gen`:
Gated on `github.event_name == 'workflow_dispatch'` so Renovate-opened PRs keep the action's native tracking.
2. Post-augmentation summary comment (workflow_dispatch only)
Posts a summary table after the PR body has been augmented:
Runs with `if: always()` so it fires even if an earlier step failed.
3. Silent-run [!NOTE] in the PR body (both triggers)
The e2e test on PR #777 ran to success but produced zero content commits -- because main was already ahead of the scratch branch's pinned version, so the skill correctly found nothing new to document. Problem: the PR body still read like a normal augmented PR with "content additions by upstream-release-docs" etc. Reviewers had no signal that nothing had actually been added.
New step (`skill_commits`) rev-counts commits between `pre_skill` SHA and HEAD. When the count is zero AND no `NO_CHANGES.md` was written, the "Augment PR body" step inserts a `[!NOTE]` block listing the three likely causes:
Applies to both `pull_request` and `workflow_dispatch` runs -- the silence problem isn't specific to the test path.
Validation
Once merged, dispatching the workflow against the e2e-test scratch branch a second time (or any silent release) should produce a PR that clearly says "no content added and here's why".