Run all system-tests scenario even if empty when running on master/sc…#11185
Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit intomasterfrom Apr 24, 2026
Merged
Conversation
Contributor
|
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
nccatoni
added
type: enhancement
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9b0e5dc028
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| scenarios_groups: tracer-release | ||
| excluded_scenarios: APM_TRACING_E2E_OTEL,APM_TRACING_E2E_SINGLE_SPAN,PROFILING # exclude flaky scenarios | ||
| skip_empty_scenarios: true | ||
| skip_empty_scenarios: ${{ github.event_name != 'push' && github.event_name != 'schedule' }} |
There was a problem hiding this comment.
When this workflow is launched manually from master, .github/workflows/run-system-tests.yaml still has github.event_name == 'workflow_dispatch', so this expression evaluates to true and the reusable system-tests workflow continues skipping empty scenarios. That leaves the documented/manual master path without the new coverage intended for master runs; include a github.ref/github.ref_name check or otherwise treat workflow_dispatch on master like the push/schedule cases.
Useful? React with 👍 / 👎.
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 12 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~9b0e5dc028, baseline=1.62.0-SNAPSHOT~44874e2a03
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.065 s) : 0, 1064537
Total [baseline] (11.013 s) : 0, 11012706
Agent [candidate] (1.065 s) : 0, 1065320
Total [candidate] (11.069 s) : 0, 11068889
section appsec
Agent [baseline] (1.265 s) : 0, 1264870
Total [baseline] (11.076 s) : 0, 11076491
Agent [candidate] (1.266 s) : 0, 1266034
Total [candidate] (11.08 s) : 0, 11080347
section iast
Agent [baseline] (1.234 s) : 0, 1234168
Total [baseline] (11.274 s) : 0, 11273619
Agent [candidate] (1.236 s) : 0, 1236145
Total [candidate] (11.312 s) : 0, 11311538
section profiling
Agent [baseline] (1.188 s) : 0, 1188137
Total [baseline] (11.082 s) : 0, 11081664
Agent [candidate] (1.184 s) : 0, 1184310
Total [candidate] (11.02 s) : 0, 11019973
gantt
title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~9b0e5dc028, baseline=1.62.0-SNAPSHOT~44874e2a03
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.239 ms) : 0, 1239
crashtracking [candidate] (1.233 ms) : 0, 1233
BytebuddyAgent [baseline] (638.095 ms) : 0, 638095
BytebuddyAgent [candidate] (638.637 ms) : 0, 638637
AgentMeter [baseline] (29.541 ms) : 0, 29541
AgentMeter [candidate] (29.718 ms) : 0, 29718
GlobalTracer [baseline] (249.572 ms) : 0, 249572
GlobalTracer [candidate] (250.464 ms) : 0, 250464
AppSec [baseline] (32.252 ms) : 0, 32252
AppSec [candidate] (32.586 ms) : 0, 32586
Debugger [baseline] (59.828 ms) : 0, 59828
Debugger [candidate] (60.203 ms) : 0, 60203
Remote Config [baseline] (588.424 µs) : 0, 588
Remote Config [candidate] (599.854 µs) : 0, 600
Telemetry [baseline] (9.559 ms) : 0, 9559
Telemetry [candidate] (8.093 ms) : 0, 8093
Flare Poller [baseline] (7.535 ms) : 0, 7535
Flare Poller [candidate] (7.481 ms) : 0, 7481
section appsec
crashtracking [baseline] (1.234 ms) : 0, 1234
crashtracking [candidate] (1.219 ms) : 0, 1219
BytebuddyAgent [baseline] (676.753 ms) : 0, 676753
BytebuddyAgent [candidate] (679.066 ms) : 0, 679066
AgentMeter [baseline] (12.167 ms) : 0, 12167
AgentMeter [candidate] (12.156 ms) : 0, 12156
GlobalTracer [baseline] (249.745 ms) : 0, 249745
GlobalTracer [candidate] (249.153 ms) : 0, 249153
AppSec [baseline] (186.22 ms) : 0, 186220
AppSec [candidate] (185.873 ms) : 0, 185873
Debugger [baseline] (65.897 ms) : 0, 65897
Debugger [candidate] (65.784 ms) : 0, 65784
Remote Config [baseline] (576.84 µs) : 0, 577
Remote Config [candidate] (585.834 µs) : 0, 586
Telemetry [baseline] (7.878 ms) : 0, 7878
Telemetry [candidate] (7.936 ms) : 0, 7936
Flare Poller [baseline] (3.456 ms) : 0, 3456
Flare Poller [candidate] (3.478 ms) : 0, 3478
IAST [baseline] (24.383 ms) : 0, 24383
IAST [candidate] (24.292 ms) : 0, 24292
section iast
crashtracking [baseline] (1.255 ms) : 0, 1255
crashtracking [candidate] (1.219 ms) : 0, 1219
BytebuddyAgent [baseline] (809.568 ms) : 0, 809568
BytebuddyAgent [candidate] (810.631 ms) : 0, 810631
AgentMeter [baseline] (11.416 ms) : 0, 11416
AgentMeter [candidate] (11.497 ms) : 0, 11497
GlobalTracer [baseline] (239.636 ms) : 0, 239636
GlobalTracer [candidate] (240.898 ms) : 0, 240898
AppSec [baseline] (29.655 ms) : 0, 29655
AppSec [candidate] (27.987 ms) : 0, 27987
Debugger [baseline] (66.142 ms) : 0, 66142
Debugger [candidate] (66.724 ms) : 0, 66724
Remote Config [baseline] (601.899 µs) : 0, 602
Remote Config [candidate] (532.254 µs) : 0, 532
Telemetry [baseline] (7.953 ms) : 0, 7953
Telemetry [candidate] (7.885 ms) : 0, 7885
Flare Poller [baseline] (3.476 ms) : 0, 3476
Flare Poller [candidate] (3.411 ms) : 0, 3411
IAST [baseline] (28.349 ms) : 0, 28349
IAST [candidate] (29.307 ms) : 0, 29307
section profiling
crashtracking [baseline] (1.174 ms) : 0, 1174
crashtracking [candidate] (1.184 ms) : 0, 1184
BytebuddyAgent [baseline] (693.455 ms) : 0, 693455
BytebuddyAgent [candidate] (691.118 ms) : 0, 691118
AgentMeter [baseline] (8.98 ms) : 0, 8980
AgentMeter [candidate] (8.976 ms) : 0, 8976
GlobalTracer [baseline] (208.242 ms) : 0, 208242
GlobalTracer [candidate] (207.665 ms) : 0, 207665
AppSec [baseline] (32.765 ms) : 0, 32765
AppSec [candidate] (32.408 ms) : 0, 32408
Debugger [baseline] (65.857 ms) : 0, 65857
Debugger [candidate] (65.649 ms) : 0, 65649
Remote Config [baseline] (581.758 µs) : 0, 582
Remote Config [candidate] (601.868 µs) : 0, 602
Telemetry [baseline] (7.919 ms) : 0, 7919
Telemetry [candidate] (7.92 ms) : 0, 7920
Flare Poller [baseline] (3.489 ms) : 0, 3489
Flare Poller [candidate] (3.486 ms) : 0, 3486
ProfilingAgent [baseline] (94.135 ms) : 0, 94135
ProfilingAgent [candidate] (94.002 ms) : 0, 94002
Profiling [baseline] (94.702 ms) : 0, 94702
Profiling [candidate] (94.582 ms) : 0, 94582
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~9b0e5dc028, baseline=1.62.0-SNAPSHOT~44874e2a03
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1058971
Total [baseline] (8.834 s) : 0, 8833964
Agent [candidate] (1.06 s) : 0, 1059699
Total [candidate] (8.822 s) : 0, 8822088
section iast
Agent [baseline] (1.23 s) : 0, 1229513
Total [baseline] (9.52 s) : 0, 9519832
Agent [candidate] (1.235 s) : 0, 1235448
Total [candidate] (9.553 s) : 0, 9553500
gantt
title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~9b0e5dc028, baseline=1.62.0-SNAPSHOT~44874e2a03
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.234 ms) : 0, 1234
crashtracking [candidate] (1.22 ms) : 0, 1220
BytebuddyAgent [baseline] (634.454 ms) : 0, 634454
BytebuddyAgent [candidate] (634.374 ms) : 0, 634374
AgentMeter [baseline] (29.563 ms) : 0, 29563
AgentMeter [candidate] (29.515 ms) : 0, 29515
GlobalTracer [baseline] (249.643 ms) : 0, 249643
GlobalTracer [candidate] (248.952 ms) : 0, 248952
AppSec [baseline] (32.488 ms) : 0, 32488
AppSec [candidate] (32.392 ms) : 0, 32392
Debugger [baseline] (59.11 ms) : 0, 59110
Debugger [candidate] (58.952 ms) : 0, 58952
Remote Config [baseline] (606.123 µs) : 0, 606
Remote Config [candidate] (589.328 µs) : 0, 589
Telemetry [baseline] (8.039 ms) : 0, 8039
Telemetry [candidate] (8.72 ms) : 0, 8720
Flare Poller [baseline] (7.58 ms) : 0, 7580
Flare Poller [candidate] (8.81 ms) : 0, 8810
section iast
crashtracking [baseline] (1.233 ms) : 0, 1233
crashtracking [candidate] (1.226 ms) : 0, 1226
BytebuddyAgent [baseline] (808.589 ms) : 0, 808589
BytebuddyAgent [candidate] (812.352 ms) : 0, 812352
AgentMeter [baseline] (11.419 ms) : 0, 11419
AgentMeter [candidate] (11.469 ms) : 0, 11469
GlobalTracer [baseline] (238.748 ms) : 0, 238748
GlobalTracer [candidate] (240.016 ms) : 0, 240016
AppSec [baseline] (27.356 ms) : 0, 27356
AppSec [candidate] (30.324 ms) : 0, 30324
Debugger [baseline] (63.423 ms) : 0, 63423
Debugger [candidate] (63.725 ms) : 0, 63725
Remote Config [baseline] (535.949 µs) : 0, 536
Remote Config [candidate] (527.384 µs) : 0, 527
Telemetry [baseline] (7.702 ms) : 0, 7702
Telemetry [candidate] (7.819 ms) : 0, 7819
Flare Poller [baseline] (3.414 ms) : 0, 3414
Flare Poller [candidate] (3.465 ms) : 0, 3465
IAST [baseline] (29.322 ms) : 0, 29322
IAST [candidate] (27.487 ms) : 0, 27487
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 2 performance regressions! Performance is the same for 19 metrics, 15 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~9b0e5dc028, baseline=1.62.0-SNAPSHOT~44874e2a03
dateFormat X
axisFormat %s
section baseline
no_agent (1.262 ms) : 1250, 1274
. : milestone, 1262,
iast (3.273 ms) : 3227, 3320
. : milestone, 3273,
iast_FULL (5.813 ms) : 5754, 5872
. : milestone, 5813,
iast_GLOBAL (3.54 ms) : 3487, 3593
. : milestone, 3540,
profiling (2.447 ms) : 2423, 2471
. : milestone, 2447,
tracing (1.897 ms) : 1882, 1912
. : milestone, 1897,
section candidate
no_agent (1.251 ms) : 1239, 1264
. : milestone, 1251,
iast (3.294 ms) : 3252, 3335
. : milestone, 3294,
iast_FULL (6.189 ms) : 6126, 6253
. : milestone, 6189,
iast_GLOBAL (3.608 ms) : 3554, 3663
. : milestone, 3608,
profiling (2.479 ms) : 2454, 2504
. : milestone, 2479,
tracing (1.908 ms) : 1892, 1924
. : milestone, 1908,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~9b0e5dc028, baseline=1.62.0-SNAPSHOT~44874e2a03
dateFormat X
axisFormat %s
section baseline
no_agent (19.304 ms) : 19105, 19503
. : milestone, 19304,
appsec (18.692 ms) : 18499, 18885
. : milestone, 18692,
code_origins (17.896 ms) : 17721, 18070
. : milestone, 17896,
iast (17.834 ms) : 17659, 18009
. : milestone, 17834,
profiling (18.509 ms) : 18323, 18696
. : milestone, 18509,
tracing (18.076 ms) : 17897, 18255
. : milestone, 18076,
section candidate
no_agent (18.295 ms) : 18109, 18482
. : milestone, 18295,
appsec (18.516 ms) : 18327, 18705
. : milestone, 18516,
code_origins (17.778 ms) : 17605, 17951
. : milestone, 17778,
iast (17.798 ms) : 17624, 17973
. : milestone, 17798,
profiling (18.258 ms) : 18073, 18442
. : milestone, 18258,
tracing (17.938 ms) : 17762, 18113
. : milestone, 17938,
DacapoParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~9b0e5dc028, baseline=1.62.0-SNAPSHOT~44874e2a03
dateFormat X
axisFormat %s
section baseline
no_agent (1.487 ms) : 1475, 1499
. : milestone, 1487,
appsec (3.839 ms) : 3616, 4062
. : milestone, 3839,
iast (2.291 ms) : 2220, 2361
. : milestone, 2291,
iast_GLOBAL (2.324 ms) : 2254, 2395
. : milestone, 2324,
profiling (2.095 ms) : 2040, 2150
. : milestone, 2095,
tracing (2.086 ms) : 2032, 2140
. : milestone, 2086,
section candidate
no_agent (1.486 ms) : 1474, 1497
. : milestone, 1486,
appsec (2.55 ms) : 2495, 2606
. : milestone, 2550,
iast (2.288 ms) : 2218, 2359
. : milestone, 2288,
iast_GLOBAL (2.323 ms) : 2252, 2394
. : milestone, 2323,
profiling (2.109 ms) : 2053, 2164
. : milestone, 2109,
tracing (2.09 ms) : 2036, 2145
. : milestone, 2090,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~9b0e5dc028, baseline=1.62.0-SNAPSHOT~44874e2a03
dateFormat X
axisFormat %s
section baseline
no_agent (15.053 s) : 15053000, 15053000
. : milestone, 15053000,
appsec (15.047 s) : 15047000, 15047000
. : milestone, 15047000,
iast (18.357 s) : 18357000, 18357000
. : milestone, 18357000,
iast_GLOBAL (18.192 s) : 18192000, 18192000
. : milestone, 18192000,
profiling (14.956 s) : 14956000, 14956000
. : milestone, 14956000,
tracing (14.74 s) : 14740000, 14740000
. : milestone, 14740000,
section candidate
no_agent (14.776 s) : 14776000, 14776000
. : milestone, 14776000,
appsec (14.999 s) : 14999000, 14999000
. : milestone, 14999000,
iast (19.099 s) : 19099000, 19099000
. : milestone, 19099000,
iast_GLOBAL (18.113 s) : 18113000, 18113000
. : milestone, 18113000,
profiling (15.142 s) : 15142000, 15142000
. : milestone, 15142000,
tracing (14.898 s) : 14898000, 14898000
. : milestone, 14898000,
|
amarziali
approved these changes
Apr 24, 2026
Contributor
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
gh-worker-dd-mergequeue-cf854d
Bot
merged commit 0d1c44d
into
master
586 of 588 checks passed
gh-worker-dd-mergequeue-cf854d
Bot
deleted the
nccatoni/no-scneario-skip-master
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
Runs all system-tests scenarios, even the empty ones, when running on master or on schedule.
Motivation
This gives feedback on easy wins and makes it easier to monitor that no scenario was dropped by accident.
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]
Note: Once your PR is ready to merge, add it to the merge queue by commenting
/merge./merge -ccancels the queue request./merge -f --reason "reason"skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.