Skip to content

E2E - QA: init, deploy, versions list, config link, 2nd deploy#7367

Merged
phyllis-sy-wu merged 2 commits intomainfrom
psyw-0421-E2E-QA-A-app-dev
Apr 24, 2026
Merged

E2E - QA: init, deploy, versions list, config link, 2nd deploy#7367
phyllis-sy-wu merged 2 commits intomainfrom
psyw-0421-E2E-QA-A-app-dev

Conversation

@phyllis-sy-wu
Copy link
Copy Markdown
Contributor

@phyllis-sy-wu phyllis-sy-wu commented Apr 22, 2026

WHY are these changes introduced?

Automates test A of the manual QA flow (tracked in #7368 / sub-issue #7369). The existing app-deploy.spec.ts only covered a single init→deploy→versions-list cycle. Test A extends it to the full manual flow, adding coverage for:

  • Version-tagged deploy + message persistence
  • app config link creating a second app (interactive PTY prompts)
  • Deploying against the linked secondary config
  • Verifying each app's active version is correctly isolated

This closes the gap for the config link-to-new-app codepath, which previously had no E2E coverage.

WHAT is this pull request doing?

Expands tests/app-deploy.spec.ts to the full A scope:

  1. shopify app init --template reactRouter (primary app)
  2. shopify app deploy --version v1 + assert active on primary
  3. shopify app versions list --json + assert versionTag matches
  4. shopify app config link --config secondary (via PTY) → creates a new app within the primary app's organization
  5. shopify app deploy --config secondary --version v2 + assert active on secondary
  6. shopify app versions list --config secondary --json + assert isolation from primary

Also adds helpers in setup/app.ts (config-link PTY driver, JSON-mode versions list, shared assertion for active-version checks) reused across this and future A–F tests.

How to test your changes?

# One-shot
pnpm --filter e2e exec playwright test app-deploy

# With live CLI output
DEBUG=1 pnpm --filter e2e exec playwright test app-deploy

Requires .env with E2E_ACCOUNT_EMAIL, E2E_ACCOUNT_PASSWORD, E2E_ORG_ID.

Runtime ~2 min. Resources created during the test are the two apps (primary + secondary) — teardown deletes both automatically.

Example

DEBUG=1 pnpm --filter e2e exec playwright test app-deploy
Expand for complete log
cli % DEBUG=1 pnpm --filter e2e exec playwright test app-deploy
[e2e][auth] global setup starting

To run this command, log in to Shopify.
User verification code: LLTN-KJVC
👉 Open this link to start the auth process: https://accounts.shopify.com/activate-with-code?device_code%5Buser_code%5D=LLTN-KJVC
✔ Logged in.
✔ Current account: genghis-khan-identity-1-2025-03-31@shopify.com.
[e2e][auth] browser sessions established for admin + dev dashboard
[e2e][auth] global setup done, config at /Users/psyw/src/github.com/Shopify/cli/.e2e-tmp/global-auth/XDG_CONFIG_HOME

Running 1 test using 1 worker

     1 tests/app-deploy.spec.ts:64:3 › App deploy › init, deploy, versions list, config link, deploy to secondary

[e2e][w0] ----- TEST: init, deploy, versions list, config link, deploy to secondary ----- 
[e2e][auth] copying session from global setup
[e2e][w0][cli] exec: node /Users/psyw/src/github.com/Shopify/cli/packages/create-app/bin/run.js
[e2e][w0][cli] app init --template reactRouter --flavor javascript --name E2E-deploy1-1776876378681 --package-manager pnpm --local --organization-id 161686155 --path /Users/psyw/src/github.com/Shopify/cli/.e2e-tmp/e2e-IM5lTV/app-OvgU1X
╭─ info ───────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  Initializing project with `pnpm`                                            │
│  Use the `--package-manager` flag to select a different package manager.     │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯


╭─ success ────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  e2-e-deploy1-1776876378681 is ready for you to build!                       │
│                                                                              │
│  Next steps                                                                  │
│    • Run `cd e2-e-deploy1-1776876378681`                                     │
│    • For extensions, run `shopify app generate extension`                    │
│    • To see your app, run `shopify app dev`                                  │
│                                                                              │
│  Reference                                                                   │
│    • Shopify docs [1]                                                        │
│    • Shopify Dev MCP, [2] connect your AI assistant to development           │
│      resources                                                               │
│    • For an overview of commands, run `shopify app --help`                   │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
[1] https://shopify.dev
[2] https://shopify.dev/docs/apps/build/devmcp

[e2e][w0][cli] exec: node /Users/psyw/src/github.com/Shopify/cli/packages/cli/bin/run.js
[e2e][w0][cli] app deploy --version E2E-v1-1776876395516 --message E2E A primary deployment --force --path /Users/psyw/src/github.com/Shopify/cli/.e2e-tmp/e2e-IM5lTV/app-OvgU1X/e2-e-deploy1-1776876378681
╭─ warning ────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  The `--force` flag is deprecated and will be removed in the next major      │
│  release.                                                                    │
│                                                                              │
│  Use `--allow-updates` for CI/CD environments, or `--allow-updates           │
│  --allow-deletes` if you also want to allow removals.                        │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯

╭─ info ───────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  Your configuration file has been modified                                   │
│                                                                              │
│  The `include_config_on_deploy` field is no longer supported, since all      │
│  apps must now include configuration on deploy. It has been removed from     │
│  your configuration file.                                                    │
│                                                                              │
│  See Shopify CLI documentation. [1]                                          │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
[1] https://shopify.dev/docs/apps/build/cli-for-apps/app-configuration#build

╭─ info ───────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  Using shopify.app.toml for default values:                                  │
│                                                                              │
│    • Org:             core-build-develop-admin-web-e2e                       │
│    • App:             E2E-deploy1-1776876378681                              │
│                                                                              │
│   You can pass `--reset` to your command to reset your app configuration.    │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯


Releasing a new app version as part of E2E-deploy1-1776876378681



╭─ success ────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  New version released to users.                                              │
│                                                                              │
│  E2E-v1-1776876395516 [1]                                                    │
│  E2E A primary deployment                                                    │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
[1] https://dev.shopify.com/dashboard/161686155/apps/351353143297/versions/93525
3475329

[e2e][w0][cli] exec: node /Users/psyw/src/github.com/Shopify/cli/packages/cli/bin/run.js
[e2e][w0][cli] app versions list --json --path /Users/psyw/src/github.com/Shopify/cli/.e2e-tmp/e2e-IM5lTV/app-OvgU1X/e2-e-deploy1-1776876378681
[
  {
    "createdAt": "2026-04-22 16:46:39",
    "createdBy": "gid://shopify/User/187591014",
    "versionTag": "E2E-v1-1776876395516",
    "status": "active",
    "versionId": "gid://shopify/Version/935253475329",
    "message": "E2E A primary deployment"
  },
  {
    "createdAt": "2026-04-22 16:46:33",
    "createdBy": "gid://shopify/User/187591014",
    "versionTag": "e2e-deploy1-1776876378681-1",
    "status": "inactive",
    "versionId": "gid://shopify/Version/935253278721",
    "message": ""
  }
]
[e2e][w0][cli] spawn: node /Users/psyw/src/github.com/Shopify/cli/packages/cli/bin/run.js
[e2e][w0][cli] app config link --config secondary --path /Users/psyw/src/github.com/Shopify/cli/.e2e-tmp/e2e-IM5lTV/app-OvgU1X/e2-e-deploy1-1776876378681
?  Which organization is this work for?
✔  core-build-develop-admin-web-e2e (161686155)

?  Create this project as a new app on Shopify?
✔  Yes, create it as a new app

?  App name:
✔  E2E-deploy2-1776876378681

╭─ success ────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  shopify.app.secondary.toml is now linked to "E2E-deploy2-1776876378681" on  │
│   Shopify                                                                    │
│                                                                              │
│  Using shopify.app.secondary.toml as your default config.                    │
│                                                                              │
│  Next steps                                                                  │
│    • Make updates to shopify.app.secondary.toml in your local project        │
│    • To upload your config, run `shopify app deploy`                         │
│                                                                              │
│  Reference                                                                   │
│    • App configuration [1]                                                   │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
[1] https://shopify.dev/docs/apps/tools/cli/configuration

[e2e][w0][cli] exec: node /Users/psyw/src/github.com/Shopify/cli/packages/cli/bin/run.js
[e2e][w0][cli] app deploy --version E2E-v2-1776876408185 --message E2E A secondary deployment --config secondary --force --path /Users/psyw/src/github.com/Shopify/cli/.e2e-tmp/e2e-IM5lTV/app-OvgU1X/e2-e-deploy1-1776876378681
╭─ warning ────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  The `--force` flag is deprecated and will be removed in the next major      │
│  release.                                                                    │
│                                                                              │
│  Use `--allow-updates` for CI/CD environments, or `--allow-updates           │
│  --allow-deletes` if you also want to allow removals.                        │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯

╭─ info ───────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  Your configuration file has been modified                                   │
│                                                                              │
│  The `include_config_on_deploy` field is no longer supported, since all      │
│  apps must now include configuration on deploy. It has been removed from     │
│  your configuration file.                                                    │
│                                                                              │
│  See Shopify CLI documentation. [1]                                          │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
[1] https://shopify.dev/docs/apps/build/cli-for-apps/app-configuration#build

╭─ info ───────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  Using shopify.app.secondary.toml for default values:                        │
│                                                                              │
│    • Org:             core-build-develop-admin-web-e2e                       │
│    • App:             E2E-deploy2-1776876378681                              │
│                                                                              │
│   You can pass `--reset` to your command to reset your app configuration.    │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯


Releasing a new app version as part of E2E-deploy2-1776876378681



╭─ success ────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  New version released to users.                                              │
│                                                                              │
│  E2E-v2-1776876408185 [1]                                                    │
│  E2E A secondary deployment                                                  │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯
[1] https://dev.shopify.com/dashboard/161686155/apps/351353241601/versions/93525
3737473

[e2e][w0][cli] exec: node /Users/psyw/src/github.com/Shopify/cli/packages/cli/bin/run.js
[e2e][w0][cli] app versions list --json --config secondary --path /Users/psyw/src/github.com/Shopify/cli/.e2e-tmp/e2e-IM5lTV/app-OvgU1X/e2-e-deploy1-1776876378681
[
  {
    "createdAt": "2026-04-22 16:46:51",
    "createdBy": "gid://shopify/User/187591014",
    "versionTag": "E2E-v2-1776876408185",
    "status": "active",
    "versionId": "gid://shopify/Version/935253737473",
    "message": "E2E A secondary deployment"
  },
  {
    "createdAt": "2026-04-22 16:46:46",
    "createdBy": "gid://shopify/User/187591014",
    "versionTag": "e2e-deploy2-1776876378681-1",
    "status": "inactive",
    "versionId": "gid://shopify/Version/935253573633",
    "message": ""
  }
]

[e2e][w0] ----- Teardown: app E2E-deploy1-1776876378681 ----- 
[e2e][w0][browser] deleting app
[e2e][w0][browser] app found, deleting
[e2e][w0][browser] app deleted

[e2e][w0] ----- Teardown: app E2E-deploy2-1776876378681 ----- 
[e2e][w0][browser] deleting app
[e2e][w0][browser] app found, deleting
[e2e][w0][browser] app deleted
  ✓  1 tests/app-deploy.spec.ts:64:3 › App deploy › init, deploy, versions list, config link, deploy to secondary (1.5m)

  1 passed (2.2m)

Post-release steps

N/A

Checklist

  • I've considered possible cross-platform impacts (Mac, Linux, Windows) — E2E tests run on macOS/Linux in CI
  • I've considered possible documentation changes
  • I've considered analytics changes to measure impact
  • The change is user-facing — no user-facing changes (test infrastructure only), no changeset needed

Copy link
Copy Markdown
Contributor Author

phyllis-sy-wu commented Apr 22, 2026

@phyllis-sy-wu phyllis-sy-wu mentioned this pull request Apr 22, 2026
4 tasks
@github-actions
Copy link
Copy Markdown
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/private/node/analytics.d.ts
@@ -19,7 +19,6 @@ interface EnvironmentData {
     env_device_id: string;
     env_cloud: string;
     env_package_manager: string;
-    env_install_package_manager: string;
     env_is_global: boolean;
     env_auth_method: string;
     env_is_wsl: boolean;
packages/cli-kit/dist/public/node/monorail.d.ts
@@ -2,7 +2,7 @@ import { JsonMap } from '../../private/common/json.js';
 import { DeepRequired } from '../common/ts/deep-required.js';
 export { DeepRequired };
 type Optional<T> = T | null;
-export declare const MONORAIL_COMMAND_TOPIC = "app_cli3_command/1.22";
+export declare const MONORAIL_COMMAND_TOPIC = "app_cli3_command/1.21";
 export interface Schemas {
     [MONORAIL_COMMAND_TOPIC]: {
         sensitive: {
@@ -129,7 +129,6 @@ export interface Schemas {
             env_ci_platform?: Optional<string>;
             env_device_id?: Optional<string>;
             env_package_manager?: Optional<string>;
-            env_install_package_manager?: Optional<string>;
             env_package_manager_workspaces?: Optional<boolean>;
             env_plugin_installed_any_custom?: Optional<boolean>;
             env_plugin_installed_shopify?: Optional<string>;

@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-improve-cli-log-format branch from 9d0a3aa to bfb1932 Compare April 22, 2026 14:52
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from 26a7e70 to 95f628a Compare April 22, 2026 14:52
@phyllis-sy-wu phyllis-sy-wu changed the base branch from psyw-0421-E2E-improve-cli-log-format to graphite-base/7367 April 22, 2026 15:15
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from 95f628a to f0005dd Compare April 22, 2026 15:15
@phyllis-sy-wu phyllis-sy-wu changed the base branch from graphite-base/7367 to psyw-0421-E2E-teardown-polish April 22, 2026 15:15
@phyllis-sy-wu phyllis-sy-wu linked an issue Apr 22, 2026 that may be closed by this pull request
@phyllis-sy-wu phyllis-sy-wu changed the base branch from psyw-0421-E2E-teardown-polish to graphite-base/7367 April 22, 2026 17:17
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from f0005dd to 633cc3c Compare April 22, 2026 17:17
@phyllis-sy-wu phyllis-sy-wu changed the base branch from graphite-base/7367 to psyw-0421-E2E-improve-cli-log-format April 22, 2026 17:17
@phyllis-sy-wu phyllis-sy-wu changed the base branch from psyw-0421-E2E-improve-cli-log-format to graphite-base/7367 April 22, 2026 18:41
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from 633cc3c to 8e559ab Compare April 22, 2026 18:41
@phyllis-sy-wu phyllis-sy-wu changed the base branch from graphite-base/7367 to psyw-0421-E2E-teardown-polish April 22, 2026 18:41
@phyllis-sy-wu phyllis-sy-wu changed the base branch from psyw-0421-E2E-teardown-polish to graphite-base/7367 April 22, 2026 19:31
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from 8e559ab to 64d0528 Compare April 22, 2026 20:46
@phyllis-sy-wu phyllis-sy-wu changed the base branch from graphite-base/7367 to psyw-0421-E2E-teardown-polish April 22, 2026 20:46
@phyllis-sy-wu phyllis-sy-wu changed the base branch from psyw-0421-E2E-teardown-polish to graphite-base/7367 April 22, 2026 21:03
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from 64d0528 to e952944 Compare April 22, 2026 21:14
@phyllis-sy-wu phyllis-sy-wu changed the base branch from graphite-base/7367 to psyw-0421-E2E-teardown-polish April 22, 2026 21:14
@phyllis-sy-wu phyllis-sy-wu marked this pull request as ready for review April 22, 2026 21:23
@phyllis-sy-wu phyllis-sy-wu requested a review from a team as a code owner April 22, 2026 21:23
@phyllis-sy-wu phyllis-sy-wu changed the base branch from psyw-0421-E2E-teardown-polish to graphite-base/7367 April 23, 2026 08:50
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from e952944 to 0cff016 Compare April 23, 2026 08:50
@phyllis-sy-wu phyllis-sy-wu changed the base branch from graphite-base/7367 to psyw-0420-E2E-utility-cleanup-all April 23, 2026 08:50
@phyllis-sy-wu phyllis-sy-wu changed the base branch from psyw-0420-E2E-utility-cleanup-all to graphite-base/7367 April 23, 2026 11:07
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from 0cff016 to 7bc783a Compare April 23, 2026 11:18
@phyllis-sy-wu phyllis-sy-wu changed the base branch from graphite-base/7367 to psyw-0420-E2E-utility-cleanup-all April 23, 2026 11:18
Copy link
Copy Markdown
Contributor

@byrichardpowell byrichardpowell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving with a few suggestions:

1) File-scope eslint-disable no-restricted-imports is too broad. The prior // eslint-disable-line was surgical — just the path import. The new file-level disable silences the rule for every current
and future import in the file. Prefer restoring the per-line form:

import * as path from 'path' // eslint-disable-line no-restricted-imports
  1. Tighten the Step 4 link assertion. expect(linkOutput).toContain('is now linked to') passes even if link succeeded against the wrong app. The CLI banner includes the app name — assert it:
expect(linkOutput).toContain(`is now linked to "${secondaryAppName}"`)

@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from 7bc783a to b4842d8 Compare April 24, 2026 02:06
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0420-E2E-utility-cleanup-all branch from 3e397ce to 8f355b6 Compare April 24, 2026 02:06
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0420-E2E-utility-cleanup-all branch from 8f355b6 to 47ed438 Compare April 24, 2026 02:41
@phyllis-sy-wu phyllis-sy-wu force-pushed the psyw-0421-E2E-QA-A-app-dev branch from b4842d8 to 36be687 Compare April 24, 2026 02:41
Copy link
Copy Markdown
Contributor Author

Fixed on latest update! Thanks for spotting these issues.

Base automatically changed from psyw-0420-E2E-utility-cleanup-all to main April 24, 2026 15:12
@phyllis-sy-wu phyllis-sy-wu added this pull request to the merge queue Apr 24, 2026
Merged via the queue into main with commit 1a46d87 Apr 24, 2026
25 checks passed
@phyllis-sy-wu phyllis-sy-wu deleted the psyw-0421-E2E-QA-A-app-dev branch April 24, 2026 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

E2E-QA: app-deploy

2 participants