You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kie.apache.org by "ljmotta (via GitHub)" <gi...@apache.org> on 2025/12/19 12:44:57 UTC
[PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
ljmotta opened a new pull request, #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382
This PR adds the option to change the Playwright configuration through environment variables.
```
# Enable running tests on Apple Silicon Google Chrome. Overrides PLAYWRIGHT_BASE__enableGoogleChromeProject.
PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon=true
# Enable/disable Chromium tests
PLAYWRIGHT_BASE__enableChromiumProject=true
# Enable/disable Google Chrome tests
PLAYWRIGHT_BASE__enableGoogleChromeProject=true
# Enable/disable Webkit tests
PLAYWRIGHT_BASE__enableWebkitProject=true
# Set project timeout
PLAYWRIGHT_BASE__projectTimeout=90000
# Set "expect" statement timeout
PLAYWRIGHT_BASE__expectTimeout=30000
# Set the maxDiffPixelRatio for the test assertion.
PLAYWRIGHT_BASE__maxDiffPixelRatio=0.001
# Set the number of retries for each test.
PLAYWRIGHT_BASE__retries=2
# Set the number of workers used when executing the test suite. Increasing this number can increase the tests flakyness.
PLAYWRIGHT_BASE__workers=6
```
This PR also:
- Creates a new CLI (`pw-e2e-container`) to handle Playwright specific containerization configurations instead of bloating the `package.json` with more commands.
- Remove 0.1 maxDiffPixel on `boxed-expression-component` and `scesim-editor`.
- Uses `start-server-and-test` on all packages.
- Improvements on `online-editor` test logs.
Now, it is possible to enable/disable suite for a specific project without using the `shell`. Example for run only webkit tests with more workers:
```sh
KIE_TOOLS_BUILD__runEndToEndTests=true PLAYWRIGHT_BASE__enableWebkitProject=true PLAYWRIGHT_BASE__enableGoogleChromeProject=false PLAYWRIGHT_BASE__enableChromiumProject=false PLAYWRIGHT_BASE__workers=6 pnpm test-e2e
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "ljmotta (via GitHub)" <gi...@apache.org>.
ljmotta commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635166407
##########
packages/playwright-base/env/index.js:
##########
@@ -25,11 +25,58 @@ module.exports = composeEnv([require("@kie-tools/root-env/env")], {
default: "false",
description: "Toggles the installation of Playwright dependencies. Can be `true` or `false`.",
},
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon: {
+ default: "true",
+ description: "Enable Google Chrome tests for ARM OSs. Overrides PLAYWRIGHT_BASE__enableGoogleChromeProject.",
+ },
Review Comment:
I believe we could create a new issue to re-asses the need for skipping tests on Google Chrome for apple arm64.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "ljmotta (via GitHub)" <gi...@apache.org>.
ljmotta commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635294387
##########
packages/playwright-base/README.md:
##########
@@ -19,35 +19,15 @@
## Overview
-This package collects common configurations to run end-to-end Playwright tests.
-
-## Installing Playwright deps
-
-Currently, all Playwright end-to-end tests run inside containers. If you need to debug a test, we recommend doing so on the host machine. To do this, you need to install the Playwright dependencies. Use the `PLAYWRIGHT_BASE__installDeps` environment variable during the bootstrap phase to install all the required dependencies.
-
-```sh
-# in the `kie-tools` root
-PLAYWRIGHT_BASE__installDeps=true pnpm bootstrap
-```
-
-or
-
-```sh
-# in the `kie-tools` root
-PLAYWRIGHT_BASE__installDeps=true pnpm bootstrap -F playwright-base
-```
-
-> **i NOTE**
->
-> Since this step install the Playwright browsers, it requires sudo permision.
+This package collects common configurations to run end-to-end Playwright tests. Currently, all Playwright end-to-end tests run inside containers. If you need to debug a test, we recommend doing so on the host machine, refer to the [Installing Playwright deps](#Installing-Playwright-deps)
## Using containers to generate screenshots
Each operating system has slight variations in UI, even within the same browser. These differences can cause screenshot comparison tests to fail. To address this issue and ensure a stable environment with consistent test results locally and in CI, containers can be used. Running Playwright tests inside a container that is also used in the CI environment makes screenshot tests reproducible, regardless of the host OS.
> **ℹ️ NOTE**
>
-> Due to compatibility issues, this containerization solution cannot yet be used on native Windows and requires running it directly within WSL (Windows Subsystem for Linux). Also, Linux arm64 doesn't support Google Chrome, and due to this caveat, some tests will be disabled for this arch.
+> Due to compatibility issues, this containerization solution cannot yet be used on native Windows and requires running it directly within WSL (Windows Subsystem for Linux). Also, Linux arm64 doesn't support Google Chrome, and due to this caveat, Google Chrome tests in macOS arm are disabled in some packages.
Review Comment:
Actually, I'll update to something like WDYT @thiagoelg :
```
Due to compatibility issues, this containerization solution cannot yet be used on native Windows and requires running it directly within WSL (Windows Subsystem for Linux). Additionally, Google Chrome is not available for Linux on ARM64, so any Chrome-based tests in an ARM64 container must be disabled. At present, tests run on AMD64 containers to ensure all browsers and test scenarios execute safely.
```
We don't have any tests running on arm64, some tests are disabled on Apple Silicon due to historical reasons.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "thiagoelg (via GitHub)" <gi...@apache.org>.
thiagoelg commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635061887
##########
packages/playwright-base/README.md:
##########
@@ -79,6 +59,76 @@ pnpm test-e2e:container:shell
pnpm test-e2e:run --update-snapshots
```
+## Playwright-base CLI
+
+A CLI tool to assist running containerized Playwright tests. It loads environment variables and passes them through the docker-compose file, and helps with starting the test suite, opening an interactive shell inside the container, and cleaning up.
+
+```sh
+pnpm pw-e2e-container --help
+pw-e2e-container <command>
+
+Commands:
+ pw-e2e-container run Run the Playwright test suite inside Docker containers. This command will start the required containers using docker-compose
+ and execute the Playwright tests in the specified container workdir.
+ pw-e2e-container shell Open an interactive shell inside the Playwright test container. This command starts the required container using
+ docker-compose and launches a shell in the specified workdir inside the container.
+ pw-e2e-container clean Stop and remove all Playwright-related containers created by docker-compose. This command runs 'docker compose down' using
+ the base Playwright compose file.
+
+Examples:
+ containerized-playwright run --container-name my_playwright_container Run the Playwright test suite locally (no CI override) using the
+ --container-workdir incubator-kie-tools/packages/my-package specified container and workdir.
+
+ containerized-playwright run --ci --container-name my_playwright_container Run the Playwright test suite in CI mode (applies CI docker-compose
+ --container-workdir incubator-kie-tools/packages/my-package override).
+
+ containerized-playwright run --container-name e2e --container-workdir Run with extra environment variables forwarded to docker-compose
+ incubator-kie-tools/packages/foo --additional-env (comma-separated KEY=VALUE pairs).
+ BUILD_ID=123,REPORT_DIR=/tmp/reports
+
+ containerized-playwright run --container-name e2e --container-workdir Pass multiple --additional-env options; later pairs override earlier
+ incubator-kie-tools/packages/foo --additional-env FOO=bar --additional-env keys if duplicated.
+ COMMIT_SHA=deadbeef
+
+ CI=true containerized-playwright run --container-name e2e Leverage CI environment variable to auto-enable CI mode (equivalent to
+ --container-workdir incubator-kie-tools/packages/foo --ci when CI=true or CI=1).
+
+ containerized-playwright shell --container-name my_playwright_container Start the container (if needed) and open an interactive bash shell in
+ --container-workdir incubator-kie-tools/packages/my-package the package workdir.
+
+ containerized-playwright shell --container-name e2e --container-workdir Open an interactive shell with extra environment variables forwarded to
+ incubator-kie-tools/packages/foo --additional-env DEBUG=true docker-compose.
+
+ containerized-playwright shell --container-name e2e --container-workdir Forward multiple env variables in a single --additional-env option
+ incubator-kie-tools/packages/foo --additional-env FOO=bar,BAZ=qux using comma-separated pairs.
+
+ containerized-playwright clean Stop and remove Playwright-related containers using the base
+ docker-compose file.
+```
+
+## Installing Playwright dependencies on host machine
+
+> **i Warning**
+>
+> The tests screenshot comparisons will not work in an environment different than the containers, meaning, this setup is only used to debug.
+> To install the Playwright dependencies use the `PLAYWRIGHT_BASE__installDeps` environment variable during the bootstrap phase to install all the required dependencies.
+
+```sh
+# in the `kie-tools` root
+PLAYWRIGHT_BASE__installDeps=true pnpm bootstrap
+```
+
+or
+
+```sh
+# in the `kie-tools` root
+PLAYWRIGHT_BASE__installDeps=true pnpm bootstrap -F playwright-base
+```
+
+> **i NOTE**
Review Comment:
```suggestion
> **ℹ️ NOTE**
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "thiagoelg (via GitHub)" <gi...@apache.org>.
thiagoelg commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635059694
##########
packages/playwright-base/README.md:
##########
@@ -79,6 +59,76 @@ pnpm test-e2e:container:shell
pnpm test-e2e:run --update-snapshots
```
+## Playwright-base CLI
+
+A CLI tool to assist running containerized Playwright tests. It loads environment variables and passes them through the docker-compose file, and helps with starting the test suite, opening an interactive shell inside the container, and cleaning up.
+
+```sh
+pnpm pw-e2e-container --help
+pw-e2e-container <command>
+
+Commands:
Review Comment:
It would be nice to also have a list of possible options/flags here (check the image-builder README).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "ljmotta (via GitHub)" <gi...@apache.org>.
ljmotta commented on PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#issuecomment-3675846961
@thiagoelg Thanks for the review. I've addressed your comments. Now, regarding the apple silicon specific environment variable, I checked it locally and the tests are do passing on a darwin arm64 machine, so I have removed all code related to this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "thiagoelg (via GitHub)" <gi...@apache.org>.
thiagoelg commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635083064
##########
packages/playwright-base/package.json:
##########
@@ -12,20 +12,33 @@
"bugs": {
"url": "https://github.com/apache/incubator-kie-tools/issues"
},
- "types": "index.d.ts",
- "main": "index.js",
+ "bin": {
+ "pw-e2e-container": "bin.js"
+ },
+ "main": "dist/bin.js",
"files": [
- "index.js"
+ "bin.js",
+ "dist",
+ "src"
],
"scripts": {
"build:dev": "run-script-if --bool \"$(build-env containerImages.build)\" --then \"pnpm image:docker:build\"",
"build:prod": "pnpm build:dev",
"image:docker:build": "docker build --platform=linux/amd64 -t dev.local/apache-kie-tools/playwright-e2e-tests-image:$(build-env root.streamName) . -f Containerfile --output type=docker",
- "install": "run-script-if --bool \"$(build-env playwrightBase.installDeps)\" --then \"pnpm playwright install --with-deps\""
+ "install": "pnpm install:cli && pnpm install:image",
+ "install:cli": "rimraf dist && tsc",
+ "install:image": "run-script-if --bool \"$(build-env playwrightBase.installDeps)\" --then \"pnpm playwright install --with-deps\""
Review Comment:
Why `install:image`? This is installing playwright dependencies only.
##########
packages/playwright-base/env/index.js:
##########
@@ -25,11 +25,58 @@ module.exports = composeEnv([require("@kie-tools/root-env/env")], {
default: "false",
description: "Toggles the installation of Playwright dependencies. Can be `true` or `false`.",
},
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon: {
+ default: "true",
+ description: "Enable Google Chrome tests for ARM OSs. Overrides PLAYWRIGHT_BASE__enableGoogleChromeProject.",
+ },
Review Comment:
The issue with Google Chrome on ARM also occurs on Linux ARM64 machines, so I think this env var should be more generic in this sense.
##########
packages/playwright-base/env/index.js:
##########
@@ -25,11 +25,58 @@ module.exports = composeEnv([require("@kie-tools/root-env/env")], {
default: "false",
description: "Toggles the installation of Playwright dependencies. Can be `true` or `false`.",
},
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon: {
+ default: "true",
+ description: "Enable Google Chrome tests for ARM OSs. Overrides PLAYWRIGHT_BASE__enableGoogleChromeProject.",
+ },
Review Comment:
```suggestion
PLAYWRIGHT_BASE__enableGoogleChromeTestsOnARM: {
default: "true",
description: "Enable or disable Google Chrome tests on ARM OSes. Overrides PLAYWRIGHT_BASE__enableGoogleChromeProject.",
},
```
##########
packages/playwright-base/src/bin.ts:
##########
@@ -0,0 +1,319 @@
+#!/usr/bin/env node
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import { execSync, spawnSync } from "child_process";
+import yargs from "yargs/yargs";
+import { terminalWidth, exit } from "yargs";
+import { hideBin } from "yargs/helpers";
+
+// eslint-disable-next-line @typescript-eslint/ban-ts-comment
+// @ts-ignore
+import { env } from "../env";
+
+const CLI_NAME = "pw-e2e-container";
+
+const buildEnv: any = env;
+
+function basePlaywrightEnv(): Record<string, string> {
+ return {
+ STREAM_NAME: buildEnv.root.streamName,
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon:
+ buildEnv.playwrightBase.enableGoogleChromeTestsForAppleSilicon,
+ PLAYWRIGHT_BASE__enableChromiumProject: buildEnv.playwrightBase.enableChromiumProject,
+ PLAYWRIGHT_BASE__enableGoogleChromeProject: buildEnv.playwrightBase.enableGoogleChromeProject,
+ PLAYWRIGHT_BASE__enableWebkitProject: buildEnv.playwrightBase.enableWebkitProject,
+ PLAYWRIGHT_BASE__projectTimeout: buildEnv.playwrightBase.projectTimeout,
+ PLAYWRIGHT_BASE__expectTimeout: buildEnv.playwrightBase.expectTimeout,
+ PLAYWRIGHT_BASE__maxDiffPixelRatio: buildEnv.playwrightBase.maxDiffPixelRatio,
+ PLAYWRIGHT_BASE__retries: buildEnv.playwrightBase.retries,
+ PLAYWRIGHT_BASE__workers: buildEnv.playwrightBase.workers,
+ };
+}
Review Comment:
If you run `pnpm build-env --print-dotenv` or `--print-dotenv:self` you'll get basically this, but this is fine.
##########
packages/playwright-base/src/bin.ts:
##########
@@ -0,0 +1,319 @@
+#!/usr/bin/env node
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import { execSync, spawnSync } from "child_process";
+import yargs from "yargs/yargs";
+import { terminalWidth, exit } from "yargs";
+import { hideBin } from "yargs/helpers";
+
+// eslint-disable-next-line @typescript-eslint/ban-ts-comment
+// @ts-ignore
+import { env } from "../env";
+
+const CLI_NAME = "pw-e2e-container";
+
+const buildEnv: any = env;
+
+function basePlaywrightEnv(): Record<string, string> {
+ return {
+ STREAM_NAME: buildEnv.root.streamName,
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon:
+ buildEnv.playwrightBase.enableGoogleChromeTestsForAppleSilicon,
+ PLAYWRIGHT_BASE__enableChromiumProject: buildEnv.playwrightBase.enableChromiumProject,
+ PLAYWRIGHT_BASE__enableGoogleChromeProject: buildEnv.playwrightBase.enableGoogleChromeProject,
+ PLAYWRIGHT_BASE__enableWebkitProject: buildEnv.playwrightBase.enableWebkitProject,
+ PLAYWRIGHT_BASE__projectTimeout: buildEnv.playwrightBase.projectTimeout,
+ PLAYWRIGHT_BASE__expectTimeout: buildEnv.playwrightBase.expectTimeout,
+ PLAYWRIGHT_BASE__maxDiffPixelRatio: buildEnv.playwrightBase.maxDiffPixelRatio,
+ PLAYWRIGHT_BASE__retries: buildEnv.playwrightBase.retries,
+ PLAYWRIGHT_BASE__workers: buildEnv.playwrightBase.workers,
+ };
+}
+
+type EnvMap = Record<string, string>;
+
+function parseCommaEnvList(input: string): EnvMap {
+ const out: EnvMap = {};
+ const s = (input ?? "").trim();
+ if (s.length === 0) return out;
+
+ for (const raw of s.split(",")) {
+ const pair = raw.trim();
+ if (!pair) continue;
+
+ const eq = pair.indexOf("=");
+ if (eq <= 0) {
+ throw new Error(`Invalid env pair "${pair}". Expected KEY=VALUE.`);
+ }
+
+ const key = pair.slice(0, eq).trim();
+ const value = pair.slice(eq + 1).trim();
+
+ if (!/^[A-Za-z_][A-Za-z0-9_]*$/.test(key)) {
+ throw new Error(`Invalid env key "${key}". Must match [A-Za-z_][A-Za-z0-9_]*.`);
+ }
+
+ out[key] = value;
+ }
+ return out;
+}
+
+function collectAdditionalEnv(argv: any): EnvMap {
+ const entries = ([] as string[]).concat(argv.additionalEnv ?? argv["additional-env"] ?? []);
+ const merged: EnvMap = {};
+ for (const entry of entries) {
+ const parsed = parseCommaEnvList(entry);
+ Object.assign(merged, parsed);
+ }
+ return merged;
+}
+
+function prettyPrintError(error: unknown) {
+ console.error(`\x1b[31m[${CLI_NAME}] %s\x1b[0m`, error instanceof Error ? error.message : String(error));
+}
+
+function isAppleSilicon(): boolean {
+ return process.platform === "darwin" && process.arch === "arm64";
+}
Review Comment:
Following up on the comments above, using `process.arch === "arm64"` should be enough, since the Google Chrome issue happens inside Linux ARM64 containers, no matter the host OS.
##########
packages/playwright-base/src/bin.ts:
##########
@@ -0,0 +1,319 @@
+#!/usr/bin/env node
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import { execSync, spawnSync } from "child_process";
+import yargs from "yargs/yargs";
+import { terminalWidth, exit } from "yargs";
+import { hideBin } from "yargs/helpers";
+
+// eslint-disable-next-line @typescript-eslint/ban-ts-comment
+// @ts-ignore
+import { env } from "../env";
+
+const CLI_NAME = "pw-e2e-container";
+
+const buildEnv: any = env;
+
+function basePlaywrightEnv(): Record<string, string> {
+ return {
+ STREAM_NAME: buildEnv.root.streamName,
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon:
+ buildEnv.playwrightBase.enableGoogleChromeTestsForAppleSilicon,
+ PLAYWRIGHT_BASE__enableChromiumProject: buildEnv.playwrightBase.enableChromiumProject,
+ PLAYWRIGHT_BASE__enableGoogleChromeProject: buildEnv.playwrightBase.enableGoogleChromeProject,
+ PLAYWRIGHT_BASE__enableWebkitProject: buildEnv.playwrightBase.enableWebkitProject,
+ PLAYWRIGHT_BASE__projectTimeout: buildEnv.playwrightBase.projectTimeout,
+ PLAYWRIGHT_BASE__expectTimeout: buildEnv.playwrightBase.expectTimeout,
+ PLAYWRIGHT_BASE__maxDiffPixelRatio: buildEnv.playwrightBase.maxDiffPixelRatio,
+ PLAYWRIGHT_BASE__retries: buildEnv.playwrightBase.retries,
+ PLAYWRIGHT_BASE__workers: buildEnv.playwrightBase.workers,
+ };
+}
+
+type EnvMap = Record<string, string>;
+
+function parseCommaEnvList(input: string): EnvMap {
+ const out: EnvMap = {};
+ const s = (input ?? "").trim();
+ if (s.length === 0) return out;
+
+ for (const raw of s.split(",")) {
+ const pair = raw.trim();
+ if (!pair) continue;
+
+ const eq = pair.indexOf("=");
+ if (eq <= 0) {
+ throw new Error(`Invalid env pair "${pair}". Expected KEY=VALUE.`);
+ }
+
+ const key = pair.slice(0, eq).trim();
+ const value = pair.slice(eq + 1).trim();
+
+ if (!/^[A-Za-z_][A-Za-z0-9_]*$/.test(key)) {
+ throw new Error(`Invalid env key "${key}". Must match [A-Za-z_][A-Za-z0-9_]*.`);
+ }
+
+ out[key] = value;
+ }
+ return out;
+}
+
+function collectAdditionalEnv(argv: any): EnvMap {
+ const entries = ([] as string[]).concat(argv.additionalEnv ?? argv["additional-env"] ?? []);
+ const merged: EnvMap = {};
+ for (const entry of entries) {
+ const parsed = parseCommaEnvList(entry);
+ Object.assign(merged, parsed);
+ }
+ return merged;
+}
+
+function prettyPrintError(error: unknown) {
+ console.error(`\x1b[31m[${CLI_NAME}] %s\x1b[0m`, error instanceof Error ? error.message : String(error));
+}
+
+function isAppleSilicon(): boolean {
+ return process.platform === "darwin" && process.arch === "arm64";
+}
+
+function mergeEnv(overrides: EnvMap = {}): NodeJS.ProcessEnv {
+ return { ...process.env, ...basePlaywrightEnv(), ...overrides };
+}
+
+function dockerComposeUp(ci: boolean, overrides: Record<string, string> = {}) {
+ const composeBase = "./tests-e2e/__containerization__/playwright-docker-compose.yml";
+ const composeCi = "./tests-e2e/__containerization__/playwright-docker-compose.ci.yml";
+ const files = ci ? [`-f`, composeBase, `-f`, composeCi] : [`-f`, composeBase];
+
+ execSync(`docker compose ${files.join(" ")} up -d`, { stdio: "inherit", env: mergeEnv(overrides) });
+}
+
+async function main() {
+ try {
+ await yargs(hideBin(process.argv))
+ .version(false)
+ .wrap(Math.min(150, terminalWidth()))
+ .scriptName(CLI_NAME)
+ .epilog(
+ `A CLI tool to assist running containerized Playwright tests.
+It loads environment variables and passes them through the docker-compose file, and helps with starting the test suite, opening an interactive shell inside the container, and cleaning up.`
+ )
+ .example(
+ "containerized-playwright run --container-name my_playwright_container --container-workdir incubator-kie-tools/packages/my-package\n",
+ "Run the Playwright test suite locally (no CI override) using the specified container and workdir.\n"
+ )
+ .example(
+ "containerized-playwright run --ci --container-name my_playwright_container --container-workdir incubator-kie-tools/packages/my-package\n",
+ "Run the Playwright test suite in CI mode (applies CI docker-compose override).\n"
+ )
+ .example(
+ "containerized-playwright run --container-name e2e --container-workdir incubator-kie-tools/packages/foo --additional-env BUILD_ID=123,REPORT_DIR=/tmp/reports\n",
+ "Run with extra environment variables forwarded to docker-compose (comma-separated KEY=VALUE pairs).\n"
+ )
+ .example(
+ "containerized-playwright run --container-name e2e --container-workdir incubator-kie-tools/packages/foo --additional-env FOO=bar --additional-env COMMIT_SHA=deadbeef\n",
+ "Pass multiple --additional-env options; later pairs override earlier keys if duplicated.\n"
+ )
+ .example(
+ "CI=true containerized-playwright run --container-name e2e --container-workdir incubator-kie-tools/packages/foo\n",
+ "Leverage CI environment variable to auto-enable CI mode (equivalent to --ci when CI=true or CI=1).\n"
+ )
+ .example(
+ "containerized-playwright shell --container-name my_playwright_container --container-workdir incubator-kie-tools/packages/my-package\n",
+ "Start the container (if needed) and open an interactive bash shell in the package workdir.\n"
+ )
+ .example(
+ "containerized-playwright shell --container-name e2e --container-workdir incubator-kie-tools/packages/foo --additional-env DEBUG=true\n",
+ "Open an interactive shell with extra environment variables forwarded to docker-compose.\n"
+ )
+ .example(
+ "containerized-playwright shell --container-name e2e --container-workdir incubator-kie-tools/packages/foo --additional-env FOO=bar,BAZ=qux\n",
+ "Forward multiple env variables in a single --additional-env option using comma-separated pairs.\n"
+ )
+ .example(
+ "containerized-playwright clean\n",
+ "Stop and remove Playwright-related containers using the base docker-compose file.\n"
+ )
+ .command(
+ "run",
+ "Run the Playwright test suite inside Docker containers. This command will start the required containers using docker-compose and execute the Playwright tests in the specified container workdir.",
+ (yargs) => {
+ yargs
+ .option("ci", {
+ type: "boolean",
+ default: process.env.CI === "true" || process.env.CI === "1",
+ describe: "Enable CI mode by applying the CI-specific docker-compose override file.",
+ })
+ .option("additional-env", {
+ type: "string",
+ default: "",
+ describe:
+ "Comma-separated KEY=VALUE pairs of additional environment variables to forward to docker-compose. Can be repeated.",
+ })
+ .option("container-name", {
+ type: "string",
+ default: "",
+ describe: "Name of the container as defined in the docker-compose file. Required.",
+ demandOption: true,
+ })
+ .option("container-workdir", {
+ type: "string",
+ default: "",
+ describe:
+ "Path inside the container where Playwright tests are located. Example: incubator-kie-tools/packages/<package_name>. Required.",
+ demandOption: true,
+ });
+ },
+ (argv) => {
+ const extraEnv = collectAdditionalEnv(argv);
+ // If Google Chrome is enabled check if tests are enabled for Apple Silicon
+ const isGoogleChromeEnabled =
+ buildEnv.playwrightBase.enableGoogleChromeProject === true
+ ? isAppleSilicon() && buildEnv.playwrightBase.enableGoogleChromeTestsForAppleSilicon === true
+ ? "true"
+ : !isAppleSilicon()
+ ? "true"
+ : "false"
+ : "false";
+
+ dockerComposeUp(!!argv.ci, {
+ PLAYWRIGHT_BASE__enableGoogleChromeProject: isGoogleChromeEnabled,
+ ...extraEnv,
+ });
+ console.info(
+ `[${CLI_NAME}] docker compose up done. Env
+CI=${!!argv.ci}
+PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon=${buildEnv.playwrightBase.enableGoogleChromeTestsForAppleSilicon}
+PLAYWRIGHT_BASE__enableChromiumProject=${buildEnv.playwrightBase.enableChromiumProject}
+PLAYWRIGHT_BASE__enableGoogleChromeProject=${isGoogleChromeEnabled}
+PLAYWRIGHT_BASE__enableWebkitProject=${buildEnv.playwrightBase.enableWebkitProject}
+PLAYWRIGHT_BASE__projectTimeout=${buildEnv.playwrightBase.projectTimeout}
+PLAYWRIGHT_BASE__expectTimeout=${buildEnv.playwrightBase.expectTimeout}
+PLAYWRIGHT_BASE__maxDiffPixelRatio=${buildEnv.playwrightBase.maxDiffPixelRatio}
+PLAYWRIGHT_BASE__retries=${buildEnv.playwrightBase.retries}
+PLAYWRIGHT_BASE__workers=${buildEnv.playwrightBase.workers}
+extraEnv=${JSON.stringify(extraEnv)}
Review Comment:
`pnpm build-env --print-dotenv` would also help here.
##########
packages/playwright-base/src/bin.ts:
##########
@@ -0,0 +1,319 @@
+#!/usr/bin/env node
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import { execSync, spawnSync } from "child_process";
+import yargs from "yargs/yargs";
+import { terminalWidth, exit } from "yargs";
+import { hideBin } from "yargs/helpers";
+
+// eslint-disable-next-line @typescript-eslint/ban-ts-comment
+// @ts-ignore
+import { env } from "../env";
+
+const CLI_NAME = "pw-e2e-container";
+
+const buildEnv: any = env;
+
+function basePlaywrightEnv(): Record<string, string> {
+ return {
+ STREAM_NAME: buildEnv.root.streamName,
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon:
+ buildEnv.playwrightBase.enableGoogleChromeTestsForAppleSilicon,
+ PLAYWRIGHT_BASE__enableChromiumProject: buildEnv.playwrightBase.enableChromiumProject,
+ PLAYWRIGHT_BASE__enableGoogleChromeProject: buildEnv.playwrightBase.enableGoogleChromeProject,
+ PLAYWRIGHT_BASE__enableWebkitProject: buildEnv.playwrightBase.enableWebkitProject,
+ PLAYWRIGHT_BASE__projectTimeout: buildEnv.playwrightBase.projectTimeout,
+ PLAYWRIGHT_BASE__expectTimeout: buildEnv.playwrightBase.expectTimeout,
+ PLAYWRIGHT_BASE__maxDiffPixelRatio: buildEnv.playwrightBase.maxDiffPixelRatio,
+ PLAYWRIGHT_BASE__retries: buildEnv.playwrightBase.retries,
+ PLAYWRIGHT_BASE__workers: buildEnv.playwrightBase.workers,
+ };
+}
+
+type EnvMap = Record<string, string>;
+
+function parseCommaEnvList(input: string): EnvMap {
+ const out: EnvMap = {};
+ const s = (input ?? "").trim();
+ if (s.length === 0) return out;
+
+ for (const raw of s.split(",")) {
+ const pair = raw.trim();
+ if (!pair) continue;
+
+ const eq = pair.indexOf("=");
+ if (eq <= 0) {
+ throw new Error(`Invalid env pair "${pair}". Expected KEY=VALUE.`);
+ }
+
+ const key = pair.slice(0, eq).trim();
+ const value = pair.slice(eq + 1).trim();
+
+ if (!/^[A-Za-z_][A-Za-z0-9_]*$/.test(key)) {
+ throw new Error(`Invalid env key "${key}". Must match [A-Za-z_][A-Za-z0-9_]*.`);
+ }
+
+ out[key] = value;
+ }
+ return out;
+}
+
+function collectAdditionalEnv(argv: any): EnvMap {
+ const entries = ([] as string[]).concat(argv.additionalEnv ?? argv["additional-env"] ?? []);
+ const merged: EnvMap = {};
+ for (const entry of entries) {
+ const parsed = parseCommaEnvList(entry);
+ Object.assign(merged, parsed);
+ }
+ return merged;
+}
+
+function prettyPrintError(error: unknown) {
+ console.error(`\x1b[31m[${CLI_NAME}] %s\x1b[0m`, error instanceof Error ? error.message : String(error));
+}
+
+function isAppleSilicon(): boolean {
+ return process.platform === "darwin" && process.arch === "arm64";
+}
+
+function mergeEnv(overrides: EnvMap = {}): NodeJS.ProcessEnv {
+ return { ...process.env, ...basePlaywrightEnv(), ...overrides };
+}
+
+function dockerComposeUp(ci: boolean, overrides: Record<string, string> = {}) {
+ const composeBase = "./tests-e2e/__containerization__/playwright-docker-compose.yml";
+ const composeCi = "./tests-e2e/__containerization__/playwright-docker-compose.ci.yml";
+ const files = ci ? [`-f`, composeBase, `-f`, composeCi] : [`-f`, composeBase];
+
+ execSync(`docker compose ${files.join(" ")} up -d`, { stdio: "inherit", env: mergeEnv(overrides) });
+}
+
+async function main() {
+ try {
+ await yargs(hideBin(process.argv))
+ .version(false)
+ .wrap(Math.min(150, terminalWidth()))
+ .scriptName(CLI_NAME)
+ .epilog(
+ `A CLI tool to assist running containerized Playwright tests.
+It loads environment variables and passes them through the docker-compose file, and helps with starting the test suite, opening an interactive shell inside the container, and cleaning up.`
+ )
+ .example(
+ "containerized-playwright run --container-name my_playwright_container --container-workdir incubator-kie-tools/packages/my-package\n",
+ "Run the Playwright test suite locally (no CI override) using the specified container and workdir.\n"
+ )
+ .example(
+ "containerized-playwright run --ci --container-name my_playwright_container --container-workdir incubator-kie-tools/packages/my-package\n",
+ "Run the Playwright test suite in CI mode (applies CI docker-compose override).\n"
+ )
+ .example(
+ "containerized-playwright run --container-name e2e --container-workdir incubator-kie-tools/packages/foo --additional-env BUILD_ID=123,REPORT_DIR=/tmp/reports\n",
+ "Run with extra environment variables forwarded to docker-compose (comma-separated KEY=VALUE pairs).\n"
+ )
+ .example(
+ "containerized-playwright run --container-name e2e --container-workdir incubator-kie-tools/packages/foo --additional-env FOO=bar --additional-env COMMIT_SHA=deadbeef\n",
+ "Pass multiple --additional-env options; later pairs override earlier keys if duplicated.\n"
+ )
+ .example(
+ "CI=true containerized-playwright run --container-name e2e --container-workdir incubator-kie-tools/packages/foo\n",
+ "Leverage CI environment variable to auto-enable CI mode (equivalent to --ci when CI=true or CI=1).\n"
+ )
+ .example(
+ "containerized-playwright shell --container-name my_playwright_container --container-workdir incubator-kie-tools/packages/my-package\n",
+ "Start the container (if needed) and open an interactive bash shell in the package workdir.\n"
+ )
+ .example(
+ "containerized-playwright shell --container-name e2e --container-workdir incubator-kie-tools/packages/foo --additional-env DEBUG=true\n",
+ "Open an interactive shell with extra environment variables forwarded to docker-compose.\n"
+ )
+ .example(
+ "containerized-playwright shell --container-name e2e --container-workdir incubator-kie-tools/packages/foo --additional-env FOO=bar,BAZ=qux\n",
+ "Forward multiple env variables in a single --additional-env option using comma-separated pairs.\n"
+ )
+ .example(
+ "containerized-playwright clean\n",
+ "Stop and remove Playwright-related containers using the base docker-compose file.\n"
+ )
+ .command(
+ "run",
+ "Run the Playwright test suite inside Docker containers. This command will start the required containers using docker-compose and execute the Playwright tests in the specified container workdir.",
+ (yargs) => {
+ yargs
+ .option("ci", {
+ type: "boolean",
+ default: process.env.CI === "true" || process.env.CI === "1",
+ describe: "Enable CI mode by applying the CI-specific docker-compose override file.",
+ })
+ .option("additional-env", {
+ type: "string",
+ default: "",
+ describe:
+ "Comma-separated KEY=VALUE pairs of additional environment variables to forward to docker-compose. Can be repeated.",
+ })
+ .option("container-name", {
+ type: "string",
+ default: "",
+ describe: "Name of the container as defined in the docker-compose file. Required.",
+ demandOption: true,
+ })
+ .option("container-workdir", {
+ type: "string",
+ default: "",
+ describe:
+ "Path inside the container where Playwright tests are located. Example: incubator-kie-tools/packages/<package_name>. Required.",
+ demandOption: true,
+ });
+ },
+ (argv) => {
+ const extraEnv = collectAdditionalEnv(argv);
+ // If Google Chrome is enabled check if tests are enabled for Apple Silicon
+ const isGoogleChromeEnabled =
+ buildEnv.playwrightBase.enableGoogleChromeProject === true
+ ? isAppleSilicon() && buildEnv.playwrightBase.enableGoogleChromeTestsForAppleSilicon === true
+ ? "true"
+ : !isAppleSilicon()
+ ? "true"
+ : "false"
+ : "false";
+
+ dockerComposeUp(!!argv.ci, {
+ PLAYWRIGHT_BASE__enableGoogleChromeProject: isGoogleChromeEnabled,
+ ...extraEnv,
+ });
+ console.info(
+ `[${CLI_NAME}] docker compose up done. Env
+CI=${!!argv.ci}
+PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon=${buildEnv.playwrightBase.enableGoogleChromeTestsForAppleSilicon}
+PLAYWRIGHT_BASE__enableChromiumProject=${buildEnv.playwrightBase.enableChromiumProject}
+PLAYWRIGHT_BASE__enableGoogleChromeProject=${isGoogleChromeEnabled}
+PLAYWRIGHT_BASE__enableWebkitProject=${buildEnv.playwrightBase.enableWebkitProject}
+PLAYWRIGHT_BASE__projectTimeout=${buildEnv.playwrightBase.projectTimeout}
+PLAYWRIGHT_BASE__expectTimeout=${buildEnv.playwrightBase.expectTimeout}
+PLAYWRIGHT_BASE__maxDiffPixelRatio=${buildEnv.playwrightBase.maxDiffPixelRatio}
+PLAYWRIGHT_BASE__retries=${buildEnv.playwrightBase.retries}
+PLAYWRIGHT_BASE__workers=${buildEnv.playwrightBase.workers}
+extraEnv=${JSON.stringify(extraEnv)}
+`
+ );
+
+ execSync(
+ `docker exec -i ${argv["container-name"]} /bin/bash -c "cd ${argv["container-workdir"]} && pnpm test-e2e:run"`,
+ {
+ stdio: "inherit",
+ }
+ );
+ }
+ )
+ .command(
+ "shell",
+ "Open an interactive shell inside the Playwright test container. This command starts the required container using docker-compose and launches a shell in the specified workdir inside the container.",
+ (yargs) => {
+ yargs
+ .option("additional-env", {
+ type: "string",
+ default: "",
+ describe:
+ "Comma-separated KEY=VALUE pairs of additional environment variables to forward to docker-compose. Can be repeated.",
+ })
+ .option("container-name", {
+ type: "string",
+ default: "",
+ describe: "Name of the container as defined in the docker-compose file. Required.",
+ demandOption: true,
+ })
+ .option("container-workdir", {
+ type: "string",
+ default: "",
+ describe:
+ "Path inside the container where Playwright tests are located. Example: incubator-kie-tools/packages/<package_name>. Required.",
+ demandOption: true,
+ });
+ },
+ (argv) => {
+ const extraEnv = collectAdditionalEnv(argv);
+ const isGoogleChromeEnabled =
+ buildEnv.playwrightBase.enableGoogleChromeProject === true
+ ? isAppleSilicon() && buildEnv.playwrightBase.enableGoogleChromeTestsForAppleSilicon === true
+ ? "true"
+ : !isAppleSilicon()
+ ? "true"
+ : "false"
+ : "false";
+ dockerComposeUp(false, {
+ PLAYWRIGHT_BASE__enableGoogleChromeProject: isGoogleChromeEnabled,
+ ...extraEnv,
+ });
+ console.info(
+ `[${CLI_NAME}] docker compose up done. Env
+CI=${!!argv.ci}
+PLAYWRIGHT_BASE__enableChromiumProject=${buildEnv.playwrightBase.enableChromiumProject}
+PLAYWRIGHT_BASE__enableGoogleChromeProject=${isGoogleChromeEnabled}
+PLAYWRIGHT_BASE__enableWebkitProject=${buildEnv.playwrightBase.enableWebkitProject}
+PLAYWRIGHT_BASE__projectTimeout=${buildEnv.playwrightBase.projectTimeout}
+PLAYWRIGHT_BASE__expectTimeout=${buildEnv.playwrightBase.expectTimeout}
+PLAYWRIGHT_BASE__maxDiffPixelRatio=${buildEnv.playwrightBase.maxDiffPixelRatio}
+PLAYWRIGHT_BASE__retries=${buildEnv.playwrightBase.retries}
+PLAYWRIGHT_BASE__workers=${buildEnv.playwrightBase.workers}
+extraEnv=${JSON.stringify(extraEnv)}
Review Comment:
Also in here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issues#2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "tiagobento (via GitHub)" <gi...@apache.org>.
tiagobento commented on PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#issuecomment-3682688498
Thanks @ljmotta. Great work standardizing the containerized approach for E2E testing with Playwright.
A couple comments/questions:
- I noticed a slight difference in the font on our screenshots inside the FEEL expressions. Is that expected? What is the reason for this change?
- Since `pw-e2e-container` is inside the `playwright-base` package... WDYT about naming it "playwright-base-container` instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issues#2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "ljmotta (via GitHub)" <gi...@apache.org>.
ljmotta commented on PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#issuecomment-3705518149
@tiagobento Could you please re-review?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "ljmotta (via GitHub)" <gi...@apache.org>.
ljmotta commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635294387
##########
packages/playwright-base/README.md:
##########
@@ -19,35 +19,15 @@
## Overview
-This package collects common configurations to run end-to-end Playwright tests.
-
-## Installing Playwright deps
-
-Currently, all Playwright end-to-end tests run inside containers. If you need to debug a test, we recommend doing so on the host machine. To do this, you need to install the Playwright dependencies. Use the `PLAYWRIGHT_BASE__installDeps` environment variable during the bootstrap phase to install all the required dependencies.
-
-```sh
-# in the `kie-tools` root
-PLAYWRIGHT_BASE__installDeps=true pnpm bootstrap
-```
-
-or
-
-```sh
-# in the `kie-tools` root
-PLAYWRIGHT_BASE__installDeps=true pnpm bootstrap -F playwright-base
-```
-
-> **i NOTE**
->
-> Since this step install the Playwright browsers, it requires sudo permision.
+This package collects common configurations to run end-to-end Playwright tests. Currently, all Playwright end-to-end tests run inside containers. If you need to debug a test, we recommend doing so on the host machine, refer to the [Installing Playwright deps](#Installing-Playwright-deps)
## Using containers to generate screenshots
Each operating system has slight variations in UI, even within the same browser. These differences can cause screenshot comparison tests to fail. To address this issue and ensure a stable environment with consistent test results locally and in CI, containers can be used. Running Playwright tests inside a container that is also used in the CI environment makes screenshot tests reproducible, regardless of the host OS.
> **ℹ️ NOTE**
>
-> Due to compatibility issues, this containerization solution cannot yet be used on native Windows and requires running it directly within WSL (Windows Subsystem for Linux). Also, Linux arm64 doesn't support Google Chrome, and due to this caveat, some tests will be disabled for this arch.
+> Due to compatibility issues, this containerization solution cannot yet be used on native Windows and requires running it directly within WSL (Windows Subsystem for Linux). Also, Linux arm64 doesn't support Google Chrome, and due to this caveat, Google Chrome tests in macOS arm are disabled in some packages.
Review Comment:
Actually, I'll update to something like WDYT @thiagoelg :
> Due to compatibility issues, this containerization solution cannot yet be used on native Windows and requires running it directly within WSL (Windows Subsystem for Linux). Additionally, Google Chrome is not available for Linux on ARM64, so any Chrome-based tests in an ARM64 container must be disabled. At present, tests run on AMD64 containers to ensure all browsers and test scenarios execute safely.
We don't have any tests running on arm64, some tests are disabled on Apple Silicon due to historical reasons.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issues#2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "ljmotta (via GitHub)" <gi...@apache.org>.
ljmotta commented on PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#issuecomment-3682937814
@tiagobento the screenshots on `boxed-expression-component` were outdated as they were using the `0.1` maxDiffPixel instead of the `0.001` maxDiffPixel that was in place for the other tests.
Sure, I'll update it the CLI name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "thiagoelg (via GitHub)" <gi...@apache.org>.
thiagoelg commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635061364
##########
packages/playwright-base/README.md:
##########
@@ -79,6 +59,76 @@ pnpm test-e2e:container:shell
pnpm test-e2e:run --update-snapshots
```
+## Playwright-base CLI
+
+A CLI tool to assist running containerized Playwright tests. It loads environment variables and passes them through the docker-compose file, and helps with starting the test suite, opening an interactive shell inside the container, and cleaning up.
+
+```sh
+pnpm pw-e2e-container --help
+pw-e2e-container <command>
+
+Commands:
+ pw-e2e-container run Run the Playwright test suite inside Docker containers. This command will start the required containers using docker-compose
+ and execute the Playwright tests in the specified container workdir.
+ pw-e2e-container shell Open an interactive shell inside the Playwright test container. This command starts the required container using
+ docker-compose and launches a shell in the specified workdir inside the container.
+ pw-e2e-container clean Stop and remove all Playwright-related containers created by docker-compose. This command runs 'docker compose down' using
+ the base Playwright compose file.
+
+Examples:
+ containerized-playwright run --container-name my_playwright_container Run the Playwright test suite locally (no CI override) using the
+ --container-workdir incubator-kie-tools/packages/my-package specified container and workdir.
+
+ containerized-playwright run --ci --container-name my_playwright_container Run the Playwright test suite in CI mode (applies CI docker-compose
+ --container-workdir incubator-kie-tools/packages/my-package override).
+
+ containerized-playwright run --container-name e2e --container-workdir Run with extra environment variables forwarded to docker-compose
+ incubator-kie-tools/packages/foo --additional-env (comma-separated KEY=VALUE pairs).
+ BUILD_ID=123,REPORT_DIR=/tmp/reports
+
+ containerized-playwright run --container-name e2e --container-workdir Pass multiple --additional-env options; later pairs override earlier
+ incubator-kie-tools/packages/foo --additional-env FOO=bar --additional-env keys if duplicated.
+ COMMIT_SHA=deadbeef
+
+ CI=true containerized-playwright run --container-name e2e Leverage CI environment variable to auto-enable CI mode (equivalent to
+ --container-workdir incubator-kie-tools/packages/foo --ci when CI=true or CI=1).
+
+ containerized-playwright shell --container-name my_playwright_container Start the container (if needed) and open an interactive bash shell in
+ --container-workdir incubator-kie-tools/packages/my-package the package workdir.
+
+ containerized-playwright shell --container-name e2e --container-workdir Open an interactive shell with extra environment variables forwarded to
+ incubator-kie-tools/packages/foo --additional-env DEBUG=true docker-compose.
+
+ containerized-playwright shell --container-name e2e --container-workdir Forward multiple env variables in a single --additional-env option
+ incubator-kie-tools/packages/foo --additional-env FOO=bar,BAZ=qux using comma-separated pairs.
+
+ containerized-playwright clean Stop and remove Playwright-related containers using the base
+ docker-compose file.
+```
+
+## Installing Playwright dependencies on host machine
+
+> **i Warning**
Review Comment:
```suggestion
> **ℹ️ Warning**
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "thiagoelg (via GitHub)" <gi...@apache.org>.
thiagoelg commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635531254
##########
packages/playwright-base/src/bin.ts:
##########
@@ -0,0 +1,319 @@
+#!/usr/bin/env node
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import { execSync, spawnSync } from "child_process";
+import yargs from "yargs/yargs";
+import { terminalWidth, exit } from "yargs";
+import { hideBin } from "yargs/helpers";
+
+// eslint-disable-next-line @typescript-eslint/ban-ts-comment
+// @ts-ignore
+import { env } from "../env";
+
+const CLI_NAME = "pw-e2e-container";
+
+const buildEnv: any = env;
+
+function basePlaywrightEnv(): Record<string, string> {
+ return {
+ STREAM_NAME: buildEnv.root.streamName,
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon:
+ buildEnv.playwrightBase.enableGoogleChromeTestsForAppleSilicon,
+ PLAYWRIGHT_BASE__enableChromiumProject: buildEnv.playwrightBase.enableChromiumProject,
+ PLAYWRIGHT_BASE__enableGoogleChromeProject: buildEnv.playwrightBase.enableGoogleChromeProject,
+ PLAYWRIGHT_BASE__enableWebkitProject: buildEnv.playwrightBase.enableWebkitProject,
+ PLAYWRIGHT_BASE__projectTimeout: buildEnv.playwrightBase.projectTimeout,
+ PLAYWRIGHT_BASE__expectTimeout: buildEnv.playwrightBase.expectTimeout,
+ PLAYWRIGHT_BASE__maxDiffPixelRatio: buildEnv.playwrightBase.maxDiffPixelRatio,
+ PLAYWRIGHT_BASE__retries: buildEnv.playwrightBase.retries,
+ PLAYWRIGHT_BASE__workers: buildEnv.playwrightBase.workers,
+ };
+}
Review Comment:
Yes, something like that, but if you feel it doesn't help that much, it's OK to leave it like this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "thiagoelg (via GitHub)" <gi...@apache.org>.
thiagoelg commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635057299
##########
packages/playwright-base/README.md:
##########
@@ -79,6 +59,76 @@ pnpm test-e2e:container:shell
pnpm test-e2e:run --update-snapshots
```
+## Playwright-base CLI
+
+A CLI tool to assist running containerized Playwright tests. It loads environment variables and passes them through the docker-compose file, and helps with starting the test suite, opening an interactive shell inside the container, and cleaning up.
+
+```sh
+pnpm pw-e2e-container --help
+pw-e2e-container <command>
+
+Commands:
+ pw-e2e-container run Run the Playwright test suite inside Docker containers. This command will start the required containers using docker-compose
+ and execute the Playwright tests in the specified container workdir.
+ pw-e2e-container shell Open an interactive shell inside the Playwright test container. This command starts the required container using
+ docker-compose and launches a shell in the specified workdir inside the container.
+ pw-e2e-container clean Stop and remove all Playwright-related containers created by docker-compose. This command runs 'docker compose down' using
+ the base Playwright compose file.
+
+Examples:
+ containerized-playwright run --container-name my_playwright_container Run the Playwright test suite locally (no CI override) using the
+ --container-workdir incubator-kie-tools/packages/my-package specified container and workdir.
+
+ containerized-playwright run --ci --container-name my_playwright_container Run the Playwright test suite in CI mode (applies CI docker-compose
+ --container-workdir incubator-kie-tools/packages/my-package override).
+
+ containerized-playwright run --container-name e2e --container-workdir Run with extra environment variables forwarded to docker-compose
+ incubator-kie-tools/packages/foo --additional-env (comma-separated KEY=VALUE pairs).
+ BUILD_ID=123,REPORT_DIR=/tmp/reports
+
+ containerized-playwright run --container-name e2e --container-workdir Pass multiple --additional-env options; later pairs override earlier
+ incubator-kie-tools/packages/foo --additional-env FOO=bar --additional-env keys if duplicated.
+ COMMIT_SHA=deadbeef
+
+ CI=true containerized-playwright run --container-name e2e Leverage CI environment variable to auto-enable CI mode (equivalent to
+ --container-workdir incubator-kie-tools/packages/foo --ci when CI=true or CI=1).
+
+ containerized-playwright shell --container-name my_playwright_container Start the container (if needed) and open an interactive bash shell in
+ --container-workdir incubator-kie-tools/packages/my-package the package workdir.
+
+ containerized-playwright shell --container-name e2e --container-workdir Open an interactive shell with extra environment variables forwarded to
+ incubator-kie-tools/packages/foo --additional-env DEBUG=true docker-compose.
+
+ containerized-playwright shell --container-name e2e --container-workdir Forward multiple env variables in a single --additional-env option
+ incubator-kie-tools/packages/foo --additional-env FOO=bar,BAZ=qux using comma-separated pairs.
+
+ containerized-playwright clean Stop and remove Playwright-related containers using the base
Review Comment:
`containerized-playwright` should be `pw-e2e-container`, no?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issues#2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "ljmotta (via GitHub)" <gi...@apache.org>.
ljmotta merged PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "ljmotta (via GitHub)" <gi...@apache.org>.
ljmotta commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635719012
##########
packages/playwright-base/src/bin.ts:
##########
@@ -0,0 +1,319 @@
+#!/usr/bin/env node
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import { execSync, spawnSync } from "child_process";
+import yargs from "yargs/yargs";
+import { terminalWidth, exit } from "yargs";
+import { hideBin } from "yargs/helpers";
+
+// eslint-disable-next-line @typescript-eslint/ban-ts-comment
+// @ts-ignore
+import { env } from "../env";
+
+const CLI_NAME = "pw-e2e-container";
+
+const buildEnv: any = env;
+
+function basePlaywrightEnv(): Record<string, string> {
+ return {
+ STREAM_NAME: buildEnv.root.streamName,
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon:
+ buildEnv.playwrightBase.enableGoogleChromeTestsForAppleSilicon,
+ PLAYWRIGHT_BASE__enableChromiumProject: buildEnv.playwrightBase.enableChromiumProject,
+ PLAYWRIGHT_BASE__enableGoogleChromeProject: buildEnv.playwrightBase.enableGoogleChromeProject,
+ PLAYWRIGHT_BASE__enableWebkitProject: buildEnv.playwrightBase.enableWebkitProject,
+ PLAYWRIGHT_BASE__projectTimeout: buildEnv.playwrightBase.projectTimeout,
+ PLAYWRIGHT_BASE__expectTimeout: buildEnv.playwrightBase.expectTimeout,
+ PLAYWRIGHT_BASE__maxDiffPixelRatio: buildEnv.playwrightBase.maxDiffPixelRatio,
+ PLAYWRIGHT_BASE__retries: buildEnv.playwrightBase.retries,
+ PLAYWRIGHT_BASE__workers: buildEnv.playwrightBase.workers,
+ };
+}
Review Comment:
I'll keep as it is, as `pnpm build-env --print-dotenv` will use the caller context. The result will vary depending on the package.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "ljmotta (via GitHub)" <gi...@apache.org>.
ljmotta commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635076122
##########
packages/playwright-base/README.md:
##########
@@ -79,6 +59,76 @@ pnpm test-e2e:container:shell
pnpm test-e2e:run --update-snapshots
```
+## Playwright-base CLI
+
+A CLI tool to assist running containerized Playwright tests. It loads environment variables and passes them through the docker-compose file, and helps with starting the test suite, opening an interactive shell inside the container, and cleaning up.
+
+```sh
+pnpm pw-e2e-container --help
+pw-e2e-container <command>
+
+Commands:
+ pw-e2e-container run Run the Playwright test suite inside Docker containers. This command will start the required containers using docker-compose
+ and execute the Playwright tests in the specified container workdir.
+ pw-e2e-container shell Open an interactive shell inside the Playwright test container. This command starts the required container using
+ docker-compose and launches a shell in the specified workdir inside the container.
+ pw-e2e-container clean Stop and remove all Playwright-related containers created by docker-compose. This command runs 'docker compose down' using
+ the base Playwright compose file.
+
+Examples:
+ containerized-playwright run --container-name my_playwright_container Run the Playwright test suite locally (no CI override) using the
+ --container-workdir incubator-kie-tools/packages/my-package specified container and workdir.
+
+ containerized-playwright run --ci --container-name my_playwright_container Run the Playwright test suite in CI mode (applies CI docker-compose
+ --container-workdir incubator-kie-tools/packages/my-package override).
+
+ containerized-playwright run --container-name e2e --container-workdir Run with extra environment variables forwarded to docker-compose
+ incubator-kie-tools/packages/foo --additional-env (comma-separated KEY=VALUE pairs).
+ BUILD_ID=123,REPORT_DIR=/tmp/reports
+
+ containerized-playwright run --container-name e2e --container-workdir Pass multiple --additional-env options; later pairs override earlier
+ incubator-kie-tools/packages/foo --additional-env FOO=bar --additional-env keys if duplicated.
+ COMMIT_SHA=deadbeef
+
+ CI=true containerized-playwright run --container-name e2e Leverage CI environment variable to auto-enable CI mode (equivalent to
+ --container-workdir incubator-kie-tools/packages/foo --ci when CI=true or CI=1).
+
+ containerized-playwright shell --container-name my_playwright_container Start the container (if needed) and open an interactive bash shell in
+ --container-workdir incubator-kie-tools/packages/my-package the package workdir.
+
+ containerized-playwright shell --container-name e2e --container-workdir Open an interactive shell with extra environment variables forwarded to
+ incubator-kie-tools/packages/foo --additional-env DEBUG=true docker-compose.
+
+ containerized-playwright shell --container-name e2e --container-workdir Forward multiple env variables in a single --additional-env option
+ incubator-kie-tools/packages/foo --additional-env FOO=bar,BAZ=qux using comma-separated pairs.
+
+ containerized-playwright clean Stop and remove Playwright-related containers using the base
Review Comment:
I changed the cli name, and I forgot to update the readme :face_with_peeking_eye:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "thiagoelg (via GitHub)" <gi...@apache.org>.
thiagoelg commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635074212
##########
packages/playwright-base/env/index.js:
##########
@@ -25,11 +25,58 @@ module.exports = composeEnv([require("@kie-tools/root-env/env")], {
default: "false",
description: "Toggles the installation of Playwright dependencies. Can be `true` or `false`.",
},
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon: {
+ default: "true",
+ description: "Enable Google Chrome tests for ARM OSs. Overrides PLAYWRIGHT_BASE__enableGoogleChromeProject.",
+ },
+ PLAYWRIGHT_BASE__enableChromiumProject: {
+ default: "true",
+ description: "Enable or disable Chromium project",
+ },
+ PLAYWRIGHT_BASE__enableGoogleChromeProject: {
+ default: "true",
+ description: "Enable or disable Google Chrome project",
+ },
+ PLAYWRIGHT_BASE__enableWebkitProject: {
+ default: "true",
+ description: "Enable or disable Webkit project",
+ },
+ PLAYWRIGHT_BASE__projectTimeout: {
+ default: "90000",
+ description: "The timeout of a test in the project (Chromium, Google Chrome and Webkit)",
+ },
+ PLAYWRIGHT_BASE__expectTimeout: {
+ default: "30000",
+ description: "The timeout of an expect assertion",
+ },
+ PLAYWRIGHT_BASE__maxDiffPixelRatio: {
+ default: "0.001",
+ description: "The threshold of the maximum difference on screenshot comparisons (in percentage)",
Review Comment:
Is `0.001` = 0.001% or 0.1%? If it's 0.1% I would use `(percentage in decimal form)` just to be a bit clearer.
```suggestion
default: "0.001",
description: "The threshold of the maximum pixel difference on screenshot comparisons (percentage in decimal form)",
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "ljmotta (via GitHub)" <gi...@apache.org>.
ljmotta commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635166407
##########
packages/playwright-base/env/index.js:
##########
@@ -25,11 +25,58 @@ module.exports = composeEnv([require("@kie-tools/root-env/env")], {
default: "false",
description: "Toggles the installation of Playwright dependencies. Can be `true` or `false`.",
},
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon: {
+ default: "true",
+ description: "Enable Google Chrome tests for ARM OSs. Overrides PLAYWRIGHT_BASE__enableGoogleChromeProject.",
+ },
Review Comment:
I believe we could create a new issue to re-asses the need for skipping tests on Google Chrome for darwin arm64.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "thiagoelg (via GitHub)" <gi...@apache.org>.
thiagoelg commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635049488
##########
packages/playwright-base/README.md:
##########
@@ -19,35 +19,15 @@
## Overview
-This package collects common configurations to run end-to-end Playwright tests.
-
-## Installing Playwright deps
-
-Currently, all Playwright end-to-end tests run inside containers. If you need to debug a test, we recommend doing so on the host machine. To do this, you need to install the Playwright dependencies. Use the `PLAYWRIGHT_BASE__installDeps` environment variable during the bootstrap phase to install all the required dependencies.
-
-```sh
-# in the `kie-tools` root
-PLAYWRIGHT_BASE__installDeps=true pnpm bootstrap
-```
-
-or
-
-```sh
-# in the `kie-tools` root
-PLAYWRIGHT_BASE__installDeps=true pnpm bootstrap -F playwright-base
-```
-
-> **i NOTE**
->
-> Since this step install the Playwright browsers, it requires sudo permision.
+This package collects common configurations to run end-to-end Playwright tests. Currently, all Playwright end-to-end tests run inside containers. If you need to debug a test, we recommend doing so on the host machine, refer to the [Installing Playwright deps](#Installing-Playwright-deps)
## Using containers to generate screenshots
Each operating system has slight variations in UI, even within the same browser. These differences can cause screenshot comparison tests to fail. To address this issue and ensure a stable environment with consistent test results locally and in CI, containers can be used. Running Playwright tests inside a container that is also used in the CI environment makes screenshot tests reproducible, regardless of the host OS.
> **ℹ️ NOTE**
>
-> Due to compatibility issues, this containerization solution cannot yet be used on native Windows and requires running it directly within WSL (Windows Subsystem for Linux). Also, Linux arm64 doesn't support Google Chrome, and due to this caveat, some tests will be disabled for this arch.
+> Due to compatibility issues, this containerization solution cannot yet be used on native Windows and requires running it directly within WSL (Windows Subsystem for Linux). Also, Linux arm64 doesn't support Google Chrome, and due to this caveat, Google Chrome tests in macOS arm are disabled in some packages.
Review Comment:
> Also, Linux arm64 doesn't support Google Chrome, and due to this caveat, Google Chrome tests in macOS arm are disabled in some packages.
This is a bit confusing... Can you replace it with something like:
> Also, Google Chrome is not available for Linux on ARM64, so any Google Chrome tests running under a Linux ARM64 container will be disabled. This is usually the case when running Linux containers on Apple Silicon machines, so Google Chrome tests are disabled in some packages under macOS ARM64.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "ljmotta (via GitHub)" <gi...@apache.org>.
ljmotta commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635192000
##########
packages/playwright-base/src/bin.ts:
##########
@@ -0,0 +1,319 @@
+#!/usr/bin/env node
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import { execSync, spawnSync } from "child_process";
+import yargs from "yargs/yargs";
+import { terminalWidth, exit } from "yargs";
+import { hideBin } from "yargs/helpers";
+
+// eslint-disable-next-line @typescript-eslint/ban-ts-comment
+// @ts-ignore
+import { env } from "../env";
+
+const CLI_NAME = "pw-e2e-container";
+
+const buildEnv: any = env;
+
+function basePlaywrightEnv(): Record<string, string> {
+ return {
+ STREAM_NAME: buildEnv.root.streamName,
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon:
+ buildEnv.playwrightBase.enableGoogleChromeTestsForAppleSilicon,
+ PLAYWRIGHT_BASE__enableChromiumProject: buildEnv.playwrightBase.enableChromiumProject,
+ PLAYWRIGHT_BASE__enableGoogleChromeProject: buildEnv.playwrightBase.enableGoogleChromeProject,
+ PLAYWRIGHT_BASE__enableWebkitProject: buildEnv.playwrightBase.enableWebkitProject,
+ PLAYWRIGHT_BASE__projectTimeout: buildEnv.playwrightBase.projectTimeout,
+ PLAYWRIGHT_BASE__expectTimeout: buildEnv.playwrightBase.expectTimeout,
+ PLAYWRIGHT_BASE__maxDiffPixelRatio: buildEnv.playwrightBase.maxDiffPixelRatio,
+ PLAYWRIGHT_BASE__retries: buildEnv.playwrightBase.retries,
+ PLAYWRIGHT_BASE__workers: buildEnv.playwrightBase.workers,
+ };
+}
Review Comment:
I'm not sure how this would help creating the object. Like run the command with `execSync` and parse the result to create an object?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "ljmotta (via GitHub)" <gi...@apache.org>.
ljmotta commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635166407
##########
packages/playwright-base/env/index.js:
##########
@@ -25,11 +25,58 @@ module.exports = composeEnv([require("@kie-tools/root-env/env")], {
default: "false",
description: "Toggles the installation of Playwright dependencies. Can be `true` or `false`.",
},
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon: {
+ default: "true",
+ description: "Enable Google Chrome tests for ARM OSs. Overrides PLAYWRIGHT_BASE__enableGoogleChromeProject.",
+ },
Review Comment:
I believe we could create a new issue to re-asses the need for skipping tests on Google Chrome on apple arm64.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "thiagoelg (via GitHub)" <gi...@apache.org>.
thiagoelg commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635528937
##########
packages/playwright-base/env/index.js:
##########
@@ -25,11 +25,58 @@ module.exports = composeEnv([require("@kie-tools/root-env/env")], {
default: "false",
description: "Toggles the installation of Playwright dependencies. Can be `true` or `false`.",
},
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon: {
+ default: "true",
+ description: "Enable Google Chrome tests for ARM OSs. Overrides PLAYWRIGHT_BASE__enableGoogleChromeProject.",
+ },
Review Comment:
Sure, sounds good to me. Or we could just stop skipping those tests and see what the CI thinks 👀
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "ljmotta (via GitHub)" <gi...@apache.org>.
ljmotta commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635162451
##########
packages/playwright-base/env/index.js:
##########
@@ -25,11 +25,58 @@ module.exports = composeEnv([require("@kie-tools/root-env/env")], {
default: "false",
description: "Toggles the installation of Playwright dependencies. Can be `true` or `false`.",
},
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon: {
+ default: "true",
+ description: "Enable Google Chrome tests for ARM OSs. Overrides PLAYWRIGHT_BASE__enableGoogleChromeProject.",
+ },
Review Comment:
@thiagoelg Currently, tests run on an amd64 container, so we don't have an incopatibility issue. Now, this PR keeps compatibility with what we currently have, and Google Chrome tests were being skipped on `online-editor` and `scesim-editor` even when running on a amd64 container.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org
Re: [PR] kie-issue-2197: Allow customizing Playwright test runs in CI without code changes [incubator-kie-tools]
Posted by "ljmotta (via GitHub)" <gi...@apache.org>.
ljmotta commented on code in PR #3382:
URL: https://github.com/apache/incubator-kie-tools/pull/3382#discussion_r2635218533
##########
packages/playwright-base/src/bin.ts:
##########
@@ -0,0 +1,319 @@
+#!/usr/bin/env node
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import { execSync, spawnSync } from "child_process";
+import yargs from "yargs/yargs";
+import { terminalWidth, exit } from "yargs";
+import { hideBin } from "yargs/helpers";
+
+// eslint-disable-next-line @typescript-eslint/ban-ts-comment
+// @ts-ignore
+import { env } from "../env";
+
+const CLI_NAME = "pw-e2e-container";
+
+const buildEnv: any = env;
+
+function basePlaywrightEnv(): Record<string, string> {
+ return {
+ STREAM_NAME: buildEnv.root.streamName,
+ PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon:
+ buildEnv.playwrightBase.enableGoogleChromeTestsForAppleSilicon,
+ PLAYWRIGHT_BASE__enableChromiumProject: buildEnv.playwrightBase.enableChromiumProject,
+ PLAYWRIGHT_BASE__enableGoogleChromeProject: buildEnv.playwrightBase.enableGoogleChromeProject,
+ PLAYWRIGHT_BASE__enableWebkitProject: buildEnv.playwrightBase.enableWebkitProject,
+ PLAYWRIGHT_BASE__projectTimeout: buildEnv.playwrightBase.projectTimeout,
+ PLAYWRIGHT_BASE__expectTimeout: buildEnv.playwrightBase.expectTimeout,
+ PLAYWRIGHT_BASE__maxDiffPixelRatio: buildEnv.playwrightBase.maxDiffPixelRatio,
+ PLAYWRIGHT_BASE__retries: buildEnv.playwrightBase.retries,
+ PLAYWRIGHT_BASE__workers: buildEnv.playwrightBase.workers,
+ };
+}
+
+type EnvMap = Record<string, string>;
+
+function parseCommaEnvList(input: string): EnvMap {
+ const out: EnvMap = {};
+ const s = (input ?? "").trim();
+ if (s.length === 0) return out;
+
+ for (const raw of s.split(",")) {
+ const pair = raw.trim();
+ if (!pair) continue;
+
+ const eq = pair.indexOf("=");
+ if (eq <= 0) {
+ throw new Error(`Invalid env pair "${pair}". Expected KEY=VALUE.`);
+ }
+
+ const key = pair.slice(0, eq).trim();
+ const value = pair.slice(eq + 1).trim();
+
+ if (!/^[A-Za-z_][A-Za-z0-9_]*$/.test(key)) {
+ throw new Error(`Invalid env key "${key}". Must match [A-Za-z_][A-Za-z0-9_]*.`);
+ }
+
+ out[key] = value;
+ }
+ return out;
+}
+
+function collectAdditionalEnv(argv: any): EnvMap {
+ const entries = ([] as string[]).concat(argv.additionalEnv ?? argv["additional-env"] ?? []);
+ const merged: EnvMap = {};
+ for (const entry of entries) {
+ const parsed = parseCommaEnvList(entry);
+ Object.assign(merged, parsed);
+ }
+ return merged;
+}
+
+function prettyPrintError(error: unknown) {
+ console.error(`\x1b[31m[${CLI_NAME}] %s\x1b[0m`, error instanceof Error ? error.message : String(error));
+}
+
+function isAppleSilicon(): boolean {
+ return process.platform === "darwin" && process.arch === "arm64";
+}
+
+function mergeEnv(overrides: EnvMap = {}): NodeJS.ProcessEnv {
+ return { ...process.env, ...basePlaywrightEnv(), ...overrides };
+}
+
+function dockerComposeUp(ci: boolean, overrides: Record<string, string> = {}) {
+ const composeBase = "./tests-e2e/__containerization__/playwright-docker-compose.yml";
+ const composeCi = "./tests-e2e/__containerization__/playwright-docker-compose.ci.yml";
+ const files = ci ? [`-f`, composeBase, `-f`, composeCi] : [`-f`, composeBase];
+
+ execSync(`docker compose ${files.join(" ")} up -d`, { stdio: "inherit", env: mergeEnv(overrides) });
+}
+
+async function main() {
+ try {
+ await yargs(hideBin(process.argv))
+ .version(false)
+ .wrap(Math.min(150, terminalWidth()))
+ .scriptName(CLI_NAME)
+ .epilog(
+ `A CLI tool to assist running containerized Playwright tests.
+It loads environment variables and passes them through the docker-compose file, and helps with starting the test suite, opening an interactive shell inside the container, and cleaning up.`
+ )
+ .example(
+ "containerized-playwright run --container-name my_playwright_container --container-workdir incubator-kie-tools/packages/my-package\n",
+ "Run the Playwright test suite locally (no CI override) using the specified container and workdir.\n"
+ )
+ .example(
+ "containerized-playwright run --ci --container-name my_playwright_container --container-workdir incubator-kie-tools/packages/my-package\n",
+ "Run the Playwright test suite in CI mode (applies CI docker-compose override).\n"
+ )
+ .example(
+ "containerized-playwright run --container-name e2e --container-workdir incubator-kie-tools/packages/foo --additional-env BUILD_ID=123,REPORT_DIR=/tmp/reports\n",
+ "Run with extra environment variables forwarded to docker-compose (comma-separated KEY=VALUE pairs).\n"
+ )
+ .example(
+ "containerized-playwright run --container-name e2e --container-workdir incubator-kie-tools/packages/foo --additional-env FOO=bar --additional-env COMMIT_SHA=deadbeef\n",
+ "Pass multiple --additional-env options; later pairs override earlier keys if duplicated.\n"
+ )
+ .example(
+ "CI=true containerized-playwright run --container-name e2e --container-workdir incubator-kie-tools/packages/foo\n",
+ "Leverage CI environment variable to auto-enable CI mode (equivalent to --ci when CI=true or CI=1).\n"
+ )
+ .example(
+ "containerized-playwright shell --container-name my_playwright_container --container-workdir incubator-kie-tools/packages/my-package\n",
+ "Start the container (if needed) and open an interactive bash shell in the package workdir.\n"
+ )
+ .example(
+ "containerized-playwright shell --container-name e2e --container-workdir incubator-kie-tools/packages/foo --additional-env DEBUG=true\n",
+ "Open an interactive shell with extra environment variables forwarded to docker-compose.\n"
+ )
+ .example(
+ "containerized-playwright shell --container-name e2e --container-workdir incubator-kie-tools/packages/foo --additional-env FOO=bar,BAZ=qux\n",
+ "Forward multiple env variables in a single --additional-env option using comma-separated pairs.\n"
+ )
+ .example(
+ "containerized-playwright clean\n",
+ "Stop and remove Playwright-related containers using the base docker-compose file.\n"
+ )
+ .command(
+ "run",
+ "Run the Playwright test suite inside Docker containers. This command will start the required containers using docker-compose and execute the Playwright tests in the specified container workdir.",
+ (yargs) => {
+ yargs
+ .option("ci", {
+ type: "boolean",
+ default: process.env.CI === "true" || process.env.CI === "1",
+ describe: "Enable CI mode by applying the CI-specific docker-compose override file.",
+ })
+ .option("additional-env", {
+ type: "string",
+ default: "",
+ describe:
+ "Comma-separated KEY=VALUE pairs of additional environment variables to forward to docker-compose. Can be repeated.",
+ })
+ .option("container-name", {
+ type: "string",
+ default: "",
+ describe: "Name of the container as defined in the docker-compose file. Required.",
+ demandOption: true,
+ })
+ .option("container-workdir", {
+ type: "string",
+ default: "",
+ describe:
+ "Path inside the container where Playwright tests are located. Example: incubator-kie-tools/packages/<package_name>. Required.",
+ demandOption: true,
+ });
+ },
+ (argv) => {
+ const extraEnv = collectAdditionalEnv(argv);
+ // If Google Chrome is enabled check if tests are enabled for Apple Silicon
+ const isGoogleChromeEnabled =
+ buildEnv.playwrightBase.enableGoogleChromeProject === true
+ ? isAppleSilicon() && buildEnv.playwrightBase.enableGoogleChromeTestsForAppleSilicon === true
+ ? "true"
+ : !isAppleSilicon()
+ ? "true"
+ : "false"
+ : "false";
+
+ dockerComposeUp(!!argv.ci, {
+ PLAYWRIGHT_BASE__enableGoogleChromeProject: isGoogleChromeEnabled,
+ ...extraEnv,
+ });
+ console.info(
+ `[${CLI_NAME}] docker compose up done. Env
+CI=${!!argv.ci}
+PLAYWRIGHT_BASE__enableGoogleChromeTestsForAppleSilicon=${buildEnv.playwrightBase.enableGoogleChromeTestsForAppleSilicon}
+PLAYWRIGHT_BASE__enableChromiumProject=${buildEnv.playwrightBase.enableChromiumProject}
+PLAYWRIGHT_BASE__enableGoogleChromeProject=${isGoogleChromeEnabled}
+PLAYWRIGHT_BASE__enableWebkitProject=${buildEnv.playwrightBase.enableWebkitProject}
+PLAYWRIGHT_BASE__projectTimeout=${buildEnv.playwrightBase.projectTimeout}
+PLAYWRIGHT_BASE__expectTimeout=${buildEnv.playwrightBase.expectTimeout}
+PLAYWRIGHT_BASE__maxDiffPixelRatio=${buildEnv.playwrightBase.maxDiffPixelRatio}
+PLAYWRIGHT_BASE__retries=${buildEnv.playwrightBase.retries}
+PLAYWRIGHT_BASE__workers=${buildEnv.playwrightBase.workers}
+extraEnv=${JSON.stringify(extraEnv)}
Review Comment:
I could extract it to a function as the same env variables are used on `run` and `shell`. The `PLAYWRIGHT_BASE__enableGoogleChromeProject` is calculated by the CLI, so using `pnpm build-env --print-dotenv` would print a wrong value
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@kie.apache.org
For additional commands, e-mail: commits-help@kie.apache.org