You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficcontrol.apache.org by "ocket8888 (via GitHub)" <gi...@apache.org> on 2024/01/06 19:09:20 UTC

[PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

ocket8888 opened a new pull request, #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894

   This PR switches the TPv2 E2E testing suite from NightwatchJS to Cypress.
   
   "Why?", you may ask. Because it took me all of two days to replicate all of our test suite in Cypress - that's how easy it is to use. For comparison: _I have never even once been able to get the Nightwatch tests to run on my machine_. With Cypress, the typings are provided by the library itself, testing on Chrome and Firefox took exactly zero time and effort to figure out, and the tests actually just do the things I want them to do without me having to spend countless hours needling them with the correct set of `await`s and chained commands. Which is why most all of our tests do little more than check for the presence of some DOM elements. Maybe with Cypress, we could fix that.
   
   tl;dr: Nightwatch is extremely hard to use and Cypress is extremely easy.
   
   <hr/>
   
   ## Which Traffic Control components are affected by this PR?
   - Traffic Portal (experimental v2)
   
   ## What is the best way to verify this PR?
   Run the end-to-end tests. Which doesn't take very long and can be depended upon to actually work.
   
   ## PR submission checklist
   - [x] This PR has tests
   - [x] This PR has documentation
   - [x] This PR doesn't need a CHANGELOG.md entry
   - [x] This PR **DOES NOT FIX A SERIOUS SECURITY VULNERABILITY**


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

Posted by "ocket8888 (via GitHub)" <gi...@apache.org>.
ocket8888 commented on code in PR #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894#discussion_r1443906187


##########
experimental/traffic-portal/cypress.config.ts:
##########


Review Comment:
   I'd rather not touch it, just because that's where the schematic put it when I `ng add`ed Cypress



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

Posted by "zrhoffman (via GitHub)" <gi...@apache.org>.
zrhoffman commented on PR #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894#issuecomment-1906778702

   If we don't want it to affect the default builds, then
   
   - It should not be built when someone just runs `./pkg` without specifying optional builds with `-o`, and
   - It should not be included in the *CDN-in-a-Box CI* GHA workflow.
   
   But because it is included in `./pkg` by default and the *CDN-in-a-Box CI* GHA workflow, it broke tests for completely unrelated components, and that's why it was reverted.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

Posted by "ocket8888 (via GitHub)" <gi...@apache.org>.
ocket8888 commented on code in PR #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894#discussion_r1443906213


##########
.github/actions/tpv2-integration-tests/entrypoint.sh:
##########


Review Comment:
   Hm. Could be, I'll look into it
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

Posted by "ocket8888 (via GitHub)" <gi...@apache.org>.
ocket8888 commented on PR #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894#issuecomment-1906771168

   Reverting something experimental because it affects the ability to build distributables for something that isn't distributed (because it's experimental) seems a bit extreme.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

Posted by "ocket8888 (via GitHub)" <gi...@apache.org>.
ocket8888 commented on code in PR #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894#discussion_r1447731377


##########
.github/actions/tpv2-integration-tests/entrypoint.sh:
##########


Review Comment:
   ah, yeah, it's still trying to run the server. Curious that doesn't crash the script, but whatever. There shouldn't need to be any TPv2 server logs because it no longer needs to run the TPv2 server.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

Posted by "zrhoffman (via GitHub)" <gi...@apache.org>.
zrhoffman commented on PR #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894#issuecomment-1906716335

   Reverted due to breaking tests in 4b6af898d8, please re-PR with tests fixed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

Posted by "github-advanced-security[bot] (via GitHub)" <gi...@apache.org>.
github-advanced-security[bot] commented on code in PR #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894#discussion_r1443832198


##########
experimental/traffic-portal/cypress.config.ts:
##########
@@ -0,0 +1,403 @@
+/*
+* Licensed under the Apache License, Version 2.0 (the "License");
+* you may not use this file except in compliance with the License.
+* You may obtain a copy of the License at
+*
+*     http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+import { promises as fs } from "fs";
+import * as https from "https";
+
+import axios, { type AxiosError } from "axios";
+import { defineConfig } from "cypress";
+import type {
+	CDN,
+	LoginRequest,
+	ProfileType,
+	RequestASN,
+	RequestCacheGroup,
+	RequestCoordinate,
+	RequestDeliveryService,
+	RequestDivision,
+	RequestParameter,
+	RequestPhysicalLocation,
+	RequestProfile,
+	RequestRegion,
+	RequestRole,
+	RequestServer,
+	RequestServerCapability,
+	RequestStatus,
+	RequestSteeringTarget,
+	RequestTenant,
+	RequestType,
+	ResponseCacheGroup,
+	ResponseDeliveryService,
+	ResponseDivision,
+	ResponseParameter,
+	ResponsePhysicalLocation,
+	ResponseProfile,
+	ResponseRegion,
+	ResponseStatus,
+	TypeFromResponse
+} from "trafficops-types";
+
+import type { CreatedData } from "./cypress/support/testing.data";
+
+/**
+ * Creates mock data needed for E2E testing.
+ *
+ * Ideally this functionality would go in a different file, but for some reason
+ * Cypress gets very upset if you import anything but a type from any TypeScript
+ * file in this config file.
+ *
+ * @param toURL The URL of a Traffic Ops instance e.g. 'https://traffic.ops/'.
+ * @param apiVersion The version of the API to use e.g. '4.1'.
+ * @param adminUser The username of the 'admin' Role user that will be used
+ * to set up testing data.
+ * @param adminPass The password of the 'admin' Role user that will be used to
+ * set up testing data.
+ * @returns The data that was created and the unique string that was appended to
+ * all names that are required to be unique.
+ */
+async function createData(toURL: string, apiVersion: string, adminUser: string, adminPass: string): Promise<CreatedData> {
+	const apiUrl = `${toURL}/api/${apiVersion}`;
+	const client = axios.create({
+		httpsAgent: new https.Agent({
+			rejectUnauthorized: false

Review Comment:
   ## Disabling certificate validation
   
   Disabling certificate validation is strongly discouraged.
   
   [Show more details](https://github.com/apache/trafficcontrol/security/code-scanning/314)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

Posted by "ocket8888 (via GitHub)" <gi...@apache.org>.
ocket8888 commented on PR #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894#issuecomment-1906683051

   that's interesting. It shouldn't be attempting to compile or run the e2e tests to build the RPM.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

Posted by "zrhoffman (via GitHub)" <gi...@apache.org>.
zrhoffman commented on PR #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894#issuecomment-1906502587

   This is making the TPv2 RPM fail to build
   
   From the PR of #7922:
   
   > npm ERR! Cypress cannot write to the cache directory due to file permissions


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

Posted by "ocket8888 (via GitHub)" <gi...@apache.org>.
ocket8888 commented on PR #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894#issuecomment-1906820403

   > - _"It should not be built when someone just runs `./pkg` without specifying optional builds with `-o`, and"_
   
   That should definitely never have been true. That's a bug in `pkg`, far as I'm concerned.
   
   > - _"It should not be included in the CDN-in-a-Box CI GHA workflow."_
   
   Also should never have been true, but additionally: why is a test that depends on and explicitly uses something not run when that thing changes? Two bugs in the GHAs.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894#issuecomment-1879790020

   ## [Codecov](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   All modified and coverable lines are covered by tests :white_check_mark:
   > Comparison is base [(`a28af5a`)](https://app.codecov.io/gh/apache/trafficcontrol/commit/a28af5a0e37fb8734a1e9caf7b342a7758d1d6a7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 31.80% compared to head [(`1d9b149`)](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 74.35%.
   
   
   <details><summary>Additional details and impacted files</summary>
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master    #7894       +/-   ##
   =============================================
   + Coverage     31.80%   74.35%   +42.55%     
   =============================================
     Files           719      117      -602     
     Lines         82819     5374    -77445     
     Branches        970      880       -90     
   =============================================
   - Hits          26340     3996    -22344     
   + Misses        54320     1310    -53010     
   + Partials       2159       68     -2091     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [golib_unit](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [grove_unit](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [t3c_generate_unit](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [t3c_unit](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [traffic_monitor_unit](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [traffic_ops_integration](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [traffic_ops_unit](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [traffic_portal_v2](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `74.35% <ø> (ø)` | |
   | [traffic_router_unit](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [traffic_stats_unit](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [unit_tests](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `74.35% <ø> (+45.23%)` | :arrow_up: |
   | [v3](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [v4](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [v5](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/trafficcontrol/pull/7894?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

Posted by "shamrickus (via GitHub)" <gi...@apache.org>.
shamrickus commented on code in PR #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894#discussion_r1443906012


##########
.github/actions/tpv2-integration-tests/entrypoint.sh:
##########


Review Comment:
   Not sure if it's related to your changes but when I download the GHA artifact for the end-to-end tests, the TP log shows an error `Error: Cannot find module '/home/runner/work/trafficcontrol/trafficcontrol/experimental/traffic-portal/dist/traffic-portal/server/main.js'`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

Posted by "shamrickus (via GitHub)" <gi...@apache.org>.
shamrickus commented on code in PR #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894#discussion_r1443905646


##########
experimental/traffic-portal/cypress.config.ts:
##########


Review Comment:
   I think it would make more sense for this to be in the `cypress` folder, the root folder is already pretty crowded.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

Posted by "shamrickus (via GitHub)" <gi...@apache.org>.
shamrickus merged PR #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Switch TPv2 E2E tests to Cypress [trafficcontrol]

Posted by "zrhoffman (via GitHub)" <gi...@apache.org>.
zrhoffman commented on PR #7894:
URL: https://github.com/apache/trafficcontrol/pull/7894#issuecomment-1906840725

   AFAIAC, the bug that lead to the revert is that TPv2 is included in the *CDN-in-a-Box CI* GHA workflow but its paths don't trigger it. If the *CDN-in-a-Box CI* GHA had failed (due to the breaking changes in #7894), it would not have gotten merged and would not need to be reverted.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org