You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2022/10/26 08:58:47 UTC
[GitHub] [hadoop] snvijaya opened a new pull request, #5080: HADOOP-17767. ABFS: Updates test scripts
snvijaya opened a new pull request, #5080:
URL: https://github.com/apache/hadoop/pull/5080
Current test run scripts need manual update across all combinations in runTests.sh for account name and is working off a single azure-auth-keys.xml file. While having to test across accounts that span various geo, the config file grows big and also needs a manual change for configs such as fs.contract.test.[abfs/abfss] which has to be uniquely set. To use the script across various combinations, dev to be aware of the names of all the combinations defined in runTests.sh as well.
This PR updates the test scripts to address above concerns. An option to delete old test containers on the account is also added. Sample run output:
`Choose action:
[Note - SET_ACTIVE_TEST_CONFIG will help activate the config for IDE/single test class runs]
1) SET_ACTIVE_TEST_CONFIG 4) SET_OR_CHANGE_TEST_ACCOUNT
2) RUN_TEST 5) PRINT_LOG4J_LOG_PATHS_FROM_LAST_RUN
3) CLEAN_UP_OLD_TEST_CONTAINERS
#? 2
Enter parallel test run process count [default - 8]: 4
Set the active test combination to run the action:
1) HNS-OAuth 3) nonHNS-SharedKey 5) AllCombinationsTestRun
2) HNS-SharedKey 4) AppendBlob-HNS-OAuth 6) Quit
#? 1
Combination specific property setting: [ key=fs.azure.account.auth.type , value=OAuth ]
Activated [src/test/resources/abfs-combination-test-configs.xml] - for account: snvijayacontracttest for combination HNS-OAuth
Running test for combination HNS-OAuth on account snvijayacontracttest [ProcessCount=4]
Test run report can be seen in dev-support/testlogs/2022-10-07_05-23-22/Test-Logs-HNS-OAuth.txt
`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5080: HADOOP-17767. ABFS: Updates test scripts
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5080:
URL: https://github.com/apache/hadoop/pull/5080#issuecomment-1291817766
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 15m 45s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available. |
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 4 new or modified test files. |
|||| _ branch-3.3 Compile Tests _ |
| -1 :x: | mvninstall | 51m 26s | [/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/artifact/out/branch-mvninstall-root.txt) | root in branch-3.3 failed. |
| -1 :x: | compile | 0m 43s | [/branch-compile-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/artifact/out/branch-compile-hadoop-tools_hadoop-azure.txt) | hadoop-azure in branch-3.3 failed. |
| -0 :warning: | checkstyle | 0m 42s | [/buildtool-branch-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/artifact/out/buildtool-branch-checkstyle-hadoop-tools_hadoop-azure.txt) | The patch fails to run checkstyle in hadoop-azure |
| -1 :x: | mvnsite | 0m 41s | [/branch-mvnsite-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/artifact/out/branch-mvnsite-hadoop-tools_hadoop-azure.txt) | hadoop-azure in branch-3.3 failed. |
| -1 :x: | javadoc | 0m 43s | [/branch-javadoc-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/artifact/out/branch-javadoc-hadoop-tools_hadoop-azure.txt) | hadoop-azure in branch-3.3 failed. |
| -1 :x: | spotbugs | 0m 45s | [/branch-spotbugs-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/artifact/out/branch-spotbugs-hadoop-tools_hadoop-azure.txt) | hadoop-azure in branch-3.3 failed. |
| +1 :green_heart: | shadedclient | 3m 3s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 30s | [/patch-mvninstall-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/artifact/out/patch-mvninstall-hadoop-tools_hadoop-azure.txt) | hadoop-azure in the patch failed. |
| -1 :x: | compile | 0m 30s | [/patch-compile-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/artifact/out/patch-compile-hadoop-tools_hadoop-azure.txt) | hadoop-azure in the patch failed. |
| -1 :x: | javac | 0m 30s | [/patch-compile-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/artifact/out/patch-compile-hadoop-tools_hadoop-azure.txt) | hadoop-azure in the patch failed. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 29s | [/buildtool-patch-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/artifact/out/buildtool-patch-checkstyle-hadoop-tools_hadoop-azure.txt) | The patch fails to run checkstyle in hadoop-azure |
| -1 :x: | mvnsite | 0m 28s | [/patch-mvnsite-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/artifact/out/patch-mvnsite-hadoop-tools_hadoop-azure.txt) | hadoop-azure in the patch failed. |
| +1 :green_heart: | shellcheck | 0m 1s | | No new issues. |
| -1 :x: | javadoc | 0m 29s | [/patch-javadoc-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/artifact/out/patch-javadoc-hadoop-tools_hadoop-azure.txt) | hadoop-azure in the patch failed. |
| -1 :x: | spotbugs | 0m 32s | [/patch-spotbugs-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/artifact/out/patch-spotbugs-hadoop-tools_hadoop-azure.txt) | hadoop-azure in the patch failed. |
| +1 :green_heart: | shadedclient | 3m 7s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 0m 32s | [/patch-unit-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt) | hadoop-azure in the patch failed. |
| +0 :ok: | asflicense | 0m 32s | | ASF License check generated no output? |
| | | 81m 7s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5080 |
| Optional Tests | dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs markdownlint compile javac javadoc mvninstall shadedclient spotbugs checkstyle xmllint |
| uname | Linux fa790f884396 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / b1b6d3a344019b94fc71cbe595bda14b6fa8d74c |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~18.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/testReport/ |
| Max. process+thread count | 88 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5080/1/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org