You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/10/28 08:01:30 UTC
[GitHub] [druid] tejaswini-imply opened a new pull request, #13270: Adds license and security vulnerabilities checks for Hadoop3 build
tejaswini-imply opened a new pull request, #13270:
URL: https://github.com/apache/druid/pull/13270
Hadoop 2 often causes red security scans on imply bundles because of the dependencies it brings. We want to move away from Hadoop 2 and have to provide Hadoop 3 distribution available.
New stages are added in the pipeline to make the license and security-vulnerabilities checks that run with Hadoop2 also run with the Hadoop3 build.
P.S. Not updating Hadoop3 IT tests pipelines since Hadoop tests are not running on Travis currently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] tejaswini-imply commented on a diff in pull request #13270: Adds license and security vulnerabilities checks for Hadoop3 build
Posted by GitBox <gi...@apache.org>.
tejaswini-imply commented on code in PR #13270:
URL: https://github.com/apache/druid/pull/13270#discussion_r1015442062
##########
.travis.yml:
##########
@@ -142,27 +151,10 @@ jobs:
" && false; }
- - name: "analyze hadoop 3 dependencies"
- script: |-
- MAVEN_OPTS='-Xmx3000m' ${MVN} ${MAVEN_SKIP} dependency:analyze -DoutputXML=true -DignoreNonCompile=true -DfailOnWarning=true -Phadoop3 || { echo "
-
- The dependency analysis has found a dependency that is either:
-
- 1) Used and undeclared: These are available as a transitive dependency but should be explicitly
- added to the POM to ensure the dependency version. The XML to add the dependencies to the POM is
- shown above.
-
- 2) Unused and declared: These are not needed and removing them from the POM will speed up the build
- and reduce the artifact size. The dependencies to remove are shown above.
-
- If there are false positive dependency analysis warnings, they can be suppressed:
- https://maven.apache.org/plugins/maven-dependency-plugin/analyze-mojo.html#usedDependencies
- https://maven.apache.org/plugins/maven-dependency-plugin/examples/exclude-dependencies-from-dependency-analysis.html
-
- For more information, refer to:
- https://maven.apache.org/plugins/maven-dependency-plugin/analyze-mojo.html
-
- " && false; }
+ - <<: *analyze_dependencies
+ name: "analyze hadoop 3 dependencies"
Review Comment:
I can also add analyze dependencies, analyze hadoop 3 dependencies to check_test_suite.py and then revert it back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] cryptoe commented on a diff in pull request #13270: Adds license and security vulnerabilities checks for Hadoop3 build
Posted by GitBox <gi...@apache.org>.
cryptoe commented on code in PR #13270:
URL: https://github.com/apache/druid/pull/13270#discussion_r1015233502
##########
.travis.yml:
##########
@@ -142,27 +151,10 @@ jobs:
" && false; }
- - name: "analyze hadoop 3 dependencies"
- script: |-
- MAVEN_OPTS='-Xmx3000m' ${MVN} ${MAVEN_SKIP} dependency:analyze -DoutputXML=true -DignoreNonCompile=true -DfailOnWarning=true -Phadoop3 || { echo "
-
- The dependency analysis has found a dependency that is either:
-
- 1) Used and undeclared: These are available as a transitive dependency but should be explicitly
- added to the POM to ensure the dependency version. The XML to add the dependencies to the POM is
- shown above.
-
- 2) Unused and declared: These are not needed and removing them from the POM will speed up the build
- and reduce the artifact size. The dependencies to remove are shown above.
-
- If there are false positive dependency analysis warnings, they can be suppressed:
- https://maven.apache.org/plugins/maven-dependency-plugin/analyze-mojo.html#usedDependencies
- https://maven.apache.org/plugins/maven-dependency-plugin/examples/exclude-dependencies-from-dependency-analysis.html
-
- For more information, refer to:
- https://maven.apache.org/plugins/maven-dependency-plugin/analyze-mojo.html
-
- " && false; }
+ - <<: *analyze_dependencies
+ name: "analyze hadoop 3 dependencies"
Review Comment:
Can we introduce a dummy code change and role it back to see if `analyze hadoop 3 dependencies` works.
We can roll it back prior to merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] abhishekagarwal87 commented on a diff in pull request #13270: Adds license and security vulnerabilities checks for Hadoop3 build
Posted by GitBox <gi...@apache.org>.
abhishekagarwal87 commented on code in PR #13270:
URL: https://github.com/apache/druid/pull/13270#discussion_r1015240718
##########
.travis.yml:
##########
@@ -142,27 +151,10 @@ jobs:
" && false; }
- - name: "analyze hadoop 3 dependencies"
- script: |-
- MAVEN_OPTS='-Xmx3000m' ${MVN} ${MAVEN_SKIP} dependency:analyze -DoutputXML=true -DignoreNonCompile=true -DfailOnWarning=true -Phadoop3 || { echo "
-
- The dependency analysis has found a dependency that is either:
-
- 1) Used and undeclared: These are available as a transitive dependency but should be explicitly
- added to the POM to ensure the dependency version. The XML to add the dependencies to the POM is
- shown above.
-
- 2) Unused and declared: These are not needed and removing them from the POM will speed up the build
- and reduce the artifact size. The dependencies to remove are shown above.
-
- If there are false positive dependency analysis warnings, they can be suppressed:
- https://maven.apache.org/plugins/maven-dependency-plugin/analyze-mojo.html#usedDependencies
- https://maven.apache.org/plugins/maven-dependency-plugin/examples/exclude-dependencies-from-dependency-analysis.html
-
- For more information, refer to:
- https://maven.apache.org/plugins/maven-dependency-plugin/analyze-mojo.html
-
- " && false; }
+ - <<: *analyze_dependencies
+ name: "analyze hadoop 3 dependencies"
Review Comment:
https://app.travis-ci.com/github/apache/druid/jobs/587017381 - does this work?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] tejaswini-imply commented on pull request #13270: Adds license and security vulnerabilities checks for Hadoop3 build
Posted by GitBox <gi...@apache.org>.
tejaswini-imply commented on PR #13270:
URL: https://github.com/apache/druid/pull/13270#issuecomment-1308113133
@cryptoe Could you please review this now, analyze dependencies and analyze hadoop3 dependencies seem to be working as expected. Previously a new line crept into the mvn command.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] abhishekagarwal87 merged pull request #13270: Adds license and security vulnerabilities checks for Hadoop3 build
Posted by GitBox <gi...@apache.org>.
abhishekagarwal87 merged PR #13270:
URL: https://github.com/apache/druid/pull/13270
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org