You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/12/08 07:42:58 UTC
[GitHub] [spark] zhouyifan279 opened a new pull request, #38978: [SPARK-39948] Exclude hive-vector-code-gen dependency to solve CVE-2020-13936
zhouyifan279 opened a new pull request, #38978:
URL: https://github.com/apache/spark/pull/38978
### What changes were proposed in this pull request?
Remove hive-vector-code-gen and its dependent jars from spark distribution
### Why are the changes needed?
hive-vector-code-gen is not used in spark but introduced org.apache.velocity:velocity:velocity-1.5.jar, which has been reported with vulnerability CVE-2020-13936.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Passed current test cases
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on PR #38978:
URL: https://github.com/apache/spark/pull/38978#issuecomment-1344111830
Merged to master branch for Apache Spark 3.4.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhouyifan279 commented on pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency
Posted by GitBox <gi...@apache.org>.
zhouyifan279 commented on PR #38978:
URL: https://github.com/apache/spark/pull/38978#issuecomment-1344154603
> I found your JIRA ID, `zhouyifan279`, and added it to Apache Spark contributor group.
>
> Also, from the commit log, I found the following three JIRAs and assigned them to you.
>
> * [SPARK-39948](https://issues.apache.org/jira/browse/SPARK-39948)
> * [SPARK-8731](https://issues.apache.org/jira/browse/SPARK-8731)
> * [SPARK-37863](https://issues.apache.org/jira/browse/SPARK-37863)
>
> Thank you for your contribution, @zhouyifan279 .
Thanks. It's my pleasure to do something for the Spark community.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhouyifan279 commented on a diff in pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency
Posted by GitBox <gi...@apache.org>.
zhouyifan279 commented on code in PR #38978:
URL: https://github.com/apache/spark/pull/38978#discussion_r1044232727
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala:
##########
@@ -117,7 +118,8 @@ package object client {
"org.apache.derby:derby:10.14.1.0"),
exclusions = Seq("org.apache.calcite:calcite-druid",
"org.apache.curator:*",
- "org.pentaho:pentaho-aggdesigner-algorithm"))
+ "org.pentaho:pentaho-aggdesigner-algorithm",
+ "org.apache.hive:hive-vector-code-gen"))
Review Comment:
@sunchao According to this [commit](https://github.com/apache/hive/commit/1f1e91aa02d726613a364678288caa8b252d8bd6) in Hive Project, hive-vector-code-gen is added as hive-exec dependency since hive-2.3.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a diff in pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on code in PR #38978:
URL: https://github.com/apache/spark/pull/38978#discussion_r1044291116
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala:
##########
@@ -117,7 +118,8 @@ package object client {
"org.apache.derby:derby:10.14.1.0"),
exclusions = Seq("org.apache.calcite:calcite-druid",
"org.apache.curator:*",
- "org.pentaho:pentaho-aggdesigner-algorithm"))
+ "org.pentaho:pentaho-aggdesigner-algorithm",
+ "org.apache.hive:hive-vector-code-gen"))
Review Comment:
Thank you for confirming, @zhouyifan279 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhouyifan279 commented on pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency to solve CVE-2020-13936
Posted by GitBox <gi...@apache.org>.
zhouyifan279 commented on PR #38978:
URL: https://github.com/apache/spark/pull/38978#issuecomment-1342407603
> BTW, @zhouyifan279 . The GitHub Action is not triggerred at your repository still. 
@dongjoon-hyun thanks for your reminding. Several workflows were triggered by previous commits. I have cancelled them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #38978: [SPARK-39948] Exclude hive-vector-code-gen dependency to solve CVE-2020-13936
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #38978:
URL: https://github.com/apache/spark/pull/38978#issuecomment-1342249588
cc @sunchao FYI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhouyifan279 commented on pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency
Posted by GitBox <gi...@apache.org>.
zhouyifan279 commented on PR #38978:
URL: https://github.com/apache/spark/pull/38978#issuecomment-1344145782
> What is your JIRA ID, @zhouyifan279 ?
I reused this issue: https://issues.apache.org/jira/browse/SPARK-39948
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency to solve CVE-2020-13936
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on PR #38978:
URL: https://github.com/apache/spark/pull/38978#issuecomment-1343174647
Yes, right, @srowen . Apache Spark is not affected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhouyifan279 commented on pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency to solve CVE-2020-13936
Posted by GitBox <gi...@apache.org>.
zhouyifan279 commented on PR #38978:
URL: https://github.com/apache/spark/pull/38978#issuecomment-1342271155
> @zhouyifan279 Title should be `[SPARK-39948][BUILD] Exclude...`
Thanks, corrected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a diff in pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on code in PR #38978:
URL: https://github.com/apache/spark/pull/38978#discussion_r1043740105
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala:
##########
@@ -117,7 +118,8 @@ package object client {
"org.apache.derby:derby:10.14.1.0"),
exclusions = Seq("org.apache.calcite:calcite-druid",
"org.apache.curator:*",
- "org.pentaho:pentaho-aggdesigner-algorithm"))
+ "org.pentaho:pentaho-aggdesigner-algorithm",
+ "org.apache.hive:hive-vector-code-gen"))
Review Comment:
@zhouyifan279 Could you answer @sunchao 's question?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency
URL: https://github.com/apache/spark/pull/38978
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on a diff in pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency to solve CVE-2020-13936
Posted by GitBox <gi...@apache.org>.
sunchao commented on code in PR #38978:
URL: https://github.com/apache/spark/pull/38978#discussion_r1043642302
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/package.scala:
##########
@@ -117,7 +118,8 @@ package object client {
"org.apache.derby:derby:10.14.1.0"),
exclusions = Seq("org.apache.calcite:calcite-druid",
"org.apache.curator:*",
- "org.pentaho:pentaho-aggdesigner-algorithm"))
+ "org.pentaho:pentaho-aggdesigner-algorithm",
+ "org.apache.hive:hive-vector-code-gen"))
Review Comment:
do other Hive versions like `v2_2` have the same issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #38978: [SPARK-39948] Exclude hive-vector-code-gen dependency to solve CVE-2020-13936
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #38978:
URL: https://github.com/apache/spark/pull/38978#issuecomment-1342250611
@zhouyifan279 Title should be `[SPARK-39948][BUILD] Exclude...`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on PR #38978:
URL: https://github.com/apache/spark/pull/38978#issuecomment-1343175344
I revised the PR title and description by removing CVE info, @srowen .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency to solve CVE-2020-13936
Posted by GitBox <gi...@apache.org>.
srowen commented on PR #38978:
URL: https://github.com/apache/spark/pull/38978#issuecomment-1342825389
If we can exclude this code, then, the CVE doesn't affect Spark to begin with right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency to solve CVE-2020-13936
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on PR #38978:
URL: https://github.com/apache/spark/pull/38978#issuecomment-1342573334
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on PR #38978:
URL: https://github.com/apache/spark/pull/38978#issuecomment-1344113277
What is your JIRA ID, @zhouyifan279 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #38978: [SPARK-39948][BUILD] Exclude hive-vector-code-gen dependency
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on PR #38978:
URL: https://github.com/apache/spark/pull/38978#issuecomment-1344117486
I found your JIRA ID, `zhouyifan279`, and added it to Apache Spark contributor group.
Also, from the commit log, I found the following three JIRAs and assigned them to you.
- SPARK-39948
- SPARK-8731
- SPARK-37863
Thank you for your contribution, @zhouyifan279 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org