You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/11/23 06:57:56 UTC
[GitHub] [spark] AngersZhuuuu opened a new pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
AngersZhuuuu opened a new pull request #34689:
URL: https://github.com/apache/spark/pull/34689
### What changes were proposed in this pull request?
Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now.
In current project, deps's path is still hadoop-3.2, it's not correct.
### Why are the changes needed?
Upgrade hadoop profile
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Not need
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-977656998
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50039/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979146988
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145622/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979050964
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50094/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-976209156
**[Test build #145539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145539/testReport)** for PR 34689 at commit [`6a446eb`](https://github.com/apache/spark/commit/6a446eb47a7810b685b2e6a5adb9f074a3f1b844).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu edited a comment on pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu edited a comment on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979610978
> Yes, renaming profiles and release file names were discussed at https://mail-archives.apache.org/mod_mbox/spark-dev/202106.mbox/%3CCAF6oT1dOVc86zPFiCEcPtq6CEDknKxu6bzGPm5NZ%3Dms706kfxg%40mail.gmail.com%3E. I don't mind who works on that but let's make sure do it in one go.
Ah, I miss the email.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r757189605
##########
File path: dev/create-release/release-build.sh
##########
@@ -322,18 +322,18 @@ if [[ "$1" == "package" ]]; then
# 'python/pyspark/install.py' and 'python/docs/source/getting_started/install.rst'
Review comment:
So we can make `PYSPARK_HADOOP_VERSION ` as 3 but still install spark-3.2.0-bin-hadoop3.2?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978811339
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50080/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978839819
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50080/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978777539
**[Test build #145606 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145606/testReport)** for PR 34689 at commit [`3e132f3`](https://github.com/apache/spark/commit/3e132f3e2c9a5d9ffbaad75d3bf939f91f4ff43f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978926969
**[Test build #145622 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145622/testReport)** for PR 34689 at commit [`d3ad6e7`](https://github.com/apache/spark/commit/d3ad6e7b0ca2adff5558dd5aa370ecec5cdbf1a0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979534061
Guys, im gonna revert this - this fix is incomplete, and we should keep the name same as `hadoop3` instead of `hadoop3.3`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979544279
cc @dongjoon-hyun too FYI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978839849
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50080/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978840035
how about the file name of the final release binaries?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979600444
Yes, renaming profiles and release file names were discussed at https://mail-archives.apache.org/mod_mbox/spark-dev/202106.mbox/browser. I don't mind who works on that but let's make sure do it in one go.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r757191223
##########
File path: dev/create-release/release-build.sh
##########
@@ -322,18 +322,18 @@ if [[ "$1" == "package" ]]; then
# 'python/pyspark/install.py' and 'python/docs/source/getting_started/install.rst'
Review comment:
Once 3.3 is released, it should be `PYSPARK_HADOOP_VERSION=3`. We should document that. This part I can help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r755676187
##########
File path: hadoop-cloud/pom.xml
##########
@@ -201,7 +201,7 @@
enables store-specific committers.
-->
<profile>
- <id>hadoop-3.2</id>
+ <id>hadoop-3.3</id>
Review comment:
> Right. I don't think we would build with -Dhadoop.version=3.2.x anyway, but someone could try it. It wouldn't mean we need a minor-version-specific Hadoop profile
So make sure, I should continue this pr with profile `hadoop-3`, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-977615470
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50039/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r755675407
##########
File path: hadoop-cloud/pom.xml
##########
@@ -201,7 +201,7 @@
enables store-specific committers.
-->
<profile>
- <id>hadoop-3.2</id>
+ <id>hadoop-3.3</id>
Review comment:
Right. I don't think we would build with -Dhadoop.version=3.2.x anyway, but someone could try it. It wouldn't mean we need a minor-version-specific Hadoop profile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-977667780
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50039/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978869261
> how about the file name of the final release binaries?
Also ping @vanzin , since this part of code is changed by him.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979645710
> > So what should I do next? raise a new pr to change all to hadoop3? or left it to @sunchao
>
> Feel free to take over @AngersZhuuuu . I can help on reviewing.
Thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979595176
> Guys, im gonna revert this - this fix is incomplete, and we should keep the name same as `hadoop3` instead of `hadoop3.3`.
So what should I do next? raise a new pr to change all to hadoop3? or left it to @sunchao
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #34689:
URL: https://github.com/apache/spark/pull/34689
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-976302458
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50011/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-977587000
**[Test build #145567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145567/testReport)** for PR 34689 at commit [`7d8e8b4`](https://github.com/apache/spark/commit/7d8e8b415706370afa561d6af1a6bbcc808dda13).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r755681520
##########
File path: hadoop-cloud/pom.xml
##########
@@ -201,7 +201,7 @@
enables store-specific committers.
-->
<profile>
- <id>hadoop-3.2</id>
+ <id>hadoop-3.3</id>
Review comment:
That's my suggestion at least, and it would solve the version confusion we observed in the 3.2.0 release. I don't see a downside.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978839849
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50080/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r757187812
##########
File path: dev/create-release/release-build.sh
##########
@@ -322,18 +322,18 @@ if [[ "$1" == "package" ]]; then
# 'python/pyspark/install.py' and 'python/docs/source/getting_started/install.rst'
Review comment:
It has to be renamed to hadoop-3, and should fix the documentation at https://spark.apache.org/docs/latest/api/python/getting_started/install.html#using-pypi accordingly. Let's don't make divergence on profile name alone. This kind of change requires to change infra in all forks, other companies, and how we release. I would like to avoid make incremental changes on this kind of item.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r757187015
##########
File path: dev/create-release/release-build.sh
##########
@@ -322,18 +322,18 @@ if [[ "$1" == "package" ]]; then
# 'python/pyspark/install.py' and 'python/docs/source/getting_started/install.rst'
Review comment:
> you missed this comment..
I know this, but since the package is not correct, change `install.py` will failed since there's no related package on it. Right?
This can be updated later after the next release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978902560
LGTM, please fix the conflicts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-976236833
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50011/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r755673891
##########
File path: hadoop-cloud/pom.xml
##########
@@ -201,7 +201,7 @@
enables store-specific committers.
-->
<profile>
- <id>hadoop-3.2</id>
+ <id>hadoop-3.3</id>
Review comment:
> I see, are we going to need a separate Hadoop 3.2 build at all going forward? If not, I think it makes sense to call it hadoop-3.
In current website, there are only binary file of `hadoop-3.3` & `hadoop-2.7`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978816412
Also ping @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978876318
**[Test build #145606 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145606/testReport)** for PR 34689 at commit [`3e132f3`](https://github.com/apache/spark/commit/3e132f3e2c9a5d9ffbaad75d3bf939f91f4ff43f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979032217
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50094/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r755655665
##########
File path: hadoop-cloud/pom.xml
##########
@@ -201,7 +201,7 @@
enables store-specific committers.
-->
<profile>
- <id>hadoop-3.2</id>
+ <id>hadoop-3.3</id>
Review comment:
> How about avoiding this going forward with `hadoop-3`? I doubt we're going to publish builds separately for two different Hadoop 3 versions in the future.
You mean here we can make profile id as `hadoop-3`, then we can build spark release with `-Dhadoop.version=3.2.x` and `-Dhadoop.version=3.3.x`?
I think it's ok if we need to maintain two version of hadoop3. Anyway better than `hadoop-3.2`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978974185
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50094/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r757166139
##########
File path: dev/create-release/release-build.sh
##########
@@ -322,18 +322,18 @@ if [[ "$1" == "package" ]]; then
# 'python/pyspark/install.py' and 'python/docs/source/getting_started/install.rst'
# if you're changing them.
declare -A BINARY_PKGS_ARGS
- BINARY_PKGS_ARGS["hadoop3.2"]="-Phadoop-3.2 $HIVE_PROFILES"
+ BINARY_PKGS_ARGS["hadoop3.3"]="-Phadoop-3 $HIVE_PROFILES"
Review comment:
Why is this `hadoop3.3` while the profile is `hadoop-3`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r757166768
##########
File path: dev/create-release/release-build.sh
##########
@@ -322,18 +322,18 @@ if [[ "$1" == "package" ]]; then
# 'python/pyspark/install.py' and 'python/docs/source/getting_started/install.rst'
Review comment:
you missed this comment..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r756572505
##########
File path: hadoop-cloud/pom.xml
##########
@@ -201,7 +201,7 @@
enables store-specific committers.
-->
<profile>
- <id>hadoop-3.2</id>
+ <id>hadoop-3.3</id>
Review comment:
+1 to `hadoop-3`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r755239983
##########
File path: hadoop-cloud/pom.xml
##########
@@ -201,7 +201,7 @@
enables store-specific committers.
-->
<profile>
- <id>hadoop-3.2</id>
+ <id>hadoop-3.3</id>
Review comment:
Same comment as hive-2.3, but, here may be better to change the name to make sure callers are clear it's not Hadoop 3.2.
How about avoiding this going forward with `hadoop-3`? I doubt we're going to publish builds separately for two different Hadoop 3 versions in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-976209156
**[Test build #145539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145539/testReport)** for PR 34689 at commit [`6a446eb`](https://github.com/apache/spark/commit/6a446eb47a7810b685b2e6a5adb9f074a3f1b844).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r755673891
##########
File path: hadoop-cloud/pom.xml
##########
@@ -201,7 +201,7 @@
enables store-specific committers.
-->
<profile>
- <id>hadoop-3.2</id>
+ <id>hadoop-3.3</id>
Review comment:
> I see, are we going to need a separate Hadoop 3.2 build at all going forward? If not, I think it makes sense to call it hadoop-3.
In current website, there are only binary file of `hadoop-3.3` & `hadoop-2.7`, make it as `hadoop-3` seems don't need us to always change this for a long time. ==
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-977671754
**[Test build #145567 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145567/testReport)** for PR 34689 at commit [`7d8e8b4`](https://github.com/apache/spark/commit/7d8e8b415706370afa561d6af1a6bbcc808dda13).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r757187015
##########
File path: dev/create-release/release-build.sh
##########
@@ -322,18 +322,18 @@ if [[ "$1" == "package" ]]; then
# 'python/pyspark/install.py' and 'python/docs/source/getting_started/install.rst'
Review comment:
> you missed this comment..
I know this, but since the package is not correct, change `install.py` will failed since there's no related package on it. Right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
sunchao commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979645285
> So what should I do next? raise a new pr to change all to hadoop3? or left it to @sunchao
Feel free to take over @AngersZhuuuu . I can help on reviewing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979050964
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50094/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979171742
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r757166689
##########
File path: dev/test-dependencies.sh
##########
@@ -35,7 +35,7 @@ HADOOP_MODULE_PROFILES="-Phive-thriftserver -Pmesos -Pkubernetes -Pyarn -Phive \
MVN="build/mvn"
HADOOP_HIVE_PROFILES=(
hadoop-2.7-hive-2.3
- hadoop-3.2-hive-2.3
+ hadoop-3.3-hive-2.3
Review comment:
why is this again 3.3 ..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979146988
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145622/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon edited a comment on pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
HyukjinKwon edited a comment on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979600444
Yes, renaming profiles and release file names were discussed at https://mail-archives.apache.org/mod_mbox/spark-dev/202106.mbox/%3CCAF6oT1dOVc86zPFiCEcPtq6CEDknKxu6bzGPm5NZ%3Dms706kfxg%40mail.gmail.com%3E. I don't mind who works on that but let's make sure do it in one go.
Namely addressing these JIRAs together: SPARK-37068 and SPARK-37132
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-976302494
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50011/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-976302494
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50011/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-976351446
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145539/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-976351446
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145539/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-977667780
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50039/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-977673273
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145567/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-977673273
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145567/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978904604
> LGTM, please fix the conflicts.
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r756574300
##########
File path: hadoop-cloud/pom.xml
##########
@@ -201,7 +201,7 @@
enables store-specific committers.
-->
<profile>
- <id>hadoop-3.2</id>
Review comment:
Sorry I'm not very familiar with the maven profile. Are you saying `-Phadoop-3.2` can also match this profile?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978891395
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145606/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978867355
> how about the file name of the final release binaries?
In `make-distribution.sh` , package name is passed by option `--name`
```
if [ "$MAKE_TGZ" == "true" ]; then
TARDIR_NAME=spark-$VERSION-bin-$NAME
TARDIR="$SPARK_HOME/$TARDIR_NAME"
rm -rf "$TARDIR"
cp -r "$DISTDIR" "$TARDIR"
tar czf "spark-$VERSION-bin-$NAME.tgz" -C "$SPARK_HOME" "$TARDIR_NAME"
rm -rf "$TARDIR"
fi
```
In `release-build.sh`, it use parameter in `BINARY_PKGS_ARGS`
```
if [[ $PUBLISH_SCALA_2_12 = 1 ]]; then
echo "Packages to build: ${!BINARY_PKGS_ARGS[@]}"
for key in ${!BINARY_PKGS_ARGS[@]}; do
args=${BINARY_PKGS_ARGS[$key]}
extra=${BINARY_PKGS_EXTRA[$key]}
if ! make_binary_release "$key" "$SCALA_2_12_PROFILES $args" "$extra" "2.12"; then
error "Failed to build $key package. Check logs for details."
fi
done
fi
```
In this PR I have change the `BINARY_PKGS_ARGS `. So the release package name should be `hadoop3.3`
I have no committer authority, so can't do an end to end test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979578221
Thank you for pinging me, @HyukjinKwon .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979610978
> Yes, renaming profiles and release file names were discussed at https://mail-archives.apache.org/mod_mbox/spark-dev/202106.mbox/%3CCAF6oT1dOVc86zPFiCEcPtq6CEDknKxu6bzGPm5NZ%3Dms706kfxg%40mail.gmail.com%3E. I don't mind who works on that but let's make sure do it in one go.
Ah, I miss the email. I will rase a new one. Thanks for your review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978891395
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145606/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979133311
**[Test build #145622 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145622/testReport)** for PR 34689 at commit [`d3ad6e7`](https://github.com/apache/spark/commit/d3ad6e7b0ca2adff5558dd5aa370ecec5cdbf1a0).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class UnclosedCommentProcessor(`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978926969
**[Test build #145622 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145622/testReport)** for PR 34689 at commit [`d3ad6e7`](https://github.com/apache/spark/commit/d3ad6e7b0ca2adff5558dd5aa370ecec5cdbf1a0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r757166337
##########
File path: hadoop-cloud/pom.xml
##########
@@ -201,7 +201,7 @@
enables store-specific committers.
-->
<profile>
- <id>hadoop-3.2</id>
+ <id>hadoop-3</id>
Review comment:
Hey, there was already WIP PT to replace this, see also https://github.com/apache/spark/pull/30891 cc @sunchao
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r757187262
##########
File path: dev/create-release/release-build.sh
##########
@@ -322,18 +322,18 @@ if [[ "$1" == "package" ]]; then
# 'python/pyspark/install.py' and 'python/docs/source/getting_started/install.rst'
# if you're changing them.
declare -A BINARY_PKGS_ARGS
- BINARY_PKGS_ARGS["hadoop3.2"]="-Phadoop-3.2 $HIVE_PROFILES"
+ BINARY_PKGS_ARGS["hadoop3.3"]="-Phadoop-3 $HIVE_PROFILES"
Review comment:
Keep the package name as hadoop3.3 to let user know the download version is hadoop3.3.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-978777539
**[Test build #145606 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145606/testReport)** for PR 34689 at commit [`3e132f3`](https://github.com/apache/spark/commit/3e132f3e2c9a5d9ffbaad75d3bf939f91f4ff43f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r757166337
##########
File path: hadoop-cloud/pom.xml
##########
@@ -201,7 +201,7 @@
enables store-specific committers.
-->
<profile>
- <id>hadoop-3.2</id>
+ <id>hadoop-3</id>
Review comment:
Hey, there was already WIP PR to replace this, see also https://github.com/apache/spark/pull/30891 cc @sunchao
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #34689:
URL: https://github.com/apache/spark/pull/34689#discussion_r755671589
##########
File path: hadoop-cloud/pom.xml
##########
@@ -201,7 +201,7 @@
enables store-specific committers.
-->
<profile>
- <id>hadoop-3.2</id>
+ <id>hadoop-3.3</id>
Review comment:
I see, are we going to need a separate Hadoop 3.2 build at all going forward? If not, I think it makes sense to call it hadoop-3.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-977587000
**[Test build #145567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145567/testReport)** for PR 34689 at commit [`7d8e8b4`](https://github.com/apache/spark/commit/7d8e8b415706370afa561d6af1a6bbcc808dda13).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon edited a comment on pull request #34689: [SPARK-37445][BUILD] Rename the maven profile hadoop-3.2 to hadoop-3 and change GA test name to hadoop3.3
Posted by GitBox <gi...@apache.org>.
HyukjinKwon edited a comment on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-979600444
Yes, renaming profiles and release file names were discussed at https://mail-archives.apache.org/mod_mbox/spark-dev/202106.mbox/%3CCAF6oT1dOVc86zPFiCEcPtq6CEDknKxu6bzGPm5NZ%3Dms706kfxg%40mail.gmail.com%3E. I don't mind who works on that but let's make sure do it in one go.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34689: [SPARK-37445][BUILD] Upgrade hadoop profile to hadoop-3.3 since we support hadoop-3.3 as default now
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34689:
URL: https://github.com/apache/spark/pull/34689#issuecomment-976336510
**[Test build #145539 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145539/testReport)** for PR 34689 at commit [`6a446eb`](https://github.com/apache/spark/commit/6a446eb47a7810b685b2e6a5adb9f074a3f1b844).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org