You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/07/07 22:57:24 UTC
[GitHub] [spark] bharatviswa504 opened a new pull request #29030: SPARK-32093. Add hadoop-ozone-filesystem jar to ozone profile.
bharatviswa504 opened a new pull request #29030:
URL: https://github.com/apache/spark/pull/29030
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. If you want to add a new configuration, please read the guideline first for naming configurations in
'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
-->
### What changes were proposed in this pull request?
Add ozone profile, so that ozone file system jar is packaged in spark tarball.
### Why are the changes needed?
This change is added so that spark can include hadoop-ozone-filesystem jar, and spark can run on top of ozone out of the box.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Created a new profile. It is a build change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #29030: [WIP][SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #29030:
URL: https://github.com/apache/spark/pull/29030
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29030: [SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar.
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29030:
URL: https://github.com/apache/spark/pull/29030#discussion_r452317722
##########
File path: pom.xml
##########
@@ -202,6 +202,9 @@
<arrow.version>0.15.1</arrow.version>
<!-- org.fusesource.leveldbjni will be used except on arm64 platform. -->
<leveldbjni.group>org.fusesource.leveldbjni</leveldbjni.group>
+
+ <!-- ozone version -->
+ <ozone.version>0.5.0-beta</ozone.version>
Review comment:
What is the next `Ozone will be GA next release`? Could you close this and reopen after that with the official version?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #29030: [SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar to ozone profile.
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #29030:
URL: https://github.com/apache/spark/pull/29030#issuecomment-655770710
Why do we need this in Spark?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29030: SPARK-32093. Add hadoop-ozone-filesystem jar to ozone profile.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29030:
URL: https://github.com/apache/spark/pull/29030#issuecomment-655179673
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29030: SPARK-32093. Add hadoop-ozone-filesystem jar to ozone profile.
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29030:
URL: https://github.com/apache/spark/pull/29030#discussion_r451202038
##########
File path: pom.xml
##########
@@ -3059,13 +3062,28 @@
<curator.version>2.7.1</curator.version>
<commons-io.version>2.4</commons-io.version>
</properties>
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-ozone-filesystem-lib-legacy</artifactId>
+ <scope>runtime</scope>
+ <version>${ozone.version}</version>
+ </dependency>
+ </dependencies>
</profile>
<profile>
<id>hadoop-3.2</id>
<!-- Default hadoop profile. Uses global properties. -->
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-ozone-filesystem-lib-current</artifactId>
+ <scope>runtime</scope>
+ <version>${ozone.version}</version>
+ </dependency>
+ </dependencies>
</profile>
-
Review comment:
nit. Let's not touch this part.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29030: SPARK-32093. Add hadoop-ozone-filesystem jar to ozone profile.
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29030:
URL: https://github.com/apache/spark/pull/29030#discussion_r451201726
##########
File path: pom.xml
##########
@@ -202,6 +202,9 @@
<arrow.version>0.15.1</arrow.version>
<!-- org.fusesource.leveldbjni will be used except on arm64 platform. -->
<leveldbjni.group>org.fusesource.leveldbjni</leveldbjni.group>
+
+ <!-- ozone version -->
+ <ozone.version>0.5.0-beta</ozone.version>
Review comment:
Thank you for making a PR, but we usually don't add `-beta` dependency. Is this stable?
cc @srowen , @dbtsai and @gatorsmile
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] bharatviswa504 commented on a change in pull request #29030: [WIP][SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #29030:
URL: https://github.com/apache/spark/pull/29030#discussion_r452614277
##########
File path: pom.xml
##########
@@ -202,6 +202,9 @@
<arrow.version>0.15.1</arrow.version>
<!-- org.fusesource.leveldbjni will be used except on arm64 platform. -->
<leveldbjni.group>org.fusesource.leveldbjni</leveldbjni.group>
+
+ <!-- ozone version -->
+ <ozone.version>0.5.0-beta</ozone.version>
Review comment:
Ozone's next release is 0.6.0.
Mostly it will be released before August 2020.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #29030: [SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar to ozone profile.
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #29030:
URL: https://github.com/apache/spark/pull/29030#issuecomment-655777802
I also think like @srowen .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] bharatviswa504 commented on a change in pull request #29030: [SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar to ozone profile.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #29030:
URL: https://github.com/apache/spark/pull/29030#discussion_r451855945
##########
File path: pom.xml
##########
@@ -202,6 +202,9 @@
<arrow.version>0.15.1</arrow.version>
<!-- org.fusesource.leveldbjni will be used except on arm64 platform. -->
<leveldbjni.group>org.fusesource.leveldbjni</leveldbjni.group>
+
+ <!-- ozone version -->
+ <ozone.version>0.5.0-beta</ozone.version>
Review comment:
Ozone will be GA next release. Discussion is happening in the community around this topic.
Ozone has been scale tested with 1 billion objects and also it is tested with TPCSDS Hive and Spark.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29030: SPARK-32093. Add hadoop-ozone-filesystem jar to ozone profile.
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29030:
URL: https://github.com/apache/spark/pull/29030#discussion_r451201889
##########
File path: pom.xml
##########
@@ -3059,13 +3062,28 @@
<curator.version>2.7.1</curator.version>
<commons-io.version>2.4</commons-io.version>
</properties>
+ <dependencies>
Review comment:
indentation?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] bharatviswa504 commented on pull request #29030: [SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on pull request #29030:
URL: https://github.com/apache/spark/pull/29030#issuecomment-655802947
> Does it work if a user app just includes it?
Yes, and also it requires configuration required to connect to the ozone cluster. They can be included in core-site or in a new file ozone-site.xml.
> Can it go in hadoop-cloud if it must be packaged with Spark in some contexts -- why a new profile?
Initially, I have thought of a new profile, later used the existing Hadoop profile only.
I have updated the PR description to reflect the same.
> but what is Ozone, a file system from the Hadoop project, and who / how many people would need this?
Right now it is a sub-project of Hadoop.
Ozone is a scalable, redundant, and distributed object store for Hadoop, and also it provides Hadoop compatible FileSystem where spark/hive applications can run on it.
Link to Ozone website
https://hadoop.apache.org/ozone/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #29030: [SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar.
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #29030:
URL: https://github.com/apache/spark/pull/29030#issuecomment-655826115
OK, if it's something that logically belongs as part of the cluster so that not every app needs to include it, I can see trying to build it into a Spark distro. It wouldn't be enabled by default. It could go in hadoop-cloud though this isn't really to help use cloud object stores, though then again maybe this is meant to be a cloud-based object store in its own right?
In any event not sure it makes sense to add to the pom here if it's still pretty beta / dev? Anyone can add this to a downstream Spark build already.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] bharatviswa504 edited a comment on pull request #29030: [SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 edited a comment on pull request #29030:
URL: https://github.com/apache/spark/pull/29030#issuecomment-655802947
> Does it work if a user app just includes it?
Yes, and also it requires configuration required to connect to the ozone cluster. They can be included in core-site or in a new file ozone-site.xml.
> Can it go in hadoop-cloud if it must be packaged with Spark in some contexts -- why a new profile?
Initially, I have thought of a new profile, later used the existing Hadoop profile only.
I have updated the PR description to reflect the same.
> but what is Ozone, a file system from the Hadoop project, and who / how many people would need this?
Right now it is a sub-project of Hadoop.
Ozone is a scalable, redundant, and distributed object store for Hadoop, and also it provides Hadoop compatible FileSystem where spark/hive applications can run on it.
Link to Ozone website
https://hadoop.apache.org/ozone/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] bharatviswa504 edited a comment on pull request #29030: [SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar to ozone profile.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 edited a comment on pull request #29030:
URL: https://github.com/apache/spark/pull/29030#issuecomment-655787847
Thank You @dongjoon-hyun and @srowen for the review.
The reason for doing this is similar to the idea of spark packaging cloud connector jars, so that spark will be out of the box can work with them. (And the user does not require to copy jars to spark executor classpath)
Similar support we want to add for ozone also. Right now to make spark work with ozone, we are using a --files and then running spark jobs. If spark executor has ozone jar included, we shall not require this step.
The purpose of this Jira is to include an ozone file system jar in spark release tarball, in this way spark can out of box work with an ozone cluster when used with ozone paths.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] bharatviswa504 commented on pull request #29030: [WIP][SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on pull request #29030:
URL: https://github.com/apache/spark/pull/29030#issuecomment-656474666
> t not every app needs to include it, I can see trying to build it into a Spark distro. It wouldn't be enabled by default. It could go in hadoop-cloud though this isn't really to help use cloud object stores, though then again maybe this is meant to be a cloud-based object store in its own right?
> In any event not sure it makes sense to add to the pom here if it's still pretty beta / dev? Anyone can add this to a downstream Spark build already.
I will modify this PR, once ozone GA version is released.
This change is to make ozone support out of box with spark tarball distro. So, that spark distro's can contain required ozone jar. This is the main intention of this.
Ozone is an on-prem object store. So, I have added this main pom.xml, as hadoop-cloud is specifically for cloud stores.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29030: [WIP][SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar.
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29030:
URL: https://github.com/apache/spark/pull/29030#discussion_r452318686
##########
File path: pom.xml
##########
@@ -202,6 +202,9 @@
<arrow.version>0.15.1</arrow.version>
<!-- org.fusesource.leveldbjni will be used except on arm64 platform. -->
<leveldbjni.group>org.fusesource.leveldbjni</leveldbjni.group>
+
+ <!-- ozone version -->
+ <ozone.version>0.5.0-beta</ozone.version>
Review comment:
I marked this PR as `[WIP]`. Please remove that when the PR is ready.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29030: SPARK-32093. Add hadoop-ozone-filesystem jar to ozone profile.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29030:
URL: https://github.com/apache/spark/pull/29030#issuecomment-655180725
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #29030: [SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar to ozone profile.
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #29030:
URL: https://github.com/apache/spark/pull/29030#issuecomment-655792837
Does it work if a user app just includes it?
Can it go in hadoop-cloud if it must be packaged with Spark in some contexts -- why a new profile?
but what is Ozone, a file system from the Hadoop project, and who / how many people would need this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #29030: [WIP][SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #29030:
URL: https://github.com/apache/spark/pull/29030#issuecomment-711453284
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29030: SPARK-32093. Add hadoop-ozone-filesystem jar to ozone profile.
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29030:
URL: https://github.com/apache/spark/pull/29030#issuecomment-655179673
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] bharatviswa504 commented on pull request #29030: [SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar to ozone profile.
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on pull request #29030:
URL: https://github.com/apache/spark/pull/29030#issuecomment-655787847
The reason for doing this is similar to the idea of spark packaging cloud connector jars, so that spark will be out of the box can work with them. (And the user does not require to copy jars to spark executor classpath)
Similar support we want to add for ozone also. Right now to make spark work with ozone, we are using a --files and then running spark jobs. If spark executor has ozone jar included, we shall not require this step.
Purpose of this Jira is to include an ozone file system jar in spark release tarball, in this way spark can out of box work with an ozone cluster when used with ozone paths.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29030: SPARK-32093. Add hadoop-ozone-filesystem jar to ozone profile.
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29030:
URL: https://github.com/apache/spark/pull/29030#discussion_r451201726
##########
File path: pom.xml
##########
@@ -202,6 +202,9 @@
<arrow.version>0.15.1</arrow.version>
<!-- org.fusesource.leveldbjni will be used except on arm64 platform. -->
<leveldbjni.group>org.fusesource.leveldbjni</leveldbjni.group>
+
+ <!-- ozone version -->
+ <ozone.version>0.5.0-beta</ozone.version>
Review comment:
Thank you for making a PR, but we usually don't add `-beta` dependency.
cc @dbtsai and @gatorsmile
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #29030: [SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar to ozone profile.
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #29030:
URL: https://github.com/apache/spark/pull/29030#issuecomment-655568755
Gentle ping, @bharatviswa504 .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29030: [SPARK-32093][BUILD] Add hadoop-ozone-filesystem jar.
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #29030:
URL: https://github.com/apache/spark/pull/29030#discussion_r452317722
##########
File path: pom.xml
##########
@@ -202,6 +202,9 @@
<arrow.version>0.15.1</arrow.version>
<!-- org.fusesource.leveldbjni will be used except on arm64 platform. -->
<leveldbjni.group>org.fusesource.leveldbjni</leveldbjni.group>
+
+ <!-- ozone version -->
+ <ozone.version>0.5.0-beta</ozone.version>
Review comment:
When is the next `Ozone will be GA next release`? Could you close this and reopen after that with the official version?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org