You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/06/29 18:14:25 UTC

[GitHub] [spark] viirya opened a new pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

viirya opened a new pull request #29326:
URL: https://github.com/apache/spark/pull/29326


   <!--
   Thanks for sending a pull request!  Here are some tips for you:
     1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
     2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
     3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
     4. Be sure to keep the PR description updated to reflect all changes.
     5. Please write your PR title to summarize what this PR proposes.
     6. If possible, provide a concise example to reproduce the issue for a faster review.
     7. If you want to add a new configuration, please read the guideline first for naming configurations in
        'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
   -->
   
   ### What changes were proposed in this pull request?
   <!--
   Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue. 
   If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
     1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
     2. If you fix some SQL features, you can provide some references of other DBMSes.
     3. If there is design documentation, please add the link.
     4. If there is a discussion in the mailing list, please add the link.
   -->
   
   This PR upgrades Guava to newer 27.0-jre and the dependency version of Hadoop 3.2 line to 3.2.1.
   
   ### Why are the changes needed?
   <!--
   Please clarify why the changes are needed. For instance,
     1. If you propose a new API, clarify the use case for a new API.
     2. If you fix a bug, you can clarify why it is a bug.
   -->
   
   Guava 14.0.1 is pretty old and is among the affected Guava versions of [CVE-2018-10237](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10237).
   
   All newer Hadoop releases are going to be built with a later guava version, e.g. 27.0-jre, including Hadoop [3.1.3](https://hadoop.apache.org/docs/r3.1.3/hadoop-project-dist/hadoop-common/dependency-analysis.html), [3.2.1](https://hadoop.apache.org/docs/r3.2.1/hadoop-project-dist/hadoop-common/dependency-analysis.html), [3.3.0](https://hadoop.apache.org/docs/r3.3.0/hadoop-project-dist/hadoop-common/dependency-analysis.html).
   
   Apache Hadoop 3.2 line has second stable release Hadoop 3.2.1 only few months after 3.2.0 release. Hadoop 3.2.1 already released for almost 1 year.
   
   ### Does this PR introduce _any_ user-facing change?
   <!--
   Note that it means *any* user-facing change including all aspects such as the documentation fix.
   If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
   If possible, please also clarify if this is a user-facing change compared to the released Spark versions or within the unreleased branches such as master.
   If no, write 'No'.
   -->
   
   No
   
   ### How was this patch tested?
   <!--
   If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
   If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
   If tests were not added, please describe why they were not added and/or why it was difficult to add.
   -->
   
   Pass the Jenkins tests.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-875060042


   > Oh I didn't even realize that Spark is using `hive-exec-core` jar. Does that mean it doesn't take advantage of the Guava shading from Hive 2.3.8+ at all?
   
   Yea, I'm afraid that it is true. If we want to completely isolate dependencies from Hive, we may need to relocate all included (but not relocated) dependencies in `hive-exec` w/o classifier.
   
   > One idea is to have Spark use [`hadoop-shaded-guava`](https://github.com/apache/hadoop-thirdparty) which is also 30.1.1-jre. It also makes sure that Spark always use the same Guava version as Hadoop.
   
   Even Spark uses `hadoop-shaded-guava`, but `hive-exec` still needs older Guava if we cannot use the version w/o classifier (due to other dependencies e.g. common-lang3, orc, parquet..)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871019144


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44916/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667801138


   It is a trouble that hive-exec uses a method that became package-private since Guava version 20. So there is incompatibility with Guava versions > 19.0.
   
   ```
   sbt.ForkMain$ForkError: sbt.ForkMain$ForkError: java.lang.IllegalAccessError: tried to access method com.google.common.collect.Iterators.emptyIterator()Lcom/google/common/collect/UnmodifiableIterator; from class org.apache.hadoop.hive.ql.exec.FetchOperator
   	at org.apache.hadoop.hive.ql.exec.FetchOperator.<init>(FetchOperator.java:108)
   	at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:87)
   	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:541)
   	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
   	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
   	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
   	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
   ```
   
   hive-exec doesn't shade Guava until https://issues.apache.org/jira/browse/HIVE-22126 that targets 4.0.0.
   
   This seems a dead end for upgrading Guava in Spark for now.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871287528


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140434/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-870811211


   try this again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667653231


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126935/
   Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871990967


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871101530


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871176432


   **[Test build #140430 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140430/testReport)** for PR 29326 at commit [`4e6da9c`](https://github.com/apache/spark/commit/4e6da9c6e730e5564c33012c2a1d72ac7c383cda).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds the following public classes _(experimental)_:
     * `class GroupBy(Generic[FrameLike], metaclass=ABCMeta):`
     * `class SparkIndexOpsMethods(Generic[IndexOpsLike], metaclass=ABCMeta):`
     * `class RollingAndExpanding(Generic[FrameLike], metaclass=ABCMeta):`
     * `class RollingLike(RollingAndExpanding[FrameLike]):`
     * `class Rolling(RollingLike[FrameLike]):`
     * `class RollingGroupby(RollingLike[FrameLike]):`
     * `class ExpandingLike(RollingAndExpanding[FrameLike]):`
     * `class Expanding(ExpandingLike[FrameLike]):`
     * `class ExpandingGroupby(ExpandingLike[FrameLike]):`
     * `sealed trait FieldPosition extends LeafExpression with Unevaluable `
     * `case class UnresolvedFieldPosition(`
     * `case class ResolvedFieldName(path: Seq[String], field: StructField) extends FieldName `
     * `case class ResolvedFieldPosition(position: ColumnPosition) extends FieldPosition`
     * `case class ArraysZip(children: Seq[Expression], names: Seq[Expression])`
     * `case class AlterTableAlterColumn(`
     * `case class ShowCreateTableExec(`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871110977


   **[Test build #140430 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140430/testReport)** for PR 29326 at commit [`4e6da9c`](https://github.com/apache/spark/commit/4e6da9c6e730e5564c33012c2a1d72ac7c383cda).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667638018






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya edited a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
viirya edited a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-872735918


   Hmm, from the failed tests below:
   
   org.apache.spark.sql.hive.DataSourceWithHiveMetastoreCatalogSuite
   org.apache.spark.sql.hive.HiveExternalCatalogSuite
   org.apache.spark.sql.hive.StatisticsSuite
   
   Since Guava 20, `com.google.common.collect.Iterators.emptyIterator()` is not public anymore. But I don't get it because Hive 2.3.8/2.3.9 shaded guava in hive-exec. Why it will use the newer guava upgraded here? 
   
   ```
   java.lang.IllegalAccessError: tried to access method com.google.common.collect.Iterators.emptyIterator()Lcom/google/common/collect/UnmodifiableIterator; from class org.apache.hadoop.hive.ql.exec.FetchOperator
   	at org.apache.hadoop.hive.ql.exec.FetchOperator.<init>(FetchOperator.java:108)
   	at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:87)
   	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:541)
   	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
   	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
   	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
   	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
   	at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$runHive$1(HiveClientImpl.scala:831)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun edited a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-670600810


   Thank you for assessment, @viirya . Is there an official plan for Apache Spark 4.0.0 release?
   
   Actually, this is 3rd try after mine and @HyukjinKwon 's . So, I was curious about what is changed more until now. At that time, we dropped the old PRs because it's hard to expect to get shaded Apache Hive 2.3.8.
   - https://github.com/apache/spark/pull/25932
   - https://github.com/apache/spark/pull/27009
   
   Apache Spark just migrated to Apache Hive 2.3. I don't think we can migrate to Apache Hive 4.0.0 in next one year.
   
   cc @gatorsmile 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-670601859


   BTW, shall we close for now? You can reopen this later when it's ready.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667634235


   **[Test build #126931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126931/testReport)** for PR 29326 at commit [`68375f0`](https://github.com/apache/spark/commit/68375f0145f28e69fd319a561cfa556573739f5c).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667634339






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-870962363


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140395/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-919308510


   #33989 seems a promising direction. Close this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-872006587


   **[Test build #140501 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140501/testReport)** for PR 29326 at commit [`d5e8ff8`](https://github.com/apache/spark/commit/d5e8ff8774289ce22cc477bc1a750fb8451e9f76).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667634339






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871971809


   **[Test build #140489 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140489/testReport)** for PR 29326 at commit [`d5e8ff8`](https://github.com/apache/spark/commit/d5e8ff8774289ce22cc477bc1a750fb8451e9f76).
    * This patch **fails to build**.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-872096470


   **[Test build #140501 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140501/testReport)** for PR 29326 at commit [`d5e8ff8`](https://github.com/apache/spark/commit/d5e8ff8774289ce22cc477bc1a750fb8451e9f76).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-872001701


   Kubernetes integration test unable to build dist.
   
   exiting with code: 1
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45001/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871968320


   **[Test build #140489 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140489/testReport)** for PR 29326 at commit [`d5e8ff8`](https://github.com/apache/spark/commit/d5e8ff8774289ce22cc477bc1a750fb8451e9f76).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871110977


   **[Test build #140430 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140430/testReport)** for PR 29326 at commit [`4e6da9c`](https://github.com/apache/spark/commit/4e6da9c6e730e5564c33012c2a1d72ac7c383cda).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-875177916


   Hmm, I looked at `isSharedClass`, looks like `common-lang3`, orc, etc. are already non-shared classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667643182


   **[Test build #126935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126935/testReport)** for PR 29326 at commit [`68375f0`](https://github.com/apache/spark/commit/68375f0145f28e69fd319a561cfa556573739f5c).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-670600810


   Thank you for confirming, @viirya . Is there an official plan for Apache Spark 4.0.0 release?
   
   Actually, this is 3rd try after mine and @HyukjinKwon 's . So, I was curious about what is changed more until now. At that time, we dropped the old PRs because it's hard to expect to get shaded Apache Hive 2.3.8.
   - https://github.com/apache/spark/pull/25932
   - https://github.com/apache/spark/pull/27009
   
   Apache Spark just migrated to Apache Hive 2.3. I don't think we can migrate to Apache Hive 4.0.0 in next one year.
   
   cc @gatorsmile 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-670608710


   @dongjoon-hyun Thanks for the comment. Yeah, it doesn't make sense to upgrade to Hive 4 in short or midterm. I'm working on upgrade Guava 27 and shading Guava in Hive too. I hope it can be part of Hive 2.3.8.
   
   I will close this for now. Once the work at Hive gets progress, I can reopen this. Thanks.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dbtsai commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
dbtsai commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-694349182


   One question, in https://issues.apache.org/jira/browse/HADOOP-14284 it seems that Hadoop shades the Guava dependency, why do we introduce breaking changes when we upgrade to Hadoop 3.2.1 or Hadoop 3.3?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-870964726


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140396/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871138101


   Kubernetes integration test unable to build dist.
   
   exiting with code: 1
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44945/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] sunchao commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
sunchao commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-875095115


   Hmm yea you are right, but shading the other dependencies will require another release though.
   
   Another thing we could try is to change `IsolatedClientLoader#isSharedClass`and have the Hive client to use its own version of `common-lang3` etc, if there is a conflict. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871246956


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44948/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-873359259


   Encountered some issues.
   
   Although we can switch to hive-exec without classifier (shaded version) to get rid of above guava version issue, the shaded hive-exec contains (without relocation) some dependencies like commons-lang3, orc that are not same version with Spark and so they conflict.
   
   Because shaded hive-exec jar already includes these dependency jars, seems dependency exclusions in pom cannot exclude them.
   
   Currently seems we can just go back to Hive to shade every included dependencies? Any other thoughts?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-870964103


   **[Test build #140396 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140396/testReport)** for PR 29326 at commit [`40fc8ee`](https://github.com/apache/spark/commit/40fc8eeab58a857db4e64ca920d33c782f7b8c5f).
    * This patch **fails Spark unit tests**.
    * This patch **does not merge cleanly**.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-870858892


   **[Test build #140395 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140395/testReport)** for PR 29326 at commit [`68375f0`](https://github.com/apache/spark/commit/68375f0145f28e69fd319a561cfa556573739f5c).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-872097517


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140501/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-870850274


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya edited a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
viirya edited a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-873359259


   Encountered some issues.
   
   Although we can switch to hive-exec without classifier (shaded version) to get rid of above guava version issue, the shaded hive-exec contains (without relocation) some dependencies like commons-lang3, orc, parquet that are not same version with Spark and so they conflict.
   
   Because shaded hive-exec jar already includes these dependency jars, seems dependency exclusions in pom cannot exclude them.
   
   Currently seems we can just go back to Hive to shade every included dependencies? Any other thoughts?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871287528


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140434/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dbtsai commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
dbtsai commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-694375736


   @viirya you are right. My bad.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667634235


   **[Test build #126931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126931/testReport)** for PR 29326 at commit [`68375f0`](https://github.com/apache/spark/commit/68375f0145f28e69fd319a561cfa556573739f5c).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-872006587


   **[Test build #140501 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140501/testReport)** for PR 29326 at commit [`d5e8ff8`](https://github.com/apache/spark/commit/d5e8ff8774289ce22cc477bc1a750fb8451e9f76).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871030380


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44916/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871177120


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140430/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871178742


   **[Test build #140434 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140434/testReport)** for PR 29326 at commit [`74d5fbd`](https://github.com/apache/spark/commit/74d5fbd01c224354149f8f449f0c1a5eb31ef007).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] danielradulov commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
danielradulov commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-691689843


   Hi Guys, I am having problems with Guava on Spark 3.0.0 and 3.0.1 with Hadoop 3.2.1 and Hive 3.12.  
   
   I am using Spark Operator developed by Google, all seems to work fine except when I try to use Spark integrated with Hive Metastore. In this case I am facing the following error:  
   
   java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument
   
   I have tried several workarounds like replacing Guava, Spark spec on client  "spark.executor.userClassPathFirst": "true" "spark.driver.userClassPathFirst": "true", shading Guava with maven-shade-plugin and unfortunately any of this alternatives are working properly.
   
   I hope you can be able to upgrade Guava soon in Spark.
   
   thanks.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667653229


   Merged build finished. Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871971844


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140489/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871007226


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44916/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871971844






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-872097258






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-691796793


   Thank you for your opinion, @danielradulov . However, it's Apache Hive issue across Apache Hadoop versions.
   > except when I try to use Spark integrated with Hive Metastore.
   
   Apache Hadoop 3.2.1 has a breaking Guava dependency change which breaks most downstream project. IIRC, there is no official Apache Hive version to work on Apache Hadoop 3.2.1. You had better ask the support to Apache Hive community.
   
   Apache Spark community tried to upgrade to Apache Hadoop 3.2.1 (Sep. 2019) and gave up due to that.
   - https://github.com/apache/spark/pull/25932


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667638018


   Merged build finished. Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-668878065


   I did some tests. Few changes are required to pass the failed Hive tests:
   
   1. Shading Guava at hive-exec packaging and a few code changes to hive-common and hive-exec regarding Guava usage
   2. Don't use `core` classifier for hive dependencies in Spark
   
   But this just upgrades Guava version used in Spark. Hive dependencies still use older Guava with the reported CVE.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871122684


   I'm not against to this point. I can change to latest guava and see what CI tells.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-870858892






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-870964726


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140396/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-872735918


   Hmm, from the failed tests below:
   
   org.apache.spark.sql.hive.DataSourceWithHiveMetastoreCatalogSuite
   org.apache.spark.sql.hive.HiveExternalCatalogSuite
   org.apache.spark.sql.hive.StatisticsSuite
   
   Since Guava 20, `com.google.common.collect.Iterators.emptyIterator()` is not public anymore. But I don't get it because Hive 2.3.8/2.3.9 shaded guava. Why it will use the newer guava upgraded here? 
   
   ```
   java.lang.IllegalAccessError: tried to access method com.google.common.collect.Iterators.emptyIterator()Lcom/google/common/collect/UnmodifiableIterator; from class org.apache.hadoop.hive.ql.exec.FetchOperator
   	at org.apache.hadoop.hive.ql.exec.FetchOperator.<init>(FetchOperator.java:108)
   	at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:87)
   	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:541)
   	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
   	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
   	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
   	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
   	at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$runHive$1(HiveClientImpl.scala:831)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667653229






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya edited a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
viirya edited a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-872735918


   Hmm, from the failed tests below:
   
   org.apache.spark.sql.hive.DataSourceWithHiveMetastoreCatalogSuite
   org.apache.spark.sql.hive.HiveExternalCatalogSuite
   org.apache.spark.sql.hive.StatisticsSuite
   
   Since Guava 20, `com.google.common.collect.Iterators.emptyIterator()` is not public anymore. But I don't get it because Hive 2.3.8/2.3.9 shaded guava in hive-exec. Why it will use the newer guava upgraded here? 
   
   ```
   java.lang.IllegalAccessError: tried to access method com.google.common.collect.Iterators.emptyIterator()Lcom/google/common/collect/UnmodifiableIterator; from class org.apache.hadoop.hive.ql.exec.FetchOperator
   	at org.apache.hadoop.hive.ql.exec.FetchOperator.<init>(FetchOperator.java:108)
   	at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:87)
   	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:541)
   	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
   	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
   	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
   	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
   	at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$runHive$1(HiveClientImpl.scala:831)
   ```
   
   I seems figured it out. Verifying it locally...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya closed pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
viirya closed pull request #29326:
URL: https://github.com/apache/spark/pull/29326


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-870896050


   **[Test build #140400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140400/testReport)** for PR 29326 at commit [`4e6da9c`](https://github.com/apache/spark/commit/4e6da9c6e730e5564c33012c2a1d72ac7c383cda).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-870858996


   **[Test build #140396 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140396/testReport)** for PR 29326 at commit [`40fc8ee`](https://github.com/apache/spark/commit/40fc8eeab58a857db4e64ca920d33c782f7b8c5f).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871258990


   **[Test build #140434 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140434/testReport)** for PR 29326 at commit [`74d5fbd`](https://github.com/apache/spark/commit/74d5fbd01c224354149f8f449f0c1a5eb31ef007).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] sunchao commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
sunchao commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-874917606


   Oh I didn't even realize that Spark is using `hive-exec-core` jar. Does that mean it doesn't take advantage of the Guava shading from Hive 2.3.8+ at all?
   
   One idea is to have Spark use [`hadoop-shaded-guava`](https://github.com/apache/hadoop-thirdparty) which is also 30.1.1-jre. It also makes sure that Spark always use the same Guava version as Hadoop.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667643311






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667638019


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126931/
   Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667637958


   **[Test build #126931 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126931/testReport)** for PR 29326 at commit [`68375f0`](https://github.com/apache/spark/commit/68375f0145f28e69fd319a561cfa556573739f5c).
    * This patch **fails due to an unknown error code, -9**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667643182


   **[Test build #126935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126935/testReport)** for PR 29326 at commit [`68375f0`](https://github.com/apache/spark/commit/68375f0145f28e69fd319a561cfa556573739f5c).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] srowen commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667690112


   Is this duplicated by https://github.com/apache/spark/pull/29325 ?
   Yes, this can only happen for Hadoop 3.2.1+, so would at best be in the Hadoop 3.2 profile.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-694355931


   Isn't HADOOP-14284 resolved as Invalid?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-872003260


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45001/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-872097233


   Kubernetes integration test unable to build dist.
   
   exiting with code: 1
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45011/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871138114






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-872097258


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45011/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-870933101


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140400/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871178742


   **[Test build #140434 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140434/testReport)** for PR 29326 at commit [`74d5fbd`](https://github.com/apache/spark/commit/74d5fbd01c224354149f8f449f0c1a5eb31ef007).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871242105


   Kubernetes integration test unable to build dist.
   
   exiting with code: 1
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44948/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871138114


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44945/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871133359


   Thanks. Ya, let's try with the latest one.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya closed pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
viirya closed pull request #29326:
URL: https://github.com/apache/spark/pull/29326


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667653114


   **[Test build #126935 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126935/testReport)** for PR 29326 at commit [`68375f0`](https://github.com/apache/spark/commit/68375f0145f28e69fd319a561cfa556573739f5c).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-670656860


   Thank you so much. Yes. I'm looking forward to seeing that~


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-870958769


   **[Test build #140395 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140395/testReport)** for PR 29326 at commit [`68375f0`](https://github.com/apache/spark/commit/68375f0145f28e69fd319a561cfa556573739f5c).
    * This patch **fails Spark unit tests**.
    * This patch **does not merge cleanly**.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871968320


   **[Test build #140489 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140489/testReport)** for PR 29326 at commit [`d5e8ff8`](https://github.com/apache/spark/commit/d5e8ff8774289ce22cc477bc1a750fb8451e9f76).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-871030380


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44916/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] maropu commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667643037


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667643311






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-668116815


   Opened https://issues.apache.org/jira/browse/HIVE-23980 and see if Hive people has some ideas.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-870962363


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140395/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-870933101


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140400/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org