You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2019/05/23 20:21:41 UTC

[spark] branch master updated: [SPARK-26045][BUILD] Leave avro, avro-ipc dependendencies as compile scope even for hadoop-provided usages

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 32f310b  [SPARK-26045][BUILD] Leave avro, avro-ipc dependendencies as compile scope even for hadoop-provided usages
32f310b is described below

commit 32f310b585fd79980a841d0b507f188123ff8121
Author: Sean Owen <se...@databricks.com>
AuthorDate: Thu May 23 13:21:21 2019 -0700

    [SPARK-26045][BUILD] Leave avro, avro-ipc dependendencies as compile scope even for hadoop-provided usages
    
    ## What changes were proposed in this pull request?
    
    Leave avro, avro-ipc dependendencies as compile scope even for hadoop-provided usages, to ensure 1.8 is used. Hadoop 2.7 has Avro 1.7, and Spark won't generally work with that. Reports from the field are that this works, to include avro 1.8 with the Spark distro on Hadoop 2.7.
    
    ## How was this patch tested?
    
    Existing tests
    
    Closes #24680 from srowen/SPARK-26045.
    
    Authored-by: Sean Owen <se...@databricks.com>
    Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
 pom.xml | 2 --
 1 file changed, 2 deletions(-)

diff --git a/pom.xml b/pom.xml
index ff891ba..00d1374 100644
--- a/pom.xml
+++ b/pom.xml
@@ -982,13 +982,11 @@
         <groupId>org.apache.avro</groupId>
         <artifactId>avro</artifactId>
         <version>${avro.version}</version>
-        <scope>${hadoop.deps.scope}</scope>
       </dependency>
       <dependency>
         <groupId>org.apache.avro</groupId>
         <artifactId>avro-ipc</artifactId>
         <version>${avro.version}</version>
-        <scope>${hadoop.deps.scope}</scope>
         <exclusions>
           <exclusion>
             <groupId>io.netty</groupId>


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org