You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@carbondata.apache.org by ja...@apache.org on 2018/06/22 01:34:27 UTC
[26/50] [abbrv] carbondata git commit: [CARBONDATA-2592][Integration]
Getting NoSuchMethod error due to aws sdk multple version jar conflicts
[CARBONDATA-2592][Integration] Getting NoSuchMethod error due to aws sdk multple version jar conflicts
## What changes were proposed in this pull request?
Currently in Carbon Spark2 project multiple dependency for the aws-sdk jar has been defined,this will create issue when we build carbon-assembly jars with
latest versions of hadoop/spark project. As part of hadoop-aws project, already aws-sdk 1.10.6 version jar will be fetched, since in the carbon-spark2 pom.xml
there is an explicit dependency defined/hardcoded for aws-sdk 1.7.4(old version) this can lead to conflicts while loading the class files. because of this problem
when we run any carbon examples passing carbon-assembly jars as the argument using spark-submit none of the testcases will work.
As a solution we can remove this dependency(aws-sdk 1.7.4) as already hadoop-aws dependency defined in the pom.xml of carbon-spark2 project
will fetch the latest aws-sdk jars.
## How was this patch tested?
After updating the pom, manually projects has been build and the use-case mentioned above is manually tested.
This closes #2364
Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/f0c88348
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/f0c88348
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/f0c88348
Branch: refs/heads/carbonstore
Commit: f0c88348a88f46b1cdaaa1e9bc00a3ab6111e5ec
Parents: f116352
Author: sujith71955 <su...@gmail.com>
Authored: Thu Jun 7 12:38:56 2018 +0530
Committer: ravipesala <ra...@gmail.com>
Committed: Fri Jun 15 12:30:54 2018 +0530
----------------------------------------------------------------------
integration/spark2/pom.xml | 19 -------------------
1 file changed, 19 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/carbondata/blob/f0c88348/integration/spark2/pom.xml
----------------------------------------------------------------------
diff --git a/integration/spark2/pom.xml b/integration/spark2/pom.xml
index 2bce694..9b9e71d 100644
--- a/integration/spark2/pom.xml
+++ b/integration/spark2/pom.xml
@@ -103,25 +103,6 @@
</exclusions>
</dependency>
<dependency>
- <groupId>com.amazonaws</groupId>
- <artifactId>aws-java-sdk</artifactId>
- <version>1.7.4</version>
- <exclusions>
- <exclusion>
- <groupId>com.fasterxml.jackson.core</groupId>
- <artifactId>jackson-core</artifactId>
- </exclusion>
- <exclusion>
- <groupId>com.fasterxml.jackson.core</groupId>
- <artifactId>jackson-annotations</artifactId>
- </exclusion>
- <exclusion>
- <groupId>com.fasterxml.jackson.core</groupId>
- <artifactId>jackson-databind</artifactId>
- </exclusion>
- </exclusions>
- </dependency>
- <dependency>
<groupId>net.java.dev.jets3t</groupId>
<artifactId>jets3t</artifactId>
<version>0.9.0</version>