NoSuchField Error : INSTANCE specify user defined httpclient jar

["sagarcasual ." <sa...@gmail.com>]

Hello,
I am using Spark 1.6.1 distribution over Cloudera CDH 5.7.0 cluster.
When I am running my fatJar - spark jar and when it is making a call to
HttpClient it is getting classic NoSuchField Error : INSTANCE. Which
happens usually when httrpclient in classpath is older than anticipated
httpclient jar version (should be 4.5.2).
Can someone help me how do I overcome this, I tried multiple options
1. specifying --conf spark.files={path-to-httpclient-4.5.2.jar} and then
using --conf spark.driver.extraClassPath=./httpclient-4.5.2.jar and
--conf spark.executor.extraClassPath=./httpclient-4.5.2.jar
2. I specified the jar in --jar option
3. I specified jar in hdfs://master:8020/path/to/jar/httpclient.4.5.2.jar
In each case I had error.
I also would like to know if
--conf spark.files={path-to-httpclient-4.5.2.jar},
is path-to-httpclient-4.5.2.jar can be giving to some local path from where
I am issuing spark-submit?
And also are there any other suggestions how should I resolve this conflict?

Any clues?
-Regards
Sagar

Re: NoSuchField Error : INSTANCE specify user defined httpclient jar

[Sean Owen <so...@cloudera.com>]

NoSuchFieldError in an HTTP client class?
This almost always means you have a conflicting versions of an
unshaded dependency on your classpath, and in this case could be
httpclient. You can often work around this with the userClassPathFirst
options for driver and executor.

On Sun, Sep 18, 2016 at 1:33 AM, sagarcasual . <sa...@gmail.com> wrote:
> Hello,
> I am using Spark 1.6.1 distribution over Cloudera CDH 5.7.0 cluster.
> When I am running my fatJar - spark jar and when it is making a call to
> HttpClient it is getting classic NoSuchField Error : INSTANCE. Which happens
> usually when httrpclient in classpath is older than anticipated httpclient
> jar version (should be 4.5.2).
> Can someone help me how do I overcome this, I tried multiple options
> 1. specifying --conf spark.files={path-to-httpclient-4.5.2.jar} and then
> using --conf spark.driver.extraClassPath=./httpclient-4.5.2.jar and --conf
> spark.executor.extraClassPath=./httpclient-4.5.2.jar
> 2. I specified the jar in --jar option
> 3. I specified jar in hdfs://master:8020/path/to/jar/httpclient.4.5.2.jar
> In each case I had error.
> I also would like to know if
> --conf spark.files={path-to-httpclient-4.5.2.jar},
> is path-to-httpclient-4.5.2.jar can be giving to some local path from where
> I am issuing spark-submit?
> And also are there any other suggestions how should I resolve this conflict?
>
> Any clues?
> -Regards
> Sagar

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org