Using 1.3.0 client jars with 1.2.1 assembly in yarn-cluster mode

[Zsolt Tóth <to...@gmail.com>]

Hi,

I submit spark jobs in yarn-cluster mode remotely from java code by calling
Client.submitApplication(). For some reason I want to use 1.3.0 jars on the
client side (e.g spark-yarn_2.10-1.3.0.jar) but I have
spark-assembly-1.2.1* on the cluster.
The problem is that the ApplicationMaster can't find the user application
jar (specified with --jar option). I think this is because of changes in
the classpath population in the Client class.
Is it possible to make this setup work without changing the codebase or the
jars?

Cheers,
Zsolt

Re: Using 1.3.0 client jars with 1.2.1 assembly in yarn-cluster mode

[Akhil Das <ak...@sigmoidanalytics.com>]

Mostly, when you use different versions of jars, it will throw up
incompatible version errors.

Thanks
Best Regards

On Fri, Mar 6, 2015 at 7:38 PM, Zsolt Tóth <to...@gmail.com> wrote:

> Hi,
>
> I submit spark jobs in yarn-cluster mode remotely from java code by
> calling Client.submitApplication(). For some reason I want to use 1.3.0
> jars on the client side (e.g spark-yarn_2.10-1.3.0.jar) but I have
> spark-assembly-1.2.1* on the cluster.
> The problem is that the ApplicationMaster can't find the user application
> jar (specified with --jar option). I think this is because of changes in
> the classpath population in the Client class.
> Is it possible to make this setup work without changing the codebase or
> the jars?
>
> Cheers,
> Zsolt
>