You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Divya Gehlot <di...@gmail.com> on 2016/09/01 09:24:10 UTC
difference between package and jar Option in Spark
Hi,
Would like to know the difference between the --package and --jars option
in Spark .
Thanks,
Divya
Re: difference between package and jar Option in Spark
Posted by Tal Grynbaum <ta...@gmail.com>.
You need to download all the dependencies of that jar as well
On Mon, Sep 5, 2016, 06:59 Divya Gehlot <di...@gmail.com> wrote:
> Hi,
> I am using spark-csv to parse my input files .
> If I use --package option it works fine but if I download
> <https://mvnrepository.com/artifact/com.databricks/spark-csv_2.10/1.4.0>
> the jar and use --jars option
> Its throwing Class not found exception.
>
>
> Thanks,
> Divya
>
> On 1 September 2016 at 17:26, Sean Owen <so...@cloudera.com> wrote:
>
>> --jars includes a local JAR file in the application's classpath.
>> --package references Maven coordinates of a dependency and retrieves
>> and includes all of those JAR files, and includes them in the app
>> classpath.
>>
>> On Thu, Sep 1, 2016 at 10:24 AM, Divya Gehlot <di...@gmail.com>
>> wrote:
>> > Hi,
>> >
>> > Would like to know the difference between the --package and --jars
>> option in
>> > Spark .
>> >
>> >
>> >
>> > Thanks,
>> > Divya
>>
>
>
Re: difference between package and jar Option in Spark
Posted by Divya Gehlot <di...@gmail.com>.
Hi,
I am using spark-csv to parse my input files .
If I use --package option it works fine but if I download
<https://mvnrepository.com/artifact/com.databricks/spark-csv_2.10/1.4.0>
the jar and use --jars option
Its throwing Class not found exception.
Thanks,
Divya
On 1 September 2016 at 17:26, Sean Owen <so...@cloudera.com> wrote:
> --jars includes a local JAR file in the application's classpath.
> --package references Maven coordinates of a dependency and retrieves
> and includes all of those JAR files, and includes them in the app
> classpath.
>
> On Thu, Sep 1, 2016 at 10:24 AM, Divya Gehlot <di...@gmail.com>
> wrote:
> > Hi,
> >
> > Would like to know the difference between the --package and --jars
> option in
> > Spark .
> >
> >
> >
> > Thanks,
> > Divya
>
Re: difference between package and jar Option in Spark
Posted by Sean Owen <so...@cloudera.com>.
--jars includes a local JAR file in the application's classpath.
--package references Maven coordinates of a dependency and retrieves
and includes all of those JAR files, and includes them in the app
classpath.
On Thu, Sep 1, 2016 at 10:24 AM, Divya Gehlot <di...@gmail.com> wrote:
> Hi,
>
> Would like to know the difference between the --package and --jars option in
> Spark .
>
>
>
> Thanks,
> Divya
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org