You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Selvam Raman <se...@gmail.com> on 2016/03/30 05:44:44 UTC

Null pointer exception when using com.databricks.spark.csv

Hi,

i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows machine.

i was trying to use databricks csv format to read csv file. i used the
below command.

[image: Inline image 1]

I got null pointer exception. Any help would be greatly appreciated.

[image: Inline image 2]

-- 
Selvam Raman
"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"

Re: Null pointer exception when using com.databricks.spark.csv

Posted by Steve Loughran <st...@hortonworks.com>.
On 30 Mar 2016, at 04:44, Selvam Raman <se...@gmail.com>> wrote:

Hi,

i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows machine.

i was trying to use databricks csv format to read csv file. i used the below command.

<image.png>

I got null pointer exception. Any help would be greatly appreciated.

<image.png>

issue is Hadoop and it's need on windows for some native libs, compounded by bad error handlng of the situation

In Hadoop 2.8+ the NPE is replaced by a warning and a link to a page telling you what to do

https://wiki.apache.org/hadoop/WindowsProblems


Re: Null pointer exception when using com.databricks.spark.csv

Posted by Selvam Raman <se...@gmail.com>.
Hi,

i can able to load and extract the data. only problem when i using this
databricks library.

thanks,
selvam R

On Wed, Mar 30, 2016 at 9:33 AM, Hyukjin Kwon <gu...@gmail.com> wrote:

> Hi,
>
> I guess this is not a CSV-datasource specific problem.
>
> Does loading any file (eg. textFile()) work as well?
>
> I think this is related with this thread,
> http://apache-spark-user-list.1001560.n3.nabble.com/Error-while-running-example-scala-application-using-spark-submit-td10056.html
> .
>
>
> 2016-03-30 12:44 GMT+09:00 Selvam Raman <se...@gmail.com>:
>
>> Hi,
>>
>> i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows
>> machine.
>>
>> i was trying to use databricks csv format to read csv file. i used the
>> below command.
>>
>> [image: Inline image 1]
>>
>> I got null pointer exception. Any help would be greatly appreciated.
>>
>> [image: Inline image 2]
>>
>> --
>> Selvam Raman
>> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>>
>
>


-- 
Selvam Raman
"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"

Re: Null pointer exception when using com.databricks.spark.csv

Posted by Hyukjin Kwon <gu...@gmail.com>.
Hi,

I guess this is not a CSV-datasource specific problem.

Does loading any file (eg. textFile()) work as well?

I think this is related with this thread,
http://apache-spark-user-list.1001560.n3.nabble.com/Error-while-running-example-scala-application-using-spark-submit-td10056.html
.


2016-03-30 12:44 GMT+09:00 Selvam Raman <se...@gmail.com>:

> Hi,
>
> i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows machine.
>
> i was trying to use databricks csv format to read csv file. i used the
> below command.
>
> [image: Inline image 1]
>
> I got null pointer exception. Any help would be greatly appreciated.
>
> [image: Inline image 2]
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>

Re: Null pointer exception when using com.databricks.spark.csv

Posted by Hyukjin Kwon <gu...@gmail.com>.
Hi,

I guess this is not a CSV-datasource specific problem.

Does loading any file (eg. textFile()) work as well?

I think this is related with this thread,
http://apache-spark-user-list.1001560.n3.nabble.com/Error-while-running-example-scala-application-using-spark-submit-td10056.html
.


2016-03-30 12:44 GMT+09:00 Selvam Raman <se...@gmail.com>:

> Hi,
>
> i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows machine.
>
> i was trying to use databricks csv format to read csv file. i used the
> below command.
>
> [image: Inline image 1]
>
> I got null pointer exception. Any help would be greatly appreciated.
>
> [image: Inline image 2]
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>

Re: Null pointer exception when using com.databricks.spark.csv

Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Looks like the winutils.exe is missing from the environment, See
https://issues.apache.org/jira/browse/SPARK-2356

Thanks
Best Regards

On Wed, Mar 30, 2016 at 10:44 AM, Selvam Raman <se...@gmail.com> wrote:

> Hi,
>
> i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows machine.
>
> i was trying to use databricks csv format to read csv file. i used the
> below command.
>
> [image: Inline image 1]
>
> I got null pointer exception. Any help would be greatly appreciated.
>
> [image: Inline image 2]
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>

Re: Null pointer exception when using com.databricks.spark.csv

Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Looks like the winutils.exe is missing from the environment, See
https://issues.apache.org/jira/browse/SPARK-2356

Thanks
Best Regards

On Wed, Mar 30, 2016 at 10:44 AM, Selvam Raman <se...@gmail.com> wrote:

> Hi,
>
> i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows machine.
>
> i was trying to use databricks csv format to read csv file. i used the
> below command.
>
> [image: Inline image 1]
>
> I got null pointer exception. Any help would be greatly appreciated.
>
> [image: Inline image 2]
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>

Re: Null pointer exception when using com.databricks.spark.csv

Posted by Steve Loughran <st...@hortonworks.com>.
On 30 Mar 2016, at 04:44, Selvam Raman <se...@gmail.com>> wrote:

Hi,

i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows machine.

i was trying to use databricks csv format to read csv file. i used the below command.

<image.png>

I got null pointer exception. Any help would be greatly appreciated.

<image.png>

issue is Hadoop and it's need on windows for some native libs, compounded by bad error handlng of the situation

In Hadoop 2.8+ the NPE is replaced by a warning and a link to a page telling you what to do

https://wiki.apache.org/hadoop/WindowsProblems