You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Selvam Raman <se...@gmail.com> on 2016/03/30 05:44:44 UTC
Null pointer exception when using com.databricks.spark.csv
Hi,
i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows machine.
i was trying to use databricks csv format to read csv file. i used the
below command.
[image: Inline image 1]
I got null pointer exception. Any help would be greatly appreciated.
[image: Inline image 2]
--
Selvam Raman
"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
Re: Null pointer exception when using com.databricks.spark.csv
Posted by Steve Loughran <st...@hortonworks.com>.
On 30 Mar 2016, at 04:44, Selvam Raman <se...@gmail.com>> wrote:
Hi,
i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows machine.
i was trying to use databricks csv format to read csv file. i used the below command.
<image.png>
I got null pointer exception. Any help would be greatly appreciated.
<image.png>
issue is Hadoop and it's need on windows for some native libs, compounded by bad error handlng of the situation
In Hadoop 2.8+ the NPE is replaced by a warning and a link to a page telling you what to do
https://wiki.apache.org/hadoop/WindowsProblems
Re: Null pointer exception when using com.databricks.spark.csv
Posted by Selvam Raman <se...@gmail.com>.
Hi,
i can able to load and extract the data. only problem when i using this
databricks library.
thanks,
selvam R
On Wed, Mar 30, 2016 at 9:33 AM, Hyukjin Kwon <gu...@gmail.com> wrote:
> Hi,
>
> I guess this is not a CSV-datasource specific problem.
>
> Does loading any file (eg. textFile()) work as well?
>
> I think this is related with this thread,
> http://apache-spark-user-list.1001560.n3.nabble.com/Error-while-running-example-scala-application-using-spark-submit-td10056.html
> .
>
>
> 2016-03-30 12:44 GMT+09:00 Selvam Raman <se...@gmail.com>:
>
>> Hi,
>>
>> i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows
>> machine.
>>
>> i was trying to use databricks csv format to read csv file. i used the
>> below command.
>>
>> [image: Inline image 1]
>>
>> I got null pointer exception. Any help would be greatly appreciated.
>>
>> [image: Inline image 2]
>>
>> --
>> Selvam Raman
>> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>>
>
>
--
Selvam Raman
"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
Re: Null pointer exception when using com.databricks.spark.csv
Posted by Hyukjin Kwon <gu...@gmail.com>.
Hi,
I guess this is not a CSV-datasource specific problem.
Does loading any file (eg. textFile()) work as well?
I think this is related with this thread,
http://apache-spark-user-list.1001560.n3.nabble.com/Error-while-running-example-scala-application-using-spark-submit-td10056.html
.
2016-03-30 12:44 GMT+09:00 Selvam Raman <se...@gmail.com>:
> Hi,
>
> i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows machine.
>
> i was trying to use databricks csv format to read csv file. i used the
> below command.
>
> [image: Inline image 1]
>
> I got null pointer exception. Any help would be greatly appreciated.
>
> [image: Inline image 2]
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>
Re: Null pointer exception when using com.databricks.spark.csv
Posted by Hyukjin Kwon <gu...@gmail.com>.
Hi,
I guess this is not a CSV-datasource specific problem.
Does loading any file (eg. textFile()) work as well?
I think this is related with this thread,
http://apache-spark-user-list.1001560.n3.nabble.com/Error-while-running-example-scala-application-using-spark-submit-td10056.html
.
2016-03-30 12:44 GMT+09:00 Selvam Raman <se...@gmail.com>:
> Hi,
>
> i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows machine.
>
> i was trying to use databricks csv format to read csv file. i used the
> below command.
>
> [image: Inline image 1]
>
> I got null pointer exception. Any help would be greatly appreciated.
>
> [image: Inline image 2]
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>
Re: Null pointer exception when using com.databricks.spark.csv
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Looks like the winutils.exe is missing from the environment, See
https://issues.apache.org/jira/browse/SPARK-2356
Thanks
Best Regards
On Wed, Mar 30, 2016 at 10:44 AM, Selvam Raman <se...@gmail.com> wrote:
> Hi,
>
> i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows machine.
>
> i was trying to use databricks csv format to read csv file. i used the
> below command.
>
> [image: Inline image 1]
>
> I got null pointer exception. Any help would be greatly appreciated.
>
> [image: Inline image 2]
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>
Re: Null pointer exception when using com.databricks.spark.csv
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Looks like the winutils.exe is missing from the environment, See
https://issues.apache.org/jira/browse/SPARK-2356
Thanks
Best Regards
On Wed, Mar 30, 2016 at 10:44 AM, Selvam Raman <se...@gmail.com> wrote:
> Hi,
>
> i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows machine.
>
> i was trying to use databricks csv format to read csv file. i used the
> below command.
>
> [image: Inline image 1]
>
> I got null pointer exception. Any help would be greatly appreciated.
>
> [image: Inline image 2]
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>
Re: Null pointer exception when using com.databricks.spark.csv
Posted by Steve Loughran <st...@hortonworks.com>.
On 30 Mar 2016, at 04:44, Selvam Raman <se...@gmail.com>> wrote:
Hi,
i am using spark 1.6.0 prebuilt hadoop 2.6.0 version in my windows machine.
i was trying to use databricks csv format to read csv file. i used the below command.
<image.png>
I got null pointer exception. Any help would be greatly appreciated.
<image.png>
issue is Hadoop and it's need on windows for some native libs, compounded by bad error handlng of the situation
In Hadoop 2.8+ the NPE is replaced by a warning and a link to a page telling you what to do
https://wiki.apache.org/hadoop/WindowsProblems