You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by dylanhogg <dy...@gmail.com> on 2014/12/06 03:26:20 UTC

Re: rdd.saveAsTextFile problem

Try the workaround for Windows found here:
http://qnalist.com/questions/4994960/run-spark-unit-test-on-windows-7.

This fix the issue when calling rdd.saveAsTextFile(..) for me with Spark
v1.1.0 on windows 8.1 in local mode.

Summary of steps:

1) download compiled winutils.exe from
http://social.msdn.microsoft.com/Forums/windowsazure/en-US/28a57efb-082b-424b-8d9e-731b1fe135de/please-read-if-experiencing-job-failures?forum=hdinsight

2) put this file into d:\winutil\bin

3) add in code: System.setProperty("hadoop.home.dir", "d:\\winutil\\")



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/rdd-saveAsTextFile-problem-tp176p20546.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: rdd.saveAsTextFile problem

Posted by Akhil Das <ak...@sigmoidanalytics.com>.
On Thu, May 21, 2015 at 4:17 PM, Howard Yang <ho...@gmail.com>
wrote:

> follow
> http://www.srccodes.com/p/article/38/build-install-configure-run-apache-hadoop-2.2.0-microsoft-windows-os
> to build latest version Hadoop in my windows machine,
> and Add Environment Variable *HADOOP_HOME* and edit *Path* Variable to
> add *bin* directory of *HADOOP_HOME* (say*C:\hadoop\bin*).
> fix this issue in my env
>
> 2015-05-21 9:55 GMT+03:00 Akhil Das <ak...@sigmoidanalytics.com>:
>
>> This thread happened a year back, can you please share what issue you are
>> facing? which version of spark you are using? What is your system
>> environment? Exception stack-trace?
>>
>> Thanks
>> Best Regards
>>
>> On Thu, May 21, 2015 at 12:19 PM, Keerthi <ke...@gmail.com>
>> wrote:
>>
>>> Hi ,
>>>
>>> I had tried the workaround shared here, but still facing the same
>>> issue...
>>>
>>> Thanks.
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://apache-spark-user-list.1001560.n3.nabble.com/rdd-saveAsTextFile-problem-tp176p22970.html
>>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: user-help@spark.apache.org
>>>
>>>
>>
>

Re: rdd.saveAsTextFile problem

Posted by Akhil Das <ak...@sigmoidanalytics.com>.
This thread happened a year back, can you please share what issue you are
facing? which version of spark you are using? What is your system
environment? Exception stack-trace?

Thanks
Best Regards

On Thu, May 21, 2015 at 12:19 PM, Keerthi <ke...@gmail.com>
wrote:

> Hi ,
>
> I had tried the workaround shared here, but still facing the same issue...
>
> Thanks.
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/rdd-saveAsTextFile-problem-tp176p22970.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Re: rdd.saveAsTextFile problem

Posted by Keerthi <ke...@gmail.com>.
Hi ,

I had tried the workaround shared here, but still facing the same issue...

Thanks.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/rdd-saveAsTextFile-problem-tp176p22970.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org