You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Sunil Kumar <pa...@yahoo.com.INVALID> on 2016/06/09 15:56:27 UTC
JIRA SPARK-2984
Hi,
I am running into SPARK-2984 while running my spark 1.6.1 jobs over yarn in AWS. I have tried with spark.speculation=false but still see the same failure with _temporary file missing for task_xxx...This ticket is in resolved state. How can this be reopened ? Is there a workaround ?
thanks
Re: JIRA SPARK-2984
Posted by Sunil Kumar <pa...@yahoo.com.INVALID>.
Since I couldnt find a related PR, I have raised JIRA-15849
Thx
On Thursday, June 9, 2016 11:07 AM, Holden Karau <ho...@pigscanfly.ca> wrote:
I'd do some searching and see if there is a JIRA related to this problem on s3 and if you don't find one go ahead and make one. Even if it is an intrinsic problem with s3 (and I'm not super sure since I'm just reading this on mobile) - it would maybe be a good thing for us to document.
On Thursday, June 9, 2016, Sunil Kumar <pa...@yahoo.com> wrote:
| Holden
Thanks for your prompt reply... Any suggestions on the next step ? Does this call for a new spark jira ticket or is this an issue for s3?
Thx |
| I think your error could possibly be different - looking at the original JIRA the issue was happening on HDFS and you seem to be experiencing the issue on s3n, and while I don't have full view of the problem I could see this being s3 specific (read-after-write on s3 is trickier than read-after-write on HDFS).
On Thursday, June 9, 2016, Sunil Kumar <pa...@yahoo.com.invalid> wrote:
Hi,
I am running into SPARK-2984 while running my spark 1.6.1 jobs over yarn in AWS. I have tried with spark.speculation=false but still see the same failure with _temporary file missing for task_xxx...This ticket is in resolved state. How can this be reopened ? Is there a workaround ?
thanks
--
Cell : 425-233-8271Twitter: https://twitter.com/holdenkarau
|
--
Cell : 425-233-8271Twitter: https://twitter.com/holdenkarau
Re: JIRA SPARK-2984
Posted by Holden Karau <ho...@pigscanfly.ca>.
I'd do some searching and see if there is a JIRA related to this problem
on s3 and if you don't find one go ahead and make one. Even if it is an
intrinsic problem with s3 (and I'm not super sure since I'm just reading
this on mobile) - it would maybe be a good thing for us to document.
On Thursday, June 9, 2016, Sunil Kumar <pa...@yahoo.com> wrote:
> Holden
> Thanks for your prompt reply... Any suggestions on the next step ? Does
> this call for a new spark jira ticket or is this an issue for s3?
> Thx
>
>
> I think your error could possibly be different - looking at the original
> JIRA the issue was happening on HDFS and you seem to be experiencing the
> issue on s3n, and while I don't have full view of the problem I could see
> this being s3 specific (read-after-write on s3 is trickier than
> read-after-write on HDFS).
>
> On Thursday, June 9, 2016, Sunil Kumar <pa...@yahoo.com.invalid>
> wrote:
>
>> Hi,
>>
>> I am running into SPARK-2984 while running my spark 1.6.1 jobs over yarn
>> in AWS. I have tried with spark.speculation=false but still see the same
>> failure with _temporary file missing for task_xxx...This ticket is in
>> resolved state. How can this be reopened ? Is there a workaround ?
>>
>> thanks
>>
>
>
> --
> Cell : 425-233-8271
> Twitter: https://twitter.com/holdenkarau
>
>
--
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau
Re: JIRA SPARK-2984
Posted by Holden Karau <ho...@pigscanfly.ca>.
I'd do some searching and see if there is a JIRA related to this problem
on s3 and if you don't find one go ahead and make one. Even if it is an
intrinsic problem with s3 (and I'm not super sure since I'm just reading
this on mobile) - it would maybe be a good thing for us to document.
On Thursday, June 9, 2016, Sunil Kumar <pa...@yahoo.com> wrote:
> Holden
> Thanks for your prompt reply... Any suggestions on the next step ? Does
> this call for a new spark jira ticket or is this an issue for s3?
> Thx
>
>
> I think your error could possibly be different - looking at the original
> JIRA the issue was happening on HDFS and you seem to be experiencing the
> issue on s3n, and while I don't have full view of the problem I could see
> this being s3 specific (read-after-write on s3 is trickier than
> read-after-write on HDFS).
>
> On Thursday, June 9, 2016, Sunil Kumar <pa...@yahoo.com.invalid>
> wrote:
>
>> Hi,
>>
>> I am running into SPARK-2984 while running my spark 1.6.1 jobs over yarn
>> in AWS. I have tried with spark.speculation=false but still see the same
>> failure with _temporary file missing for task_xxx...This ticket is in
>> resolved state. How can this be reopened ? Is there a workaround ?
>>
>> thanks
>>
>
>
> --
> Cell : 425-233-8271
> Twitter: https://twitter.com/holdenkarau
>
>
--
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau
Re: JIRA SPARK-2984
Posted by Sunil Kumar <pa...@yahoo.com.INVALID>.
Holden
Thanks for your prompt reply... Any suggestions on the next step ? Does this call for a new spark jira ticket or is this an issue for s3?
Thx
Re: JIRA SPARK-2984
Posted by Holden Karau <ho...@pigscanfly.ca>.
I think your error could possibly be different - looking at the original
JIRA the issue was happening on HDFS and you seem to be experiencing the
issue on s3n, and while I don't have full view of the problem I could see
this being s3 specific (read-after-write on s3 is trickier than
read-after-write on HDFS).
On Thursday, June 9, 2016, Sunil Kumar <pa...@yahoo.com.invalid>
wrote:
> Hi,
>
> I am running into SPARK-2984 while running my spark 1.6.1 jobs over yarn
> in AWS. I have tried with spark.speculation=false but still see the same
> failure with _temporary file missing for task_xxx...This ticket is in
> resolved state. How can this be reopened ? Is there a workaround ?
>
> thanks
>
--
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau
Re: JIRA SPARK-2984
Posted by Holden Karau <ho...@pigscanfly.ca>.
I think your error could possibly be different - looking at the original
JIRA the issue was happening on HDFS and you seem to be experiencing the
issue on s3n, and while I don't have full view of the problem I could see
this being s3 specific (read-after-write on s3 is trickier than
read-after-write on HDFS).
On Thursday, June 9, 2016, Sunil Kumar <pa...@yahoo.com.invalid>
wrote:
> Hi,
>
> I am running into SPARK-2984 while running my spark 1.6.1 jobs over yarn
> in AWS. I have tried with spark.speculation=false but still see the same
> failure with _temporary file missing for task_xxx...This ticket is in
> resolved state. How can this be reopened ? Is there a workaround ?
>
> thanks
>
--
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau