You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Hu...@Dell.com on 2014/01/18 01:34:36 UTC
SparkException: Expect only DirectTaskResults when using
localScheduler()
Hi,
Can someone please explain what it means the following exception 'SparkException: Expect only DirectTaskResults when using localScheduler()'?
This is related to spark.akka.frameSize?
I am running single node local spark 0.8.1
Thanks,
Hussam
Re: SparkException: Expect only DirectTaskResults when using localScheduler()
Posted by Patrick Wendell <pw...@gmail.com>.
It's 10 MB:
http://spark.incubator.apache.org/docs/latest/configuration.html
- Patrick
On Sat, Jan 18, 2014 at 12:08 AM, Nick Pentreath
<ni...@gmail.com> wrote:
> Ok that makes sense thanks. What is the default all a frame size?
> —
> Sent from Mailbox for iPhone
>
>
> On Sat, Jan 18, 2014 at 9:54 AM, Patrick Wendell <pw...@gmail.com> wrote:
>>
>> This is a bug that was fixed and will be part of 0.8.2:
>>
>> https://github.com/apache/incubator-spark/pull/281
>>
>> A workaround is setting the akka frame size to be larger using
>> spark.akka.frameSize.
>>
>> The issue is that we added a mechanism to circumvent akka for large
>> task results, but the implementation assumed this would never happen
>> in local mode. That assumption turned out to be wrong, so it was
>> patched in #281.
>>
>> - Patrick
>>
>> On Fri, Jan 17, 2014 at 9:53 PM, Nick Pentreath
>> <ni...@gmail.com> wrote:
>> > I'm also running into this issue. If it is related to a setting then the
>> > error message should surely be cleaned up a bit to be more helpful in
>> > suggesting a fix?
>> > —
>> > Sent from Mailbox for iPhone
>> >
>> >
>> > On Sat, Jan 18, 2014 at 2:35 AM, Hussam_Jarada@dell.com
>> > <Hu...@dell.com> wrote:
>> >>
>> >> Hi,
>> >>
>> >>
>> >>
>> >> Can someone please explain what it means the following exception
>> >> ‘SparkException: Expect only DirectTaskResults when using
>> >> localScheduler()’?
>> >>
>> >> This is related to spark.akka.frameSize?
>> >>
>> >> I am running single node local spark 0.8.1
>> >>
>> >>
>> >>
>> >> Thanks,
>> >>
>> >> Hussam
>> >
>> >
>
>
Re: SparkException: Expect only DirectTaskResults when using
localScheduler()
Posted by Nick Pentreath <ni...@gmail.com>.
Ok that makes sense thanks. What is the default all a frame size?—
Sent from Mailbox for iPhone
On Sat, Jan 18, 2014 at 9:54 AM, Patrick Wendell <pw...@gmail.com>
wrote:
> This is a bug that was fixed and will be part of 0.8.2:
> https://github.com/apache/incubator-spark/pull/281
> A workaround is setting the akka frame size to be larger using
> spark.akka.frameSize.
> The issue is that we added a mechanism to circumvent akka for large
> task results, but the implementation assumed this would never happen
> in local mode. That assumption turned out to be wrong, so it was
> patched in #281.
> - Patrick
> On Fri, Jan 17, 2014 at 9:53 PM, Nick Pentreath
> <ni...@gmail.com> wrote:
>> I'm also running into this issue. If it is related to a setting then the
>> error message should surely be cleaned up a bit to be more helpful in
>> suggesting a fix?
>> —
>> Sent from Mailbox for iPhone
>>
>>
>> On Sat, Jan 18, 2014 at 2:35 AM, Hussam_Jarada@dell.com
>> <Hu...@dell.com> wrote:
>>>
>>> Hi,
>>>
>>>
>>>
>>> Can someone please explain what it means the following exception
>>> ‘SparkException: Expect only DirectTaskResults when using localScheduler()’?
>>>
>>> This is related to spark.akka.frameSize?
>>>
>>> I am running single node local spark 0.8.1
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Hussam
>>
>>
RE: SparkException: Expect only DirectTaskResults when using
localScheduler()
Posted by Hu...@Dell.com.
Thank u Patrick.
-----Original Message-----
From: Patrick Wendell [mailto:pwendell@gmail.com]
Sent: Friday, January 17, 2014 11:54 PM
To: user@spark.incubator.apache.org
Subject: Re: SparkException: Expect only DirectTaskResults when using localScheduler()
This is a bug that was fixed and will be part of 0.8.2:
https://github.com/apache/incubator-spark/pull/281
A workaround is setting the akka frame size to be larger using spark.akka.frameSize.
The issue is that we added a mechanism to circumvent akka for large task results, but the implementation assumed this would never happen in local mode. That assumption turned out to be wrong, so it was patched in #281.
- Patrick
On Fri, Jan 17, 2014 at 9:53 PM, Nick Pentreath <ni...@gmail.com> wrote:
> I'm also running into this issue. If it is related to a setting then
> the error message should surely be cleaned up a bit to be more helpful
> in suggesting a fix?
> -
> Sent from Mailbox for iPhone
>
>
> On Sat, Jan 18, 2014 at 2:35 AM, Hussam_Jarada@dell.com
> <Hu...@dell.com> wrote:
>>
>> Hi,
>>
>>
>>
>> Can someone please explain what it means the following exception
>> 'SparkException: Expect only DirectTaskResults when using localScheduler()'?
>>
>> This is related to spark.akka.frameSize?
>>
>> I am running single node local spark 0.8.1
>>
>>
>>
>> Thanks,
>>
>> Hussam
>
>
Re: SparkException: Expect only DirectTaskResults when using localScheduler()
Posted by Patrick Wendell <pw...@gmail.com>.
This is a bug that was fixed and will be part of 0.8.2:
https://github.com/apache/incubator-spark/pull/281
A workaround is setting the akka frame size to be larger using
spark.akka.frameSize.
The issue is that we added a mechanism to circumvent akka for large
task results, but the implementation assumed this would never happen
in local mode. That assumption turned out to be wrong, so it was
patched in #281.
- Patrick
On Fri, Jan 17, 2014 at 9:53 PM, Nick Pentreath
<ni...@gmail.com> wrote:
> I'm also running into this issue. If it is related to a setting then the
> error message should surely be cleaned up a bit to be more helpful in
> suggesting a fix?
> —
> Sent from Mailbox for iPhone
>
>
> On Sat, Jan 18, 2014 at 2:35 AM, Hussam_Jarada@dell.com
> <Hu...@dell.com> wrote:
>>
>> Hi,
>>
>>
>>
>> Can someone please explain what it means the following exception
>> ‘SparkException: Expect only DirectTaskResults when using localScheduler()’?
>>
>> This is related to spark.akka.frameSize?
>>
>> I am running single node local spark 0.8.1
>>
>>
>>
>> Thanks,
>>
>> Hussam
>
>
Re: SparkException: Expect only DirectTaskResults when using
localScheduler()
Posted by Nick Pentreath <ni...@gmail.com>.
I'm also running into this issue. If it is related to a setting then the error message should surely be cleaned up a bit to be more helpful in suggesting a fix?—
Sent from Mailbox for iPhone
On Sat, Jan 18, 2014 at 2:35 AM, null <Hu...@dell.com> wrote:
> Hi,
> Can someone please explain what it means the following exception 'SparkException: Expect only DirectTaskResults when using localScheduler()'?
> This is related to spark.akka.frameSize?
> I am running single node local spark 0.8.1
> Thanks,
> Hussam