You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Srikrishna S <sr...@gmail.com> on 2014/07/12 17:50:25 UTC
Akka Client disconnected
I am run logistic regression with SGD on a problem with about 19M
parameters (the kdda dataset from the libsvm library)
I consistently see that the nodes on my computer get disconnected and
soon the whole job goes to a grinding halt.
14/07/12 03:05:16 ERROR cluster.YarnClientClusterScheduler: Lost
executor 2 on pachy4 remote Akka client disassociated
Does this have anything to do with the akka.frame_size? I have tried
upto 1024 MB and I still get the same thing.
I don't have any more information in the logs about why the clients
are getting disconnected. Any thoughts?
Regards,
Krishna
Re: Akka Client disconnected
Posted by DB Tsai <db...@dbtsai.com>.
https://issues.apache.org/jira/browse/SPARK-2156
Sincerely,
DB Tsai
-------------------------------------------------------
My Blog: https://www.dbtsai.com
LinkedIn: https://www.linkedin.com/in/dbtsai
On Sat, Jul 12, 2014 at 5:23 PM, Srikrishna S <sr...@gmail.com> wrote:
> I am using the master that I compiled 2 days ago. Can you point me to the JIRA?
>
> On Sat, Jul 12, 2014 at 9:13 AM, DB Tsai <db...@dbtsai.com> wrote:
>> Are you using 1.0 or current master? A bug related to this is fixed in
>> master.
>>
>> On Jul 12, 2014 8:50 AM, "Srikrishna S" <sr...@gmail.com> wrote:
>>>
>>> I am run logistic regression with SGD on a problem with about 19M
>>> parameters (the kdda dataset from the libsvm library)
>>>
>>> I consistently see that the nodes on my computer get disconnected and
>>> soon the whole job goes to a grinding halt.
>>>
>>> 14/07/12 03:05:16 ERROR cluster.YarnClientClusterScheduler: Lost
>>> executor 2 on pachy4 remote Akka client disassociated
>>>
>>> Does this have anything to do with the akka.frame_size? I have tried
>>> upto 1024 MB and I still get the same thing.
>>>
>>> I don't have any more information in the logs about why the clients
>>> are getting disconnected. Any thoughts?
>>>
>>> Regards,
>>> Krishna
Re: Akka Client disconnected
Posted by Srikrishna S <sr...@gmail.com>.
I am using the master that I compiled 2 days ago. Can you point me to the JIRA?
On Sat, Jul 12, 2014 at 9:13 AM, DB Tsai <db...@dbtsai.com> wrote:
> Are you using 1.0 or current master? A bug related to this is fixed in
> master.
>
> On Jul 12, 2014 8:50 AM, "Srikrishna S" <sr...@gmail.com> wrote:
>>
>> I am run logistic regression with SGD on a problem with about 19M
>> parameters (the kdda dataset from the libsvm library)
>>
>> I consistently see that the nodes on my computer get disconnected and
>> soon the whole job goes to a grinding halt.
>>
>> 14/07/12 03:05:16 ERROR cluster.YarnClientClusterScheduler: Lost
>> executor 2 on pachy4 remote Akka client disassociated
>>
>> Does this have anything to do with the akka.frame_size? I have tried
>> upto 1024 MB and I still get the same thing.
>>
>> I don't have any more information in the logs about why the clients
>> are getting disconnected. Any thoughts?
>>
>> Regards,
>> Krishna
Re: Akka Client disconnected
Posted by DB Tsai <db...@dbtsai.com>.
Are you using 1.0 or current master? A bug related to this is fixed in
master.
On Jul 12, 2014 8:50 AM, "Srikrishna S" <sr...@gmail.com> wrote:
> I am run logistic regression with SGD on a problem with about 19M
> parameters (the kdda dataset from the libsvm library)
>
> I consistently see that the nodes on my computer get disconnected and
> soon the whole job goes to a grinding halt.
>
> 14/07/12 03:05:16 ERROR cluster.YarnClientClusterScheduler: Lost
> executor 2 on pachy4 remote Akka client disassociated
>
> Does this have anything to do with the akka.frame_size? I have tried
> upto 1024 MB and I still get the same thing.
>
> I don't have any more information in the logs about why the clients
> are getting disconnected. Any thoughts?
>
> Regards,
> Krishna
>