You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by 舒琦 <sh...@eefung.com> on 2017/07/17 10:07:41 UTC
Can't start container in newly added hosts
Hi Guys,
After add 3 hosts to yarn cluster, some job can’t startup, then I found some informations as shown below:
2017-07-17 17:37:29.928 [AMRM Callback Handler Thread] YarnClusterResourceManager [INFO] Container allocated from RM on rflow79.antfact.com
2017-07-17 17:37:29.929 [AMRM Callback Handler Thread] YarnClusterResourceManager [INFO] Container allocated from RM on rflow6.antfact.com
2017-07-17 17:37:29.929 [AMRM Callback Handler Thread] YarnClusterResourceManager [INFO] Container allocated from RM on rflow5.antfact.com
2017-07-17 17:37:29.929 [AMRM Callback Handler Thread] ContainerProcessManager [INFO] Container allocated from RM on rflow79.antfact.com
2017-07-17 17:37:29.929 [AMRM Callback Handler Thread] ResourceRequestState [INFO] Host affinity not enabled. Saving the samzaResource container_e38_1500040925911_0028_01_000007 in the buffer for ANY_HOST
2017-07-17 17:37:29.929 [AMRM Callback Handler Thread] ContainerProcessManager [INFO] Container allocated from RM on rflow6.antfact.com
2017-07-17 17:37:29.929 [AMRM Callback Handler Thread] ResourceRequestState [INFO] Host affinity not enabled. Saving the samzaResource container_e38_1500040925911_0028_01_000008 in the buffer for ANY_HOST
2017-07-17 17:37:29.929 [AMRM Callback Handler Thread] ContainerProcessManager [INFO] Container allocated from RM on rflow5.antfact.com
2017-07-17 17:37:29.929 [AMRM Callback Handler Thread] ResourceRequestState [INFO] Host affinity not enabled. Saving the samzaResource container_e38_1500040925911_0028_01_000010 in the buffer for ANY_HOST
What does “Host affinity” mean?
Thanks!
————————
Qi Shu
Re: Can't start container in newly added hosts
Posted by 舒琦 <sh...@eefung.com>.
Hi Jagadish,
Thanks for your help!
————————
Qi Shu
> 在 2017年7月17日，23:17，Jagadish Venkatraman <ja...@gmail.com> 写道：
>
> These log lines are benign, and are at the INFO level.
>
> Host affinity means - Samza will ensure that your containers start up on
> the same host each time and re-use the on-disk snapshot of your RocksDb
> state (instead of bootstrapping the entire state from Kafka's changelog).
> This feature is beneficial for jobs that store a lot of data in RocksDb.
>
> Please refer
> https://samza.apache.org/learn/documentation/0.13/yarn/yarn-host-affinity.html
> for more information.
>
>
> On Mon, Jul 17, 2017 at 3:07 AM, 舒琦 <sh...@eefung.com> wrote:
>
>> Hi Guys,
>>
>> After add 3 hosts to yarn cluster, some job can’t startup, then I
>> found some informations as shown below:
>>
>> 2017-07-17 17:37:29.928 [AMRM Callback Handler Thread]
>> YarnClusterResourceManager [INFO] Container allocated from RM on
>> rflow79.antfact.com
>>
>> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
>> YarnClusterResourceManager [INFO] Container allocated from RM on
>> rflow6.antfact.com
>>
>> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
>> YarnClusterResourceManager [INFO] Container allocated from RM on
>> rflow5.antfact.com
>>
>> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
>> ContainerProcessManager [INFO] Container allocated from RM on
>> rflow79.antfact.com
>>
>> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
>> ResourceRequestState [INFO] Host affinity not enabled. Saving the
>> samzaResource container_e38_1500040925911_0028_01_000007 in the buffer
>> for ANY_HOST
>>
>> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
>> ContainerProcessManager [INFO] Container allocated from RM on
>> rflow6.antfact.com
>>
>> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
>> ResourceRequestState [INFO] Host affinity not enabled. Saving the
>> samzaResource container_e38_1500040925911_0028_01_000008 in the buffer
>> for ANY_HOST
>>
>> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
>> ContainerProcessManager [INFO] Container allocated from RM on
>> rflow5.antfact.com
>>
>> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
>> ResourceRequestState [INFO] Host affinity not enabled. Saving the
>> samzaResource container_e38_1500040925911_0028_01_000010 in the buffer
>> for ANY_HOST
>>
>> What does “Host affinity” mean?
>>
>> Thanks!
>>
>> ————————
>> Qi Shu
>>
>>
>
>
> --
> Jagadish V,
> Graduate Student,
> Department of Computer Science,
> Stanford University
Re: Can't start container in newly added hosts
Posted by Jagadish Venkatraman <ja...@gmail.com>.
These log lines are benign, and are at the INFO level.
Host affinity means - Samza will ensure that your containers start up on
the same host each time and re-use the on-disk snapshot of your RocksDb
state (instead of bootstrapping the entire state from Kafka's changelog).
This feature is beneficial for jobs that store a lot of data in RocksDb.
Please refer
https://samza.apache.org/learn/documentation/0.13/yarn/yarn-host-affinity.html
for more information.
On Mon, Jul 17, 2017 at 3:07 AM, 舒琦 <sh...@eefung.com> wrote:
> Hi Guys,
>
> After add 3 hosts to yarn cluster, some job can’t startup, then I
> found some informations as shown below:
>
> 2017-07-17 17:37:29.928 [AMRM Callback Handler Thread]
> YarnClusterResourceManager [INFO] Container allocated from RM on
> rflow79.antfact.com
>
> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> YarnClusterResourceManager [INFO] Container allocated from RM on
> rflow6.antfact.com
>
> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> YarnClusterResourceManager [INFO] Container allocated from RM on
> rflow5.antfact.com
>
> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> ContainerProcessManager [INFO] Container allocated from RM on
> rflow79.antfact.com
>
> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> ResourceRequestState [INFO] Host affinity not enabled. Saving the
> samzaResource container_e38_1500040925911_0028_01_000007 in the buffer
> for ANY_HOST
>
> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> ContainerProcessManager [INFO] Container allocated from RM on
> rflow6.antfact.com
>
> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> ResourceRequestState [INFO] Host affinity not enabled. Saving the
> samzaResource container_e38_1500040925911_0028_01_000008 in the buffer
> for ANY_HOST
>
> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> ContainerProcessManager [INFO] Container allocated from RM on
> rflow5.antfact.com
>
> 2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> ResourceRequestState [INFO] Host affinity not enabled. Saving the
> samzaResource container_e38_1500040925911_0028_01_000010 in the buffer
> for ANY_HOST
>
> What does “Host affinity” mean?
>
> Thanks!
>
> ————————
> Qi Shu
>
>
--
Jagadish V,
Graduate Student,
Department of Computer Science,
Stanford University