Re: Is it correct that master runs on a datanode?

[Eli Reisman <ap...@gmail.com>]

Hi Cristina,

The YARN framework decides which node the different processes run on.
Nowadays, the application master is considered just another YARN process
and its not bound to a particular master node the way an HDFS name node
would be. The application master is transient, managing the lifecycle of
each YARN job you run (Hadoop, Giraph, anything) and then cleans up and
goes away when the job is completed.

Hope that helps,
Eli

On Sun, Sep 2, 2018 at 1:31 PM Cristina Bovi <cr...@gmail.com> wrote:

> I apologize again for my trivial question. I understood that hadoop/giraph
> works creating containers on datanodes. Hadoop creates a container for
> application master, then giraph creates a container for the master.
> Furthermore giraph creates a number of container for workers corresponding
> to -w parameter.
>
> Il giorno gio 30 ago 2018 alle ore 15:50 Cristina Bovi <cr...@gmail.com>
> ha scritto:
>
>> I'm using giraph-1.3 built with yarn profile. For starting I configured 1
>> namenode and 2 datanodes on a ec2 cluster.
>> My application properly works, I launched giraph with -w 2 because I have
>> two datanodes.
>>
>> In userlogs of datanode1 I found log of first worker.
>> in userlogs of datanode2 I found log of second worker and log of master
>> too.
>>
>> I expected to find log of master in the namenode i.e. I expected that
>> master runs on namenode. Is it right?
>>
>> Maybe I have to configure another datanode and then I will find master
>> logs on this new datanode?
>>
>> --
>> Cristina Bovi
>>
>
>
> --
> Cristina Bovi
>