You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Pankaj Malhotra <pa...@gmail.com> on 2014/03/06 06:53:33 UTC

Is it possible to know the mapper task a particular vertex is assigned to?

Hi,

How can I find the mapper task a particular vertex is assigned to?
I can do this by doing a sysout and then looking at the logs. But there
must be a smarter way to do this. Please suggest.

Thanks,
Pankaj

Re: Is it possible to know the mapper task a particular vertex is assigned to?

Posted by Pankaj Malhotra <pa...@gmail.com>.
If that is so, then how to find the mapper task which has a particular
vertex?

Thanks,
Pankaj


On 6 March 2014 14:49, Claudio Martella <cl...@gmail.com> wrote:

> by default vertices stay where they are when they are loaded.
>
>
> On Thu, Mar 6, 2014 at 7:31 AM, Pankaj Malhotra <pa...@gmail.com>wrote:
>
>> There is a vertex with a large outgoing edge-list. I wanted to compare
>> the memory usage, number of messages, and few other statistics for the
>> worker with this vertex and the average statistics across workers.
>>
>> Does the mapping change within the same job?
>>
>> Thanks,
>> Pankaj
>>
>>
>> On 6 March 2014 11:38, Roman Shaposhnik <sh...@gmail.com> wrote:
>>
>>> On Wed, Mar 5, 2014 at 9:53 PM, Pankaj Malhotra <pa...@gmail.com>
>>> wrote:
>>> > Hi,
>>> >
>>> > How can I find the mapper task a particular vertex is assigned to?
>>> > I can do this by doing a sysout and then looking at the logs. But
>>> there must
>>> > be a smarter way to do this. Please suggest.
>>>
>>> That mapping is not static and can change. In theory you can rely on
>>> the info in ZK, but that would be relying on what is, essentially, an
>>> implementation detail of Giraph.
>>>
>>> What's the reason for you to need this info?
>>>
>>> Thanks,
>>> Roman.
>>>
>>
>>
>
>
> --
>    Claudio Martella
>
>

Re: Is it possible to know the mapper task a particular vertex is assigned to?

Posted by Claudio Martella <cl...@gmail.com>.
by default vertices stay where they are when they are loaded.


On Thu, Mar 6, 2014 at 7:31 AM, Pankaj Malhotra <pa...@gmail.com>wrote:

> There is a vertex with a large outgoing edge-list. I wanted to compare the
> memory usage, number of messages, and few other statistics for the worker
> with this vertex and the average statistics across workers.
>
> Does the mapping change within the same job?
>
> Thanks,
> Pankaj
>
>
> On 6 March 2014 11:38, Roman Shaposhnik <sh...@gmail.com> wrote:
>
>> On Wed, Mar 5, 2014 at 9:53 PM, Pankaj Malhotra <pa...@gmail.com>
>> wrote:
>> > Hi,
>> >
>> > How can I find the mapper task a particular vertex is assigned to?
>> > I can do this by doing a sysout and then looking at the logs. But there
>> must
>> > be a smarter way to do this. Please suggest.
>>
>> That mapping is not static and can change. In theory you can rely on
>> the info in ZK, but that would be relying on what is, essentially, an
>> implementation detail of Giraph.
>>
>> What's the reason for you to need this info?
>>
>> Thanks,
>> Roman.
>>
>
>


-- 
   Claudio Martella

Re: Is it possible to know the mapper task a particular vertex is assigned to?

Posted by Pankaj Malhotra <pa...@gmail.com>.
There is a vertex with a large outgoing edge-list. I wanted to compare the
memory usage, number of messages, and few other statistics for the worker
with this vertex and the average statistics across workers.

Does the mapping change within the same job?

Thanks,
Pankaj


On 6 March 2014 11:38, Roman Shaposhnik <sh...@gmail.com> wrote:

> On Wed, Mar 5, 2014 at 9:53 PM, Pankaj Malhotra <pa...@gmail.com>
> wrote:
> > Hi,
> >
> > How can I find the mapper task a particular vertex is assigned to?
> > I can do this by doing a sysout and then looking at the logs. But there
> must
> > be a smarter way to do this. Please suggest.
>
> That mapping is not static and can change. In theory you can rely on
> the info in ZK, but that would be relying on what is, essentially, an
> implementation detail of Giraph.
>
> What's the reason for you to need this info?
>
> Thanks,
> Roman.
>

Re: Is it possible to know the mapper task a particular vertex is assigned to?

Posted by Roman Shaposhnik <sh...@gmail.com>.
On Wed, Mar 5, 2014 at 9:53 PM, Pankaj Malhotra <pa...@gmail.com> wrote:
> Hi,
>
> How can I find the mapper task a particular vertex is assigned to?
> I can do this by doing a sysout and then looking at the logs. But there must
> be a smarter way to do this. Please suggest.

That mapping is not static and can change. In theory you can rely on
the info in ZK, but that would be relying on what is, essentially, an
implementation detail of Giraph.

What's the reason for you to need this info?

Thanks,
Roman.