You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hama.apache.org by 顾荣 <gu...@gmail.com> on 2012/09/18 17:52:34 UTC
Question about the "peer.getPeerIndex()" API in Hama
Hi,
I have a Hama cluster of 3 nodes. The host name of the nodes are slave019,
slave020, slave021. I wirte a simple program to test the PeerIndex and
PeerName info in Hama.
In setup function, I choose a peer as master, and the in bsp function each
peer get its PeerIndex and its PeerName by peer.getPeerIndex() and
peer.getPeerName() API and send it to the master peer. Lastly, in cleanup
function of the master peer, it write out the messages in its queue. The
result is below:
peerName: slave020:61001 peerIndex: -7
peerName: slave019:61002 peerIndex: 4
peerName: slave021:61001 peerIndex: 6
peerName: slave019:61001 peerIndex: -1
peerName: slave021:61002 peerIndex: -8
peerName: slave020:61002 peerIndex: -7
peerName: slave019:61003 peerIndex: 5
peerName: slave021:61003 peerIndex: 7
peerName: slave020:61003 peerIndex: -7
The peerNames are reasonable, but I am confused with the peerIndex
information. What does they mean? According to the source code of
BSPPeer.java, the comment message is below
/**
* @return the index of this peer from sorted array by name.
*/
public int getPeerIndex();
However, the peerIndex messages I got seem not to be so....Can you give me
a hint?
Thanks in advance.
Walker
Re: Question about the "peer.getPeerIndex()" API in Hama
Posted by 顾荣 <gu...@gmail.com>.
Oh,thanks for replying, Thomas. That's not a problem. I set up a project
for Hama source in Eclipse now. Thus, when needing, I can conveniently view
the source code of related classes. I find it's also a good way to explore
something in deep of Hama :)
2012/9/19 Thomas Jungblut <th...@gmail.com>
> There should be a doc directory in each of our modules.
> We possibly could add this to our website.
>
> 2012/9/19 顾荣 <gu...@gmail.com>:
> > Hi，
> >
> > BTW, is there a java user API doc for Hama?
> >
> > Regards,
> > Walker
> >
> > 2012/9/19 顾荣 <gu...@gmail.com>
> >
> >> Okay, I got it ,thanks.
> >>
> >>
> >> 2012/9/19 Suraj Menon <su...@apache.org>
> >>
> >>> Hi Walker,
> >>>
> >>> The old contract was changed because if a task gets re-allocated on a
> >>> different machine on failure, that can change the peer-Ids of all the
> >>> tasks.
> >>> The current trunk version returns the task-id, which remains same
> >>> throughout the lifecycle of the job execution.
> >>> With this peerIndex could be made a part of your computation logic.
> >>>
> >>> -Suraj
> >>>
> >>> On Tue, Sep 18, 2012 at 11:59 AM, Thomas Jungblut <
> >>> thomas.jungblut@gmail.com
> >>> > wrote:
> >>>
> >>> > Hi Walker,
> >>> >
> >>> > seems the contract is broken and the array is not sorted anymore.
> >>> >
> >>> > Try to use:
> >>> >
> >>> > String[] names = peer.getAllPeerNames();
> >>> > Arrays.sort(names);
> >>> > int index = Arrays.binarySearch(names, peer.getPeerName());
> >>> >
> >>> > Sorry for the inconvenience, we have replaced it with the task id in
> >>> > the latest trunk, so this will be fixed in the next release.
> >>> >
> >>> > 2012/9/18 顾荣 <gu...@gmail.com>:
> >>> > > Hi,
> >>> > >
> >>> > > I have a Hama cluster of 3 nodes. The host name of the nodes are
> >>> > slave019,
> >>> > > slave020, slave021. I wirte a simple program to test the PeerIndex
> and
> >>> > > PeerName info in Hama.
> >>> > >
> >>> > > In setup function, I choose a peer as master, and the in bsp
> function
> >>> > each
> >>> > > peer get its PeerIndex and its PeerName by peer.getPeerIndex() and
> >>> > > peer.getPeerName() API and send it to the master peer. Lastly, in
> >>> > cleanup
> >>> > > function of the master peer, it write out the messages in its
> queue.
> >>> The
> >>> > > result is below:
> >>> > >
> >>> > > peerName: slave020:61001 peerIndex: -7
> >>> > > peerName: slave019:61002 peerIndex: 4
> >>> > > peerName: slave021:61001 peerIndex: 6
> >>> > > peerName: slave019:61001 peerIndex: -1
> >>> > > peerName: slave021:61002 peerIndex: -8
> >>> > > peerName: slave020:61002 peerIndex: -7
> >>> > > peerName: slave019:61003 peerIndex: 5
> >>> > > peerName: slave021:61003 peerIndex: 7
> >>> > > peerName: slave020:61003 peerIndex: -7
> >>> > >
> >>> > > The peerNames are reasonable, but I am confused with the peerIndex
> >>> > > information. What does they mean? According to the source code of
> >>> > > BSPPeer.java, the comment message is below
> >>> > > /**
> >>> > > * @return the index of this peer from sorted array by name.
> >>> > > */
> >>> > > public int getPeerIndex();
> >>> > >
> >>> > > However, the peerIndex messages I got seem not to be so....Can you
> >>> give
> >>> > me
> >>> > > a hint?
> >>> > >
> >>> > > Thanks in advance.
> >>> > >
> >>> > > Walker
> >>> >
> >>>
> >>
> >>
>
Re: Question about the "peer.getPeerIndex()" API in Hama
Posted by Thomas Jungblut <th...@gmail.com>.
There should be a doc directory in each of our modules.
We possibly could add this to our website.
2012/9/19 顾荣 <gu...@gmail.com>:
> Hi，
>
> BTW, is there a java user API doc for Hama?
>
> Regards,
> Walker
>
> 2012/9/19 顾荣 <gu...@gmail.com>
>
>> Okay, I got it ,thanks.
>>
>>
>> 2012/9/19 Suraj Menon <su...@apache.org>
>>
>>> Hi Walker,
>>>
>>> The old contract was changed because if a task gets re-allocated on a
>>> different machine on failure, that can change the peer-Ids of all the
>>> tasks.
>>> The current trunk version returns the task-id, which remains same
>>> throughout the lifecycle of the job execution.
>>> With this peerIndex could be made a part of your computation logic.
>>>
>>> -Suraj
>>>
>>> On Tue, Sep 18, 2012 at 11:59 AM, Thomas Jungblut <
>>> thomas.jungblut@gmail.com
>>> > wrote:
>>>
>>> > Hi Walker,
>>> >
>>> > seems the contract is broken and the array is not sorted anymore.
>>> >
>>> > Try to use:
>>> >
>>> > String[] names = peer.getAllPeerNames();
>>> > Arrays.sort(names);
>>> > int index = Arrays.binarySearch(names, peer.getPeerName());
>>> >
>>> > Sorry for the inconvenience, we have replaced it with the task id in
>>> > the latest trunk, so this will be fixed in the next release.
>>> >
>>> > 2012/9/18 顾荣 <gu...@gmail.com>:
>>> > > Hi,
>>> > >
>>> > > I have a Hama cluster of 3 nodes. The host name of the nodes are
>>> > slave019,
>>> > > slave020, slave021. I wirte a simple program to test the PeerIndex and
>>> > > PeerName info in Hama.
>>> > >
>>> > > In setup function, I choose a peer as master, and the in bsp function
>>> > each
>>> > > peer get its PeerIndex and its PeerName by peer.getPeerIndex() and
>>> > > peer.getPeerName() API and send it to the master peer. Lastly, in
>>> > cleanup
>>> > > function of the master peer, it write out the messages in its queue.
>>> The
>>> > > result is below:
>>> > >
>>> > > peerName: slave020:61001 peerIndex: -7
>>> > > peerName: slave019:61002 peerIndex: 4
>>> > > peerName: slave021:61001 peerIndex: 6
>>> > > peerName: slave019:61001 peerIndex: -1
>>> > > peerName: slave021:61002 peerIndex: -8
>>> > > peerName: slave020:61002 peerIndex: -7
>>> > > peerName: slave019:61003 peerIndex: 5
>>> > > peerName: slave021:61003 peerIndex: 7
>>> > > peerName: slave020:61003 peerIndex: -7
>>> > >
>>> > > The peerNames are reasonable, but I am confused with the peerIndex
>>> > > information. What does they mean? According to the source code of
>>> > > BSPPeer.java, the comment message is below
>>> > > /**
>>> > > * @return the index of this peer from sorted array by name.
>>> > > */
>>> > > public int getPeerIndex();
>>> > >
>>> > > However, the peerIndex messages I got seem not to be so....Can you
>>> give
>>> > me
>>> > > a hint?
>>> > >
>>> > > Thanks in advance.
>>> > >
>>> > > Walker
>>> >
>>>
>>
>>
Re: Question about the "peer.getPeerIndex()" API in Hama
Posted by 顾荣 <gu...@gmail.com>.
Hi，
BTW, is there a java user API doc for Hama?
Regards,
Walker
2012/9/19 顾荣 <gu...@gmail.com>
> Okay, I got it ,thanks.
>
>
> 2012/9/19 Suraj Menon <su...@apache.org>
>
>> Hi Walker,
>>
>> The old contract was changed because if a task gets re-allocated on a
>> different machine on failure, that can change the peer-Ids of all the
>> tasks.
>> The current trunk version returns the task-id, which remains same
>> throughout the lifecycle of the job execution.
>> With this peerIndex could be made a part of your computation logic.
>>
>> -Suraj
>>
>> On Tue, Sep 18, 2012 at 11:59 AM, Thomas Jungblut <
>> thomas.jungblut@gmail.com
>> > wrote:
>>
>> > Hi Walker,
>> >
>> > seems the contract is broken and the array is not sorted anymore.
>> >
>> > Try to use:
>> >
>> > String[] names = peer.getAllPeerNames();
>> > Arrays.sort(names);
>> > int index = Arrays.binarySearch(names, peer.getPeerName());
>> >
>> > Sorry for the inconvenience, we have replaced it with the task id in
>> > the latest trunk, so this will be fixed in the next release.
>> >
>> > 2012/9/18 顾荣 <gu...@gmail.com>:
>> > > Hi,
>> > >
>> > > I have a Hama cluster of 3 nodes. The host name of the nodes are
>> > slave019,
>> > > slave020, slave021. I wirte a simple program to test the PeerIndex and
>> > > PeerName info in Hama.
>> > >
>> > > In setup function, I choose a peer as master, and the in bsp function
>> > each
>> > > peer get its PeerIndex and its PeerName by peer.getPeerIndex() and
>> > > peer.getPeerName() API and send it to the master peer. Lastly, in
>> > cleanup
>> > > function of the master peer, it write out the messages in its queue.
>> The
>> > > result is below:
>> > >
>> > > peerName: slave020:61001 peerIndex: -7
>> > > peerName: slave019:61002 peerIndex: 4
>> > > peerName: slave021:61001 peerIndex: 6
>> > > peerName: slave019:61001 peerIndex: -1
>> > > peerName: slave021:61002 peerIndex: -8
>> > > peerName: slave020:61002 peerIndex: -7
>> > > peerName: slave019:61003 peerIndex: 5
>> > > peerName: slave021:61003 peerIndex: 7
>> > > peerName: slave020:61003 peerIndex: -7
>> > >
>> > > The peerNames are reasonable, but I am confused with the peerIndex
>> > > information. What does they mean? According to the source code of
>> > > BSPPeer.java, the comment message is below
>> > > /**
>> > > * @return the index of this peer from sorted array by name.
>> > > */
>> > > public int getPeerIndex();
>> > >
>> > > However, the peerIndex messages I got seem not to be so....Can you
>> give
>> > me
>> > > a hint?
>> > >
>> > > Thanks in advance.
>> > >
>> > > Walker
>> >
>>
>
>
Re: Question about the "peer.getPeerIndex()" API in Hama
Posted by 顾荣 <gu...@gmail.com>.
Okay, I got it ,thanks.
2012/9/19 Suraj Menon <su...@apache.org>
> Hi Walker,
>
> The old contract was changed because if a task gets re-allocated on a
> different machine on failure, that can change the peer-Ids of all the
> tasks.
> The current trunk version returns the task-id, which remains same
> throughout the lifecycle of the job execution.
> With this peerIndex could be made a part of your computation logic.
>
> -Suraj
>
> On Tue, Sep 18, 2012 at 11:59 AM, Thomas Jungblut <
> thomas.jungblut@gmail.com
> > wrote:
>
> > Hi Walker,
> >
> > seems the contract is broken and the array is not sorted anymore.
> >
> > Try to use:
> >
> > String[] names = peer.getAllPeerNames();
> > Arrays.sort(names);
> > int index = Arrays.binarySearch(names, peer.getPeerName());
> >
> > Sorry for the inconvenience, we have replaced it with the task id in
> > the latest trunk, so this will be fixed in the next release.
> >
> > 2012/9/18 顾荣 <gu...@gmail.com>:
> > > Hi,
> > >
> > > I have a Hama cluster of 3 nodes. The host name of the nodes are
> > slave019,
> > > slave020, slave021. I wirte a simple program to test the PeerIndex and
> > > PeerName info in Hama.
> > >
> > > In setup function, I choose a peer as master, and the in bsp function
> > each
> > > peer get its PeerIndex and its PeerName by peer.getPeerIndex() and
> > > peer.getPeerName() API and send it to the master peer. Lastly, in
> > cleanup
> > > function of the master peer, it write out the messages in its queue.
> The
> > > result is below:
> > >
> > > peerName: slave020:61001 peerIndex: -7
> > > peerName: slave019:61002 peerIndex: 4
> > > peerName: slave021:61001 peerIndex: 6
> > > peerName: slave019:61001 peerIndex: -1
> > > peerName: slave021:61002 peerIndex: -8
> > > peerName: slave020:61002 peerIndex: -7
> > > peerName: slave019:61003 peerIndex: 5
> > > peerName: slave021:61003 peerIndex: 7
> > > peerName: slave020:61003 peerIndex: -7
> > >
> > > The peerNames are reasonable, but I am confused with the peerIndex
> > > information. What does they mean? According to the source code of
> > > BSPPeer.java, the comment message is below
> > > /**
> > > * @return the index of this peer from sorted array by name.
> > > */
> > > public int getPeerIndex();
> > >
> > > However, the peerIndex messages I got seem not to be so....Can you give
> > me
> > > a hint?
> > >
> > > Thanks in advance.
> > >
> > > Walker
> >
>
Re: Question about the "peer.getPeerIndex()" API in Hama
Posted by Suraj Menon <su...@apache.org>.
Hi Walker,
The old contract was changed because if a task gets re-allocated on a
different machine on failure, that can change the peer-Ids of all the
tasks.
The current trunk version returns the task-id, which remains same
throughout the lifecycle of the job execution.
With this peerIndex could be made a part of your computation logic.
-Suraj
On Tue, Sep 18, 2012 at 11:59 AM, Thomas Jungblut <thomas.jungblut@gmail.com
> wrote:
> Hi Walker,
>
> seems the contract is broken and the array is not sorted anymore.
>
> Try to use:
>
> String[] names = peer.getAllPeerNames();
> Arrays.sort(names);
> int index = Arrays.binarySearch(names, peer.getPeerName());
>
> Sorry for the inconvenience, we have replaced it with the task id in
> the latest trunk, so this will be fixed in the next release.
>
> 2012/9/18 顾荣 <gu...@gmail.com>:
> > Hi,
> >
> > I have a Hama cluster of 3 nodes. The host name of the nodes are
> slave019,
> > slave020, slave021. I wirte a simple program to test the PeerIndex and
> > PeerName info in Hama.
> >
> > In setup function, I choose a peer as master, and the in bsp function
> each
> > peer get its PeerIndex and its PeerName by peer.getPeerIndex() and
> > peer.getPeerName() API and send it to the master peer. Lastly, in
> cleanup
> > function of the master peer, it write out the messages in its queue. The
> > result is below:
> >
> > peerName: slave020:61001 peerIndex: -7
> > peerName: slave019:61002 peerIndex: 4
> > peerName: slave021:61001 peerIndex: 6
> > peerName: slave019:61001 peerIndex: -1
> > peerName: slave021:61002 peerIndex: -8
> > peerName: slave020:61002 peerIndex: -7
> > peerName: slave019:61003 peerIndex: 5
> > peerName: slave021:61003 peerIndex: 7
> > peerName: slave020:61003 peerIndex: -7
> >
> > The peerNames are reasonable, but I am confused with the peerIndex
> > information. What does they mean? According to the source code of
> > BSPPeer.java, the comment message is below
> > /**
> > * @return the index of this peer from sorted array by name.
> > */
> > public int getPeerIndex();
> >
> > However, the peerIndex messages I got seem not to be so....Can you give
> me
> > a hint?
> >
> > Thanks in advance.
> >
> > Walker
>
Re: Question about the "peer.getPeerIndex()" API in Hama
Posted by Thomas Jungblut <th...@gmail.com>.
Hi Walker,
seems the contract is broken and the array is not sorted anymore.
Try to use:
String[] names = peer.getAllPeerNames();
Arrays.sort(names);
int index = Arrays.binarySearch(names, peer.getPeerName());
Sorry for the inconvenience, we have replaced it with the task id in
the latest trunk, so this will be fixed in the next release.
2012/9/18 顾荣 <gu...@gmail.com>:
> Hi,
>
> I have a Hama cluster of 3 nodes. The host name of the nodes are slave019,
> slave020, slave021. I wirte a simple program to test the PeerIndex and
> PeerName info in Hama.
>
> In setup function, I choose a peer as master, and the in bsp function each
> peer get its PeerIndex and its PeerName by peer.getPeerIndex() and
> peer.getPeerName() API and send it to the master peer. Lastly, in cleanup
> function of the master peer, it write out the messages in its queue. The
> result is below:
>
> peerName: slave020:61001 peerIndex: -7
> peerName: slave019:61002 peerIndex: 4
> peerName: slave021:61001 peerIndex: 6
> peerName: slave019:61001 peerIndex: -1
> peerName: slave021:61002 peerIndex: -8
> peerName: slave020:61002 peerIndex: -7
> peerName: slave019:61003 peerIndex: 5
> peerName: slave021:61003 peerIndex: 7
> peerName: slave020:61003 peerIndex: -7
>
> The peerNames are reasonable, but I am confused with the peerIndex
> information. What does they mean? According to the source code of
> BSPPeer.java, the comment message is below
> /**
> * @return the index of this peer from sorted array by name.
> */
> public int getPeerIndex();
>
> However, the peerIndex messages I got seem not to be so....Can you give me
> a hint?
>
> Thanks in advance.
>
> Walker