You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-user@hadoop.apache.org by Gaurav Gupta <ga...@datatorrent.com> on 2014/02/27 19:33:17 UTC

RM AM_RESYNC signal to AM

Hi,

 

I killed the node manager on the node where AM was running and the AM master
got the AM_RESYNC command signal from RM. I have following questions

1.      In what all scenarios does the RM sends AM_RESYNC signal to AM?

2.      Should the RM not send the AM_SHUTDOWN signal to AM when node
manager is killed?

 

Thanks

-Gaurav

RE: RM AM_RESYNC signal to AM

Posted by Rohith Sharma K S <ro...@huawei.com>.

Hi Gaurav

If NodeManage is killed, then containers running on this NM won't be killed immediately. RM holds node information for 10 minutes(default node expiry). Possibly there should be

1.       After 10 minutes , container is killed.

2.       NM is killed and restarted before 10 minutes.


1.       In what all scenarios does the RM sends AM_RESYNC signal to AM?
        >>>  In two scenario's RM sends AM_RESYNC to AM.

a.       When there is responseID mismatch.  AM sends response id to RM in registration and every heart beat. RM validate responseId in every heartbeat sent by AM.

b.       When application attempts does not exist in RM cache. In your case, this scenario might be occurring. When NM is killed, it removed all the attempt data from RM. But still appliclation master is trying to connect RM.



2.       Should the RM not send the AM_SHUTDOWN signal to AM when node manager is killed?

>> As such AM_SHUTDOWN is NOT sent from RM. Community may be planning improvement on this.



Thanks & Regards
Rohith Sharma K S


From: Gaurav Gupta [mailto:gaurav@datatorrent.com]
Sent: 28 February 2014 00:03
To: user@hadoop.apache.org
Subject: RM AM_RESYNC signal to AM

Hi,

I killed the node manager on the node where AM was running and the AM master got the AM_RESYNC command signal from RM. I have following questions

3.       In what all scenarios does the RM sends AM_RESYNC signal to AM?

4.       Should the RM not send the AM_SHUTDOWN signal to AM when node manager is killed?

Thanks
-Gaurav

RE: RM AM_RESYNC signal to AM

Posted by Rohith Sharma K S <ro...@huawei.com>.

Hi Gaurav

If NodeManage is killed, then containers running on this NM won't be killed immediately. RM holds node information for 10 minutes(default node expiry). Possibly there should be

1.       After 10 minutes , container is killed.

2.       NM is killed and restarted before 10 minutes.


1.       In what all scenarios does the RM sends AM_RESYNC signal to AM?
        >>>  In two scenario's RM sends AM_RESYNC to AM.

a.       When there is responseID mismatch.  AM sends response id to RM in registration and every heart beat. RM validate responseId in every heartbeat sent by AM.

b.       When application attempts does not exist in RM cache. In your case, this scenario might be occurring. When NM is killed, it removed all the attempt data from RM. But still appliclation master is trying to connect RM.



2.       Should the RM not send the AM_SHUTDOWN signal to AM when node manager is killed?

>> As such AM_SHUTDOWN is NOT sent from RM. Community may be planning improvement on this.



Thanks & Regards
Rohith Sharma K S


From: Gaurav Gupta [mailto:gaurav@datatorrent.com]
Sent: 28 February 2014 00:03
To: user@hadoop.apache.org
Subject: RM AM_RESYNC signal to AM

Hi,

I killed the node manager on the node where AM was running and the AM master got the AM_RESYNC command signal from RM. I have following questions

3.       In what all scenarios does the RM sends AM_RESYNC signal to AM?

4.       Should the RM not send the AM_SHUTDOWN signal to AM when node manager is killed?

Thanks
-Gaurav

RE: RM AM_RESYNC signal to AM

Posted by Rohith Sharma K S <ro...@huawei.com>.

Hi Gaurav

If NodeManage is killed, then containers running on this NM won't be killed immediately. RM holds node information for 10 minutes(default node expiry). Possibly there should be

1.       After 10 minutes , container is killed.

2.       NM is killed and restarted before 10 minutes.


1.       In what all scenarios does the RM sends AM_RESYNC signal to AM?
        >>>  In two scenario's RM sends AM_RESYNC to AM.

a.       When there is responseID mismatch.  AM sends response id to RM in registration and every heart beat. RM validate responseId in every heartbeat sent by AM.

b.       When application attempts does not exist in RM cache. In your case, this scenario might be occurring. When NM is killed, it removed all the attempt data from RM. But still appliclation master is trying to connect RM.



2.       Should the RM not send the AM_SHUTDOWN signal to AM when node manager is killed?

>> As such AM_SHUTDOWN is NOT sent from RM. Community may be planning improvement on this.



Thanks & Regards
Rohith Sharma K S


From: Gaurav Gupta [mailto:gaurav@datatorrent.com]
Sent: 28 February 2014 00:03
To: user@hadoop.apache.org
Subject: RM AM_RESYNC signal to AM

Hi,

I killed the node manager on the node where AM was running and the AM master got the AM_RESYNC command signal from RM. I have following questions

3.       In what all scenarios does the RM sends AM_RESYNC signal to AM?

4.       Should the RM not send the AM_SHUTDOWN signal to AM when node manager is killed?

Thanks
-Gaurav

RE: RM AM_RESYNC signal to AM

Posted by Rohith Sharma K S <ro...@huawei.com>.

Hi Gaurav

If NodeManage is killed, then containers running on this NM won't be killed immediately. RM holds node information for 10 minutes(default node expiry). Possibly there should be

1.       After 10 minutes , container is killed.

2.       NM is killed and restarted before 10 minutes.


1.       In what all scenarios does the RM sends AM_RESYNC signal to AM?
        >>>  In two scenario's RM sends AM_RESYNC to AM.

a.       When there is responseID mismatch.  AM sends response id to RM in registration and every heart beat. RM validate responseId in every heartbeat sent by AM.

b.       When application attempts does not exist in RM cache. In your case, this scenario might be occurring. When NM is killed, it removed all the attempt data from RM. But still appliclation master is trying to connect RM.



2.       Should the RM not send the AM_SHUTDOWN signal to AM when node manager is killed?

>> As such AM_SHUTDOWN is NOT sent from RM. Community may be planning improvement on this.



Thanks & Regards
Rohith Sharma K S


From: Gaurav Gupta [mailto:gaurav@datatorrent.com]
Sent: 28 February 2014 00:03
To: user@hadoop.apache.org
Subject: RM AM_RESYNC signal to AM

Hi,

I killed the node manager on the node where AM was running and the AM master got the AM_RESYNC command signal from RM. I have following questions

3.       In what all scenarios does the RM sends AM_RESYNC signal to AM?

4.       Should the RM not send the AM_SHUTDOWN signal to AM when node manager is killed?

Thanks
-Gaurav