You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@ambari.apache.org by Dmytro Sen <ds...@hortonworks.com> on 2014/03/13 13:23:48 UTC

Review Request 19185: Rolling restart should also handle clients on the same machine as the restarting component

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19185/
-----------------------------------------------------------

Review request for Ambari and Sid Wagle.

Bugs: AMBARI-5065
https://issues.apache.org/jira/browse/AMBARI-5065

Repository: ambari

Description
-------

3-host cluster, c6401 has nn, c6402 and c6403 have datanodes + hdfs clients
1. create host group in Hdfs
2. put one host in the group (say c6403) that has datanode and hdfs client
3. change datanode max java heap size so it only affects the datanode
4. you'll see a restart is needed only on c6403 as expected (correct) for datanode and hdfs client
5. Issue a rolling restart and only on hosts that need configs (so just one host)
6. the works fine and it only restarts datanode c6403 as expected (correct)
problem: hdfs client is not marked as restarted so i still see "1 HDFS Client" component requiring a restart on that host.
Need to handle clients on the machines as well as part of rolling restart.
Solution:
Update client tags on Master/Slave component update in-memory and in individual files.

Diffs
-----

ambari-agent/src/main/python/ambari_agent/ActionQueue.py 731ac54
ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py 024c575
ambari-agent/src/main/python/ambari_agent/LiveStatus.py 6f9f4db
ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py e62dd1a
ambari-agent/src/test/python/ambari_agent/TestLiveStatus.py 96a21b9

Diff: https://reviews.apache.org/r/19185/diff/

Testing
-------

Ran 284 tests in 11.538s

Thanks,

Dmytro Sen

Re: Review Request 19185: Rolling restart should also handle clients on the same machine as the restarting component

Posted by Dmytro Sen <ds...@hortonworks.com>.


> On March 14, 2014, 4:34 p.m., Sumit Mohanty wrote:
> > Dmytro Sen, why would the command['configurationTags'] be None?

Don't know. Actually, before this patch configurationTags as well were copied from config.json to DATANODE_config.json, but not stored from command['configurationTags'] as I tried to do.


- Dmytro


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19185/#review37213
-----------------------------------------------------------


On March 14, 2014, 2:56 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19185/
> -----------------------------------------------------------
> 
> (Updated March 14, 2014, 2:56 p.m.)
> 
> 
> Review request for Ambari and Sid Wagle.
> 
> 
> Bugs: AMBARI-5065
>     https://issues.apache.org/jira/browse/AMBARI-5065
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 3-host cluster, c6401 has nn, c6402 and c6403 have datanodes + hdfs clients
> 1. create host group in Hdfs
> 2. put one host in the group (say c6403) that has datanode and hdfs client
> 3. change datanode max java heap size so it only affects the datanode
> 4. you'll see a restart is needed only on c6403 as expected (correct) for datanode and hdfs client
> 5. Issue a rolling restart and only on hosts that need configs (so just one host)
> 6. the works fine and it only restarts datanode c6403 as expected (correct)
> problem: hdfs client is not marked as restarted so i still see "1 HDFS Client" component requiring a restart on that host.
> Need to handle clients on the machines as well as part of rolling restart.
> Solution:
> Update client tags on Master/Slave component update in-memory and in individual files.
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 549651a 
>   ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py 024c575 
>   ambari-agent/src/main/python/ambari_agent/LiveStatus.py 6f9f4db 
>   ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py e62dd1a 
>   ambari-agent/src/test/python/ambari_agent/TestLiveStatus.py 96a21b9 
> 
> Diff: https://reviews.apache.org/r/19185/diff/
> 
> 
> Testing
> -------
> 
> Ran 284 tests in 11.538s
> 
> OK
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 19185: Rolling restart should also handle clients on the same machine as the restarting component

Posted by Sumit Mohanty <sm...@hortonworks.com>.


> On March 14, 2014, 4:34 p.m., Sumit Mohanty wrote:
> > Dmytro Sen, why would the command['configurationTags'] be None?
> 
> Dmytro Sen wrote:
>     Don't know. Actually, before this patch configurationTags as well were copied from config.json to DATANODE_config.json, but not stored from command['configurationTags'] as I tried to do.

Can you also write a short summary of how the overall fix works?


- Sumit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19185/#review37213
-----------------------------------------------------------


On March 14, 2014, 2:56 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19185/
> -----------------------------------------------------------
> 
> (Updated March 14, 2014, 2:56 p.m.)
> 
> 
> Review request for Ambari and Sid Wagle.
> 
> 
> Bugs: AMBARI-5065
>     https://issues.apache.org/jira/browse/AMBARI-5065
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 3-host cluster, c6401 has nn, c6402 and c6403 have datanodes + hdfs clients
> 1. create host group in Hdfs
> 2. put one host in the group (say c6403) that has datanode and hdfs client
> 3. change datanode max java heap size so it only affects the datanode
> 4. you'll see a restart is needed only on c6403 as expected (correct) for datanode and hdfs client
> 5. Issue a rolling restart and only on hosts that need configs (so just one host)
> 6. the works fine and it only restarts datanode c6403 as expected (correct)
> problem: hdfs client is not marked as restarted so i still see "1 HDFS Client" component requiring a restart on that host.
> Need to handle clients on the machines as well as part of rolling restart.
> Solution:
> Update client tags on Master/Slave component update in-memory and in individual files.
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 549651a 
>   ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py 024c575 
>   ambari-agent/src/main/python/ambari_agent/LiveStatus.py 6f9f4db 
>   ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py e62dd1a 
>   ambari-agent/src/test/python/ambari_agent/TestLiveStatus.py 96a21b9 
> 
> Diff: https://reviews.apache.org/r/19185/diff/
> 
> 
> Testing
> -------
> 
> Ran 284 tests in 11.538s
> 
> OK
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 19185: Rolling restart should also handle clients on the same machine as the restarting component

Posted by Sumit Mohanty <sm...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19185/#review37213
-----------------------------------------------------------


Dmytro Sen, why would the command['configurationTags'] be None?

- Sumit Mohanty


On March 14, 2014, 2:56 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19185/
> -----------------------------------------------------------
> 
> (Updated March 14, 2014, 2:56 p.m.)
> 
> 
> Review request for Ambari and Sid Wagle.
> 
> 
> Bugs: AMBARI-5065
>     https://issues.apache.org/jira/browse/AMBARI-5065
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 3-host cluster, c6401 has nn, c6402 and c6403 have datanodes + hdfs clients
> 1. create host group in Hdfs
> 2. put one host in the group (say c6403) that has datanode and hdfs client
> 3. change datanode max java heap size so it only affects the datanode
> 4. you'll see a restart is needed only on c6403 as expected (correct) for datanode and hdfs client
> 5. Issue a rolling restart and only on hosts that need configs (so just one host)
> 6. the works fine and it only restarts datanode c6403 as expected (correct)
> problem: hdfs client is not marked as restarted so i still see "1 HDFS Client" component requiring a restart on that host.
> Need to handle clients on the machines as well as part of rolling restart.
> Solution:
> Update client tags on Master/Slave component update in-memory and in individual files.
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 549651a 
>   ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py 024c575 
>   ambari-agent/src/main/python/ambari_agent/LiveStatus.py 6f9f4db 
>   ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py e62dd1a 
>   ambari-agent/src/test/python/ambari_agent/TestLiveStatus.py 96a21b9 
> 
> Diff: https://reviews.apache.org/r/19185/diff/
> 
> 
> Testing
> -------
> 
> Ran 284 tests in 11.538s
> 
> OK
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 19185: Rolling restart should also handle clients on the same machine as the restarting component

Posted by Sid Wagle <sw...@hortonworks.com>.


> On March 14, 2014, 5:30 p.m., Sumit Mohanty wrote:
> > ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py, line 104
> > <https://reviews.apache.org/r/19185/diff/2/?file=519635#file519635line104>
> >
> >     Should we assert that HBASE_CLIENT is not tag2?

Agree with Sumit's comment about the assertion to make sure HBASE_CLIENT is not updated. 


- Sid


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19185/#review37226
-----------------------------------------------------------


On March 14, 2014, 2:56 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19185/
> -----------------------------------------------------------
> 
> (Updated March 14, 2014, 2:56 p.m.)
> 
> 
> Review request for Ambari and Sid Wagle.
> 
> 
> Bugs: AMBARI-5065
>     https://issues.apache.org/jira/browse/AMBARI-5065
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 3-host cluster, c6401 has nn, c6402 and c6403 have datanodes + hdfs clients
> 1. create host group in Hdfs
> 2. put one host in the group (say c6403) that has datanode and hdfs client
> 3. change datanode max java heap size so it only affects the datanode
> 4. you'll see a restart is needed only on c6403 as expected (correct) for datanode and hdfs client
> 5. Issue a rolling restart and only on hosts that need configs (so just one host)
> 6. the works fine and it only restarts datanode c6403 as expected (correct)
> problem: hdfs client is not marked as restarted so i still see "1 HDFS Client" component requiring a restart on that host.
> Need to handle clients on the machines as well as part of rolling restart.
> Solution:
> Update client tags on Master/Slave component update in-memory and in individual files.
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 549651a 
>   ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py 024c575 
>   ambari-agent/src/main/python/ambari_agent/LiveStatus.py 6f9f4db 
>   ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py e62dd1a 
>   ambari-agent/src/test/python/ambari_agent/TestLiveStatus.py 96a21b9 
> 
> Diff: https://reviews.apache.org/r/19185/diff/
> 
> 
> Testing
> -------
> 
> Ran 284 tests in 11.538s
> 
> OK
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 19185: Rolling restart should also handle clients on the same machine as the restarting component

Posted by Sumit Mohanty <sm...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19185/#review37226
-----------------------------------------------------------



ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py
<https://reviews.apache.org/r/19185/#comment68646>

    Should we assert that HBASE_CLIENT is not tag2?


- Sumit Mohanty


On March 14, 2014, 2:56 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19185/
> -----------------------------------------------------------
> 
> (Updated March 14, 2014, 2:56 p.m.)
> 
> 
> Review request for Ambari and Sid Wagle.
> 
> 
> Bugs: AMBARI-5065
>     https://issues.apache.org/jira/browse/AMBARI-5065
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 3-host cluster, c6401 has nn, c6402 and c6403 have datanodes + hdfs clients
> 1. create host group in Hdfs
> 2. put one host in the group (say c6403) that has datanode and hdfs client
> 3. change datanode max java heap size so it only affects the datanode
> 4. you'll see a restart is needed only on c6403 as expected (correct) for datanode and hdfs client
> 5. Issue a rolling restart and only on hosts that need configs (so just one host)
> 6. the works fine and it only restarts datanode c6403 as expected (correct)
> problem: hdfs client is not marked as restarted so i still see "1 HDFS Client" component requiring a restart on that host.
> Need to handle clients on the machines as well as part of rolling restart.
> Solution:
> Update client tags on Master/Slave component update in-memory and in individual files.
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 549651a 
>   ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py 024c575 
>   ambari-agent/src/main/python/ambari_agent/LiveStatus.py 6f9f4db 
>   ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py e62dd1a 
>   ambari-agent/src/test/python/ambari_agent/TestLiveStatus.py 96a21b9 
> 
> Diff: https://reviews.apache.org/r/19185/diff/
> 
> 
> Testing
> -------
> 
> Ran 284 tests in 11.538s
> 
> OK
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 19185: Rolling restart should also handle clients on the same machine as the restarting component

Posted by Sid Wagle <sw...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19185/#review37207
-----------------------------------------------------------

Ship it!


Ship It!

- Sid Wagle


On March 14, 2014, 2:56 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19185/
> -----------------------------------------------------------
> 
> (Updated March 14, 2014, 2:56 p.m.)
> 
> 
> Review request for Ambari and Sid Wagle.
> 
> 
> Bugs: AMBARI-5065
>     https://issues.apache.org/jira/browse/AMBARI-5065
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 3-host cluster, c6401 has nn, c6402 and c6403 have datanodes + hdfs clients
> 1. create host group in Hdfs
> 2. put one host in the group (say c6403) that has datanode and hdfs client
> 3. change datanode max java heap size so it only affects the datanode
> 4. you'll see a restart is needed only on c6403 as expected (correct) for datanode and hdfs client
> 5. Issue a rolling restart and only on hosts that need configs (so just one host)
> 6. the works fine and it only restarts datanode c6403 as expected (correct)
> problem: hdfs client is not marked as restarted so i still see "1 HDFS Client" component requiring a restart on that host.
> Need to handle clients on the machines as well as part of rolling restart.
> Solution:
> Update client tags on Master/Slave component update in-memory and in individual files.
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 549651a 
>   ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py 024c575 
>   ambari-agent/src/main/python/ambari_agent/LiveStatus.py 6f9f4db 
>   ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py e62dd1a 
>   ambari-agent/src/test/python/ambari_agent/TestLiveStatus.py 96a21b9 
> 
> Diff: https://reviews.apache.org/r/19185/diff/
> 
> 
> Testing
> -------
> 
> Ran 284 tests in 11.538s
> 
> OK
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 19185: Rolling restart should also handle clients on the same machine as the restarting component

Posted by Sumit Mohanty <sm...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19185/#review37460
-----------------------------------------------------------

Ship it!


Ship It!

- Sumit Mohanty


On March 17, 2014, 3:21 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19185/
> -----------------------------------------------------------
> 
> (Updated March 17, 2014, 3:21 p.m.)
> 
> 
> Review request for Ambari and Sid Wagle.
> 
> 
> Bugs: AMBARI-5065
>     https://issues.apache.org/jira/browse/AMBARI-5065
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 3-host cluster, c6401 has nn, c6402 and c6403 have datanodes + hdfs clients
> 1. create host group in Hdfs
> 2. put one host in the group (say c6403) that has datanode and hdfs client
> 3. change datanode max java heap size so it only affects the datanode
> 4. you'll see a restart is needed only on c6403 as expected (correct) for datanode and hdfs client
> 5. Issue a rolling restart and only on hosts that need configs (so just one host)
> 6. the works fine and it only restarts datanode c6403 as expected (correct)
> problem: hdfs client is not marked as restarted so i still see "1 HDFS Client" component requiring a restart on that host.
> Need to handle clients on the machines as well as part of rolling restart.
> Solution:
> Update client tags on Master/Slave component update in-memory and in individual files.
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 549651a 
>   ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py 024c575 
>   ambari-agent/src/main/python/ambari_agent/LiveStatus.py 6f9f4db 
>   ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py e62dd1a 
>   ambari-agent/src/test/python/ambari_agent/TestLiveStatus.py 96a21b9 
> 
> Diff: https://reviews.apache.org/r/19185/diff/
> 
> 
> Testing
> -------
> 
> Ran 284 tests in 11.538s
> 
> OK
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 19185: Rolling restart should also handle clients on the same machine as the restarting component

Posted by Sumit Mohanty <sm...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19185/#review37459
-----------------------------------------------------------



ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py
<https://reviews.apache.org/r/19185/#comment68993>

    Can you add an assert that HBASE_CLIENT 's config did not change to "tag2"?


- Sumit Mohanty


On March 17, 2014, 3:21 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19185/
> -----------------------------------------------------------
> 
> (Updated March 17, 2014, 3:21 p.m.)
> 
> 
> Review request for Ambari and Sid Wagle.
> 
> 
> Bugs: AMBARI-5065
>     https://issues.apache.org/jira/browse/AMBARI-5065
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 3-host cluster, c6401 has nn, c6402 and c6403 have datanodes + hdfs clients
> 1. create host group in Hdfs
> 2. put one host in the group (say c6403) that has datanode and hdfs client
> 3. change datanode max java heap size so it only affects the datanode
> 4. you'll see a restart is needed only on c6403 as expected (correct) for datanode and hdfs client
> 5. Issue a rolling restart and only on hosts that need configs (so just one host)
> 6. the works fine and it only restarts datanode c6403 as expected (correct)
> problem: hdfs client is not marked as restarted so i still see "1 HDFS Client" component requiring a restart on that host.
> Need to handle clients on the machines as well as part of rolling restart.
> Solution:
> Update client tags on Master/Slave component update in-memory and in individual files.
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 549651a 
>   ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py 024c575 
>   ambari-agent/src/main/python/ambari_agent/LiveStatus.py 6f9f4db 
>   ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py e62dd1a 
>   ambari-agent/src/test/python/ambari_agent/TestLiveStatus.py 96a21b9 
> 
> Diff: https://reviews.apache.org/r/19185/diff/
> 
> 
> Testing
> -------
> 
> Ran 284 tests in 11.538s
> 
> OK
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 19185: Rolling restart should also handle clients on the same machine as the restarting component

Posted by Sid Wagle <sw...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19185/#review37413
-----------------------------------------------------------

Ship it!


BTW, do ambari really has to store contigurationTags for the client components, if they are updated automatic
- We need this for hosts with Client only hosts with no co-hosted Master / Slave.

- Sid Wagle


On March 17, 2014, 3:21 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19185/
> -----------------------------------------------------------
> 
> (Updated March 17, 2014, 3:21 p.m.)
> 
> 
> Review request for Ambari and Sid Wagle.
> 
> 
> Bugs: AMBARI-5065
>     https://issues.apache.org/jira/browse/AMBARI-5065
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 3-host cluster, c6401 has nn, c6402 and c6403 have datanodes + hdfs clients
> 1. create host group in Hdfs
> 2. put one host in the group (say c6403) that has datanode and hdfs client
> 3. change datanode max java heap size so it only affects the datanode
> 4. you'll see a restart is needed only on c6403 as expected (correct) for datanode and hdfs client
> 5. Issue a rolling restart and only on hosts that need configs (so just one host)
> 6. the works fine and it only restarts datanode c6403 as expected (correct)
> problem: hdfs client is not marked as restarted so i still see "1 HDFS Client" component requiring a restart on that host.
> Need to handle clients on the machines as well as part of rolling restart.
> Solution:
> Update client tags on Master/Slave component update in-memory and in individual files.
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 549651a 
>   ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py 024c575 
>   ambari-agent/src/main/python/ambari_agent/LiveStatus.py 6f9f4db 
>   ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py e62dd1a 
>   ambari-agent/src/test/python/ambari_agent/TestLiveStatus.py 96a21b9 
> 
> Diff: https://reviews.apache.org/r/19185/diff/
> 
> 
> Testing
> -------
> 
> Ran 284 tests in 11.538s
> 
> OK
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 19185: Rolling restart should also handle clients on the same machine as the restarting component

Posted by Dmytro Sen <ds...@hortonworks.com>.

(Updated March 17, 2014, 3:21 p.m.)

Review request for Ambari and Sid Wagle.

Bugs: AMBARI-5065
https://issues.apache.org/jira/browse/AMBARI-5065

Repository: ambari

Description
-------

Diffs (updated)
-----

ambari-agent/src/main/python/ambari_agent/ActionQueue.py 549651a
ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py 024c575
ambari-agent/src/main/python/ambari_agent/LiveStatus.py 6f9f4db
ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py e62dd1a
ambari-agent/src/test/python/ambari_agent/TestLiveStatus.py 96a21b9

Diff: https://reviews.apache.org/r/19185/diff/

Testing
-------

Ran 284 tests in 11.538s

Thanks,

Dmytro Sen

Re: Review Request 19185: Rolling restart should also handle clients on the same machine as the restarting component

Posted by Dmytro Sen <ds...@hortonworks.com>.

(Updated March 17, 2014, 3:02 p.m.)

Review request for Ambari and Sid Wagle.

Changes
-------

Uploaded new patch.

Short summary:
When in ActionQueue any component has configurationTags updated, the agent updated them for all installed client components for the same service by calling ActualConfigHandler.write_client_components(self, serviceName, tags)

+ in-memory storing of configuration tags for every installed component.

BTW, do ambari really has to store contigurationTags for the client components, if they are updated automatically ?

Bugs: AMBARI-5065
https://issues.apache.org/jira/browse/AMBARI-5065

Repository: ambari

Description
-------

Diffs (updated)
-----

Diff: https://reviews.apache.org/r/19185/diff/

Testing
-------

Ran 284 tests in 11.538s

Thanks,

Dmytro Sen

Re: Review Request 19185: Rolling restart should also handle clients on the same machine as the restarting component

Posted by Sumit Mohanty <sm...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19185/#review37225
-----------------------------------------------------------



ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py
<https://reviews.apache.org/r/19185/#comment68645>

    This part I am not able to follow. We go through all client components and if they are present in self.configTags and the tags are different we update the file associated with the client component. However, should we not be updating HDFS_CLIENT only when DATANODE/NAMENODE/SNAMENODE.etc. are updated.


- Sumit Mohanty


On March 14, 2014, 2:56 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19185/
> -----------------------------------------------------------
> 
> (Updated March 14, 2014, 2:56 p.m.)
> 
> 
> Review request for Ambari and Sid Wagle.
> 
> 
> Bugs: AMBARI-5065
>     https://issues.apache.org/jira/browse/AMBARI-5065
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 3-host cluster, c6401 has nn, c6402 and c6403 have datanodes + hdfs clients
> 1. create host group in Hdfs
> 2. put one host in the group (say c6403) that has datanode and hdfs client
> 3. change datanode max java heap size so it only affects the datanode
> 4. you'll see a restart is needed only on c6403 as expected (correct) for datanode and hdfs client
> 5. Issue a rolling restart and only on hosts that need configs (so just one host)
> 6. the works fine and it only restarts datanode c6403 as expected (correct)
> problem: hdfs client is not marked as restarted so i still see "1 HDFS Client" component requiring a restart on that host.
> Need to handle clients on the machines as well as part of rolling restart.
> Solution:
> Update client tags on Master/Slave component update in-memory and in individual files.
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 549651a 
>   ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py 024c575 
>   ambari-agent/src/main/python/ambari_agent/LiveStatus.py 6f9f4db 
>   ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py e62dd1a 
>   ambari-agent/src/test/python/ambari_agent/TestLiveStatus.py 96a21b9 
> 
> Diff: https://reviews.apache.org/r/19185/diff/
> 
> 
> Testing
> -------
> 
> Ran 284 tests in 11.538s
> 
> OK
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 19185: Rolling restart should also handle clients on the same machine as the restarting component

Posted by Dmytro Sen <ds...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19185/
-----------------------------------------------------------

(Updated March 14, 2014, 2:56 p.m.)


Review request for Ambari and Sid Wagle.


Changes
-------

If command['configurationTags'] is None then get error:
{noformat}
raceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/ambari_agent/ActionQueue.py", line 131, in process_command
    self.execute_command(command)
  File "/usr/lib/python2.6/site-packages/ambari_agent/ActionQueue.py", line 229, in execute_command
    configHandler.write_actual_component(command['role'], command['configurationTags'])
  File "/usr/lib/python2.6/site-packages/ambari_agent/ActualConfigHandler.py", line 50, in write_actual_component
    self.write_client_components(tags)
  File "/usr/lib/python2.6/site-packages/ambari_agent/ActualConfigHandler.py", line 58, in write_client_components
    if config_name in self.configTags[componentName] and \
TypeError: argument of type 'NoneType' is not iterable
Opening /proc/modules: No such file or directory
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/ambari_agent/ActionQueue.py", line 131, in process_command
    self.execute_command(command)
  File "/usr/lib/python2.6/site-packages/ambari_agent/ActionQueue.py", line 229, in execute_command
    configHandler.write_actual_component(command['role'], command['configurationTags'])
  File "/usr/lib/python2.6/site-packages/ambari_agent/ActualConfigHandler.py", line 50, in write_actual_component
    self.write_client_components(tags)
  File "/usr/lib/python2.6/site-packages/ambari_agent/ActualConfigHandler.py", line 58, in write_client_components
    if config_name in self.configTags[componentName] and \
TypeError: argument of type 'NoneType' is not iterable
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/ambari_agent/ActionQueue.py", line 131, in process_command
    self.execute_command(command)
  File "/usr/lib/python2.6/site-packages/ambari_agent/ActionQueue.py", line 229, in execute_command
    configHandler.write_actual_component(command['role'], command['configurationTags'])
  File "/usr/lib/python2.6/site-packages/ambari_agent/ActualConfigHandler.py", line 50, in write_actual_component
    self.write_client_components(tags)
  File "/usr/lib/python2.6/site-packages/ambari_agent/ActualConfigHandler.py", line 58, in write_client_components
{noformat}


Bugs: AMBARI-5065
    https://issues.apache.org/jira/browse/AMBARI-5065


Repository: ambari


Description
-------

3-host cluster, c6401 has nn, c6402 and c6403 have datanodes + hdfs clients
1. create host group in Hdfs
2. put one host in the group (say c6403) that has datanode and hdfs client
3. change datanode max java heap size so it only affects the datanode
4. you'll see a restart is needed only on c6403 as expected (correct) for datanode and hdfs client
5. Issue a rolling restart and only on hosts that need configs (so just one host)
6. the works fine and it only restarts datanode c6403 as expected (correct)
problem: hdfs client is not marked as restarted so i still see "1 HDFS Client" component requiring a restart on that host.
Need to handle clients on the machines as well as part of rolling restart.
Solution:
Update client tags on Master/Slave component update in-memory and in individual files.


Diffs (updated)
-----

  ambari-agent/src/main/python/ambari_agent/ActionQueue.py 549651a 
  ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py 024c575 
  ambari-agent/src/main/python/ambari_agent/LiveStatus.py 6f9f4db 
  ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py e62dd1a 
  ambari-agent/src/test/python/ambari_agent/TestLiveStatus.py 96a21b9 

Diff: https://reviews.apache.org/r/19185/diff/


Testing
-------

Ran 284 tests in 11.538s

OK


Thanks,

Dmytro Sen

Re: Review Request 19185: Rolling restart should also handle clients on the same machine as the restarting component

Posted by Sid Wagle <sw...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19185/#review37070
-----------------------------------------------------------

Ship it!


Ship It!

- Sid Wagle


On March 13, 2014, 12:23 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19185/
> -----------------------------------------------------------
> 
> (Updated March 13, 2014, 12:23 p.m.)
> 
> 
> Review request for Ambari and Sid Wagle.
> 
> 
> Bugs: AMBARI-5065
>     https://issues.apache.org/jira/browse/AMBARI-5065
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 3-host cluster, c6401 has nn, c6402 and c6403 have datanodes + hdfs clients
> 1. create host group in Hdfs
> 2. put one host in the group (say c6403) that has datanode and hdfs client
> 3. change datanode max java heap size so it only affects the datanode
> 4. you'll see a restart is needed only on c6403 as expected (correct) for datanode and hdfs client
> 5. Issue a rolling restart and only on hosts that need configs (so just one host)
> 6. the works fine and it only restarts datanode c6403 as expected (correct)
> problem: hdfs client is not marked as restarted so i still see "1 HDFS Client" component requiring a restart on that host.
> Need to handle clients on the machines as well as part of rolling restart.
> Solution:
> Update client tags on Master/Slave component update in-memory and in individual files.
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 731ac54 
>   ambari-agent/src/main/python/ambari_agent/ActualConfigHandler.py 024c575 
>   ambari-agent/src/main/python/ambari_agent/LiveStatus.py 6f9f4db 
>   ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py e62dd1a 
>   ambari-agent/src/test/python/ambari_agent/TestLiveStatus.py 96a21b9 
> 
> Diff: https://reviews.apache.org/r/19185/diff/
> 
> 
> Testing
> -------
> 
> Ran 284 tests in 11.538s
> 
> OK
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>