You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Tom Beerbower <tb...@hortonworks.com> on 2015/03/02 15:54:32 UTC
Review Request 31624: SECONDARY_NAMENODE persist object in memory,
causes HostVersion to stay in UPGRADING
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31624/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.
Bugs: AMBARI-9869
https://issues.apache.org/jira/browse/AMBARI-9869
Repository: ambari
Description
-------
Finalize fails because one of the hosts (always the one that had the secondary namenode before) still has a host_version with a state of UPGRADING and doesn't transition it to UPGRADED.
When the SECONDARY_NAMENODE is deleted via the Namenode HA wizard, the HostEntity may still retain a reference to it, thereby causing the object to remain in memory.
For some reason the bi-directional relationships between the host entity and the state entities are not cleaned up on the host side by JPA.
The patch uses a brute force approach to remove the state entites from the collections held by the host entity.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java 9635f30
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java 66e91d3
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java fe5397b
Diff: https://reviews.apache.org/r/31624/diff/
Testing
-------
Manual testing to verify that the HostEntity references to the state entities are cleaned up.
Thanks,
Tom Beerbower
Re: Review Request 31624: SECONDARY_NAMENODE persist object in memory,
causes HostVersion to stay in UPGRADING
Posted by Tom Beerbower <tb...@hortonworks.com>.
> On March 2, 2015, 4:04 p.m., Nate Cole wrote:
> > No tests?
Thanks for the review. I'll add some unit tests.
- Tom
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31624/#review74740
-----------------------------------------------------------
On March 2, 2015, 3:38 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31624/
> -----------------------------------------------------------
>
> (Updated March 2, 2015, 3:38 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-9869
> https://issues.apache.org/jira/browse/AMBARI-9869
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Finalize fails because one of the hosts (always the one that had the secondary namenode before) still has a host_version with a state of UPGRADING and doesn't transition it to UPGRADED.
>
> When the SECONDARY_NAMENODE is deleted via the Namenode HA wizard, the HostEntity may still retain a reference to it, thereby causing the object to remain in memory.
>
> For some reason the bi-directional relationships between the host entity and the state entities are not cleaned up on the host side by JPA.
>
> The patch uses a brute force approach to remove the state entites from the collections held by the host entity.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java 9635f30
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java 66e91d3
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java fe5397b
>
> Diff: https://reviews.apache.org/r/31624/diff/
>
>
> Testing
> -------
>
> Manual testing to verify that the HostEntity references to the state entities are cleaned up.
>
> All existing tests pass ...
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 33:16 min
> [INFO] Finished at: 2015-03-02T10:37:09-05:00
> [INFO] Final Memory: 42M/568M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 31624: SECONDARY_NAMENODE persist object in memory,
causes HostVersion to stay in UPGRADING
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31624/#review74740
-----------------------------------------------------------
Ship it!
No tests?
- Nate Cole
On March 2, 2015, 10:38 a.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31624/
> -----------------------------------------------------------
>
> (Updated March 2, 2015, 10:38 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-9869
> https://issues.apache.org/jira/browse/AMBARI-9869
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Finalize fails because one of the hosts (always the one that had the secondary namenode before) still has a host_version with a state of UPGRADING and doesn't transition it to UPGRADED.
>
> When the SECONDARY_NAMENODE is deleted via the Namenode HA wizard, the HostEntity may still retain a reference to it, thereby causing the object to remain in memory.
>
> For some reason the bi-directional relationships between the host entity and the state entities are not cleaned up on the host side by JPA.
>
> The patch uses a brute force approach to remove the state entites from the collections held by the host entity.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java 9635f30
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java 66e91d3
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java fe5397b
>
> Diff: https://reviews.apache.org/r/31624/diff/
>
>
> Testing
> -------
>
> Manual testing to verify that the HostEntity references to the state entities are cleaned up.
>
> All existing tests pass ...
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 33:16 min
> [INFO] Finished at: 2015-03-02T10:37:09-05:00
> [INFO] Final Memory: 42M/568M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 31624: SECONDARY_NAMENODE persist object in memory,
causes HostVersion to stay in UPGRADING
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31624/#review74746
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java
<https://reviews.apache.org/r/31624/#comment121439>
I think it would be nicer to have the remove method as part of the entity so that we don't need to worry about concurrent modification exceptions. What if something is iterating while this remove happens?
Something we could track in a different Jira if you didn't want to fix it now.
- Jonathan Hurley
On March 2, 2015, 10:38 a.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31624/
> -----------------------------------------------------------
>
> (Updated March 2, 2015, 10:38 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-9869
> https://issues.apache.org/jira/browse/AMBARI-9869
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Finalize fails because one of the hosts (always the one that had the secondary namenode before) still has a host_version with a state of UPGRADING and doesn't transition it to UPGRADED.
>
> When the SECONDARY_NAMENODE is deleted via the Namenode HA wizard, the HostEntity may still retain a reference to it, thereby causing the object to remain in memory.
>
> For some reason the bi-directional relationships between the host entity and the state entities are not cleaned up on the host side by JPA.
>
> The patch uses a brute force approach to remove the state entites from the collections held by the host entity.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java 9635f30
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java 66e91d3
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java fe5397b
>
> Diff: https://reviews.apache.org/r/31624/diff/
>
>
> Testing
> -------
>
> Manual testing to verify that the HostEntity references to the state entities are cleaned up.
>
> All existing tests pass ...
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 33:16 min
> [INFO] Finished at: 2015-03-02T10:37:09-05:00
> [INFO] Final Memory: 42M/568M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 31624: SECONDARY_NAMENODE persist object in memory,
causes HostVersion to stay in UPGRADING
Posted by Tom Beerbower <tb...@hortonworks.com>.
> On March 2, 2015, 4:47 p.m., Jonathan Hurley wrote:
> > I made a comment about the fact that we return the actual collections and then use the Collection.remove() method which could be dangerous. Might be better to encapsulate this in the entities as a remove() method which then can ensure that a ConccurrentModException can't happen.
> >
> > If you agree with me, then let's fix it now. If you don't think it's a real-world scenario then we can create a Jira to track it.
Thanks for the review. Good point. Better to fix it now.
- Tom
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31624/#review74747
-----------------------------------------------------------
On March 2, 2015, 5:21 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31624/
> -----------------------------------------------------------
>
> (Updated March 2, 2015, 5:21 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-9869
> https://issues.apache.org/jira/browse/AMBARI-9869
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Finalize fails because one of the hosts (always the one that had the secondary namenode before) still has a host_version with a state of UPGRADING and doesn't transition it to UPGRADED.
>
> When the SECONDARY_NAMENODE is deleted via the Namenode HA wizard, the HostEntity may still retain a reference to it, thereby causing the object to remain in memory.
>
> For some reason the bi-directional relationships between the host entity and the state entities are not cleaned up on the host side by JPA.
>
> The patch uses a brute force approach to remove the state entites from the collections held by the host entity.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java 9635f30
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java 66e91d3
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java fe5397b
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAOTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentStateDAOTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/31624/diff/
>
>
> Testing
> -------
>
> Manual testing to verify that the HostEntity references to the state entities are cleaned up.
>
> All existing tests pass ...
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 33:16 min
> [INFO] Finished at: 2015-03-02T10:37:09-05:00
> [INFO] Final Memory: 42M/568M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 31624: SECONDARY_NAMENODE persist object in memory,
causes HostVersion to stay in UPGRADING
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31624/#review74747
-----------------------------------------------------------
Ship it!
I made a comment about the fact that we return the actual collections and then use the Collection.remove() method which could be dangerous. Might be better to encapsulate this in the entities as a remove() method which then can ensure that a ConccurrentModException can't happen.
If you agree with me, then let's fix it now. If you don't think it's a real-world scenario then we can create a Jira to track it.
- Jonathan Hurley
On March 2, 2015, 10:38 a.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31624/
> -----------------------------------------------------------
>
> (Updated March 2, 2015, 10:38 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-9869
> https://issues.apache.org/jira/browse/AMBARI-9869
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Finalize fails because one of the hosts (always the one that had the secondary namenode before) still has a host_version with a state of UPGRADING and doesn't transition it to UPGRADED.
>
> When the SECONDARY_NAMENODE is deleted via the Namenode HA wizard, the HostEntity may still retain a reference to it, thereby causing the object to remain in memory.
>
> For some reason the bi-directional relationships between the host entity and the state entities are not cleaned up on the host side by JPA.
>
> The patch uses a brute force approach to remove the state entites from the collections held by the host entity.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java 9635f30
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java 66e91d3
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java fe5397b
>
> Diff: https://reviews.apache.org/r/31624/diff/
>
>
> Testing
> -------
>
> Manual testing to verify that the HostEntity references to the state entities are cleaned up.
>
> All existing tests pass ...
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 33:16 min
> [INFO] Finished at: 2015-03-02T10:37:09-05:00
> [INFO] Final Memory: 42M/568M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 31624: SECONDARY_NAMENODE persist object in memory,
causes HostVersion to stay in UPGRADING
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31624/#review74789
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On March 2, 2015, 1:31 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31624/
> -----------------------------------------------------------
>
> (Updated March 2, 2015, 1:31 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-9869
> https://issues.apache.org/jira/browse/AMBARI-9869
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Finalize fails because one of the hosts (always the one that had the secondary namenode before) still has a host_version with a state of UPGRADING and doesn't transition it to UPGRADED.
>
> When the SECONDARY_NAMENODE is deleted via the Namenode HA wizard, the HostEntity may still retain a reference to it, thereby causing the object to remain in memory.
>
> For some reason the bi-directional relationships between the host entity and the state entities are not cleaned up on the host side by JPA.
>
> The patch uses a brute force approach to remove the state entites from the collections held by the host entity.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java 9635f30
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java 66e91d3
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostEntity.java df22de1
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java fe5397b
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog150.java 4ecfe44
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAOTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentStateDAOTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/orm/entities/HostEntityTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogHelper.java 2e59db4
>
> Diff: https://reviews.apache.org/r/31624/diff/
>
>
> Testing
> -------
>
> Manual testing to verify that the HostEntity references to the state entities are cleaned up.
>
> All existing tests pass ...
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 33:16 min
> [INFO] Finished at: 2015-03-02T10:37:09-05:00
> [INFO] Final Memory: 42M/568M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 31624: SECONDARY_NAMENODE persist object in memory,
causes HostVersion to stay in UPGRADING
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31624/#review74792
-----------------------------------------------------------
Ship it!
Ship It!
- Alejandro Fernandez
On March 2, 2015, 6:31 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31624/
> -----------------------------------------------------------
>
> (Updated March 2, 2015, 6:31 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-9869
> https://issues.apache.org/jira/browse/AMBARI-9869
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Finalize fails because one of the hosts (always the one that had the secondary namenode before) still has a host_version with a state of UPGRADING and doesn't transition it to UPGRADED.
>
> When the SECONDARY_NAMENODE is deleted via the Namenode HA wizard, the HostEntity may still retain a reference to it, thereby causing the object to remain in memory.
>
> For some reason the bi-directional relationships between the host entity and the state entities are not cleaned up on the host side by JPA.
>
> The patch uses a brute force approach to remove the state entites from the collections held by the host entity.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java 9635f30
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java 66e91d3
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostEntity.java df22de1
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java fe5397b
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog150.java 4ecfe44
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAOTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentStateDAOTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/orm/entities/HostEntityTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogHelper.java 2e59db4
>
> Diff: https://reviews.apache.org/r/31624/diff/
>
>
> Testing
> -------
>
> Manual testing to verify that the HostEntity references to the state entities are cleaned up.
>
> All existing tests pass ...
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 33:16 min
> [INFO] Finished at: 2015-03-02T10:37:09-05:00
> [INFO] Final Memory: 42M/568M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 31624: SECONDARY_NAMENODE persist object in memory,
causes HostVersion to stay in UPGRADING
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On March 2, 2015, 2:04 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java, line 96
> > <https://reviews.apache.org/r/31624/diff/3/?file=882239#file882239line96>
> >
> > If at some point we fix this JPA bug, is it possible for this call to fail?
>
> Tom Beerbower wrote:
> Thanks for the review. I don't think so. It should have the same semantics as Collection.remove(). If the state entity has already been removed then this should do nothing.
No, I don't think so. We're removing by matching on equals() of the hostComponentStateEntity; if EclipseLink already removed it, then this does nothing. If it didn't remove it, then on the merge it will see it's gone anyway and merge properly.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31624/#review74788
-----------------------------------------------------------
On March 2, 2015, 1:31 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31624/
> -----------------------------------------------------------
>
> (Updated March 2, 2015, 1:31 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-9869
> https://issues.apache.org/jira/browse/AMBARI-9869
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Finalize fails because one of the hosts (always the one that had the secondary namenode before) still has a host_version with a state of UPGRADING and doesn't transition it to UPGRADED.
>
> When the SECONDARY_NAMENODE is deleted via the Namenode HA wizard, the HostEntity may still retain a reference to it, thereby causing the object to remain in memory.
>
> For some reason the bi-directional relationships between the host entity and the state entities are not cleaned up on the host side by JPA.
>
> The patch uses a brute force approach to remove the state entites from the collections held by the host entity.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java 9635f30
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java 66e91d3
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostEntity.java df22de1
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java fe5397b
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog150.java 4ecfe44
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAOTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentStateDAOTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/orm/entities/HostEntityTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogHelper.java 2e59db4
>
> Diff: https://reviews.apache.org/r/31624/diff/
>
>
> Testing
> -------
>
> Manual testing to verify that the HostEntity references to the state entities are cleaned up.
>
> All existing tests pass ...
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 33:16 min
> [INFO] Finished at: 2015-03-02T10:37:09-05:00
> [INFO] Final Memory: 42M/568M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 31624: SECONDARY_NAMENODE persist object in memory,
causes HostVersion to stay in UPGRADING
Posted by Tom Beerbower <tb...@hortonworks.com>.
> On March 2, 2015, 7:04 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java, line 96
> > <https://reviews.apache.org/r/31624/diff/3/?file=882239#file882239line96>
> >
> > If at some point we fix this JPA bug, is it possible for this call to fail?
Thanks for the review. I don't think so. It should have the same semantics as Collection.remove(). If the state entity has already been removed then this should do nothing.
- Tom
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31624/#review74788
-----------------------------------------------------------
On March 2, 2015, 6:31 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31624/
> -----------------------------------------------------------
>
> (Updated March 2, 2015, 6:31 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-9869
> https://issues.apache.org/jira/browse/AMBARI-9869
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Finalize fails because one of the hosts (always the one that had the secondary namenode before) still has a host_version with a state of UPGRADING and doesn't transition it to UPGRADED.
>
> When the SECONDARY_NAMENODE is deleted via the Namenode HA wizard, the HostEntity may still retain a reference to it, thereby causing the object to remain in memory.
>
> For some reason the bi-directional relationships between the host entity and the state entities are not cleaned up on the host side by JPA.
>
> The patch uses a brute force approach to remove the state entites from the collections held by the host entity.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java 9635f30
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java 66e91d3
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostEntity.java df22de1
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java fe5397b
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog150.java 4ecfe44
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAOTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentStateDAOTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/orm/entities/HostEntityTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogHelper.java 2e59db4
>
> Diff: https://reviews.apache.org/r/31624/diff/
>
>
> Testing
> -------
>
> Manual testing to verify that the HostEntity references to the state entities are cleaned up.
>
> All existing tests pass ...
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 33:16 min
> [INFO] Finished at: 2015-03-02T10:37:09-05:00
> [INFO] Final Memory: 42M/568M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 31624: SECONDARY_NAMENODE persist object in memory,
causes HostVersion to stay in UPGRADING
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31624/#review74788
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java
<https://reviews.apache.org/r/31624/#comment121543>
If at some point we fix this JPA bug, is it possible for this call to fail?
- Alejandro Fernandez
On March 2, 2015, 6:31 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31624/
> -----------------------------------------------------------
>
> (Updated March 2, 2015, 6:31 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-9869
> https://issues.apache.org/jira/browse/AMBARI-9869
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Finalize fails because one of the hosts (always the one that had the secondary namenode before) still has a host_version with a state of UPGRADING and doesn't transition it to UPGRADED.
>
> When the SECONDARY_NAMENODE is deleted via the Namenode HA wizard, the HostEntity may still retain a reference to it, thereby causing the object to remain in memory.
>
> For some reason the bi-directional relationships between the host entity and the state entities are not cleaned up on the host side by JPA.
>
> The patch uses a brute force approach to remove the state entites from the collections held by the host entity.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java 9635f30
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java 66e91d3
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostEntity.java df22de1
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java fe5397b
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog150.java 4ecfe44
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAOTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentStateDAOTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/orm/entities/HostEntityTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogHelper.java 2e59db4
>
> Diff: https://reviews.apache.org/r/31624/diff/
>
>
> Testing
> -------
>
> Manual testing to verify that the HostEntity references to the state entities are cleaned up.
>
> All existing tests pass ...
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 33:16 min
> [INFO] Finished at: 2015-03-02T10:37:09-05:00
> [INFO] Final Memory: 42M/568M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 31624: SECONDARY_NAMENODE persist object in memory,
causes HostVersion to stay in UPGRADING
Posted by Tom Beerbower <tb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31624/
-----------------------------------------------------------
(Updated March 2, 2015, 6:31 p.m.)
Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.
Changes
-------
Updated diff.
Bugs: AMBARI-9869
https://issues.apache.org/jira/browse/AMBARI-9869
Repository: ambari
Description
-------
Finalize fails because one of the hosts (always the one that had the secondary namenode before) still has a host_version with a state of UPGRADING and doesn't transition it to UPGRADED.
When the SECONDARY_NAMENODE is deleted via the Namenode HA wizard, the HostEntity may still retain a reference to it, thereby causing the object to remain in memory.
For some reason the bi-directional relationships between the host entity and the state entities are not cleaned up on the host side by JPA.
The patch uses a brute force approach to remove the state entites from the collections held by the host entity.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java 9635f30
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java 66e91d3
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostEntity.java df22de1
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java fe5397b
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog150.java 4ecfe44
ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAOTest.java PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentStateDAOTest.java PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/orm/entities/HostEntityTest.java PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogHelper.java 2e59db4
Diff: https://reviews.apache.org/r/31624/diff/
Testing
-------
Manual testing to verify that the HostEntity references to the state entities are cleaned up.
All existing tests pass ...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 33:16 min
[INFO] Finished at: 2015-03-02T10:37:09-05:00
[INFO] Final Memory: 42M/568M
[INFO] ------------------------------------------------------------------------
Thanks,
Tom Beerbower
Re: Review Request 31624: SECONDARY_NAMENODE persist object in memory,
causes HostVersion to stay in UPGRADING
Posted by Tom Beerbower <tb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31624/
-----------------------------------------------------------
(Updated March 2, 2015, 5:21 p.m.)
Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.
Changes
-------
Added unit tests.
Bugs: AMBARI-9869
https://issues.apache.org/jira/browse/AMBARI-9869
Repository: ambari
Description
-------
Finalize fails because one of the hosts (always the one that had the secondary namenode before) still has a host_version with a state of UPGRADING and doesn't transition it to UPGRADED.
When the SECONDARY_NAMENODE is deleted via the Namenode HA wizard, the HostEntity may still retain a reference to it, thereby causing the object to remain in memory.
For some reason the bi-directional relationships between the host entity and the state entities are not cleaned up on the host side by JPA.
The patch uses a brute force approach to remove the state entites from the collections held by the host entity.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java 9635f30
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java 66e91d3
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java fe5397b
ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAOTest.java PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentStateDAOTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/31624/diff/
Testing
-------
Manual testing to verify that the HostEntity references to the state entities are cleaned up.
All existing tests pass ...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 33:16 min
[INFO] Finished at: 2015-03-02T10:37:09-05:00
[INFO] Final Memory: 42M/568M
[INFO] ------------------------------------------------------------------------
Thanks,
Tom Beerbower
Re: Review Request 31624: SECONDARY_NAMENODE persist object in memory,
causes HostVersion to stay in UPGRADING
Posted by Tom Beerbower <tb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31624/
-----------------------------------------------------------
(Updated March 2, 2015, 3:38 p.m.)
Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.
Bugs: AMBARI-9869
https://issues.apache.org/jira/browse/AMBARI-9869
Repository: ambari
Description
-------
Finalize fails because one of the hosts (always the one that had the secondary namenode before) still has a host_version with a state of UPGRADING and doesn't transition it to UPGRADED.
When the SECONDARY_NAMENODE is deleted via the Namenode HA wizard, the HostEntity may still retain a reference to it, thereby causing the object to remain in memory.
For some reason the bi-directional relationships between the host entity and the state entities are not cleaned up on the host side by JPA.
The patch uses a brute force approach to remove the state entites from the collections held by the host entity.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java 9635f30
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentStateDAO.java 66e91d3
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java fe5397b
Diff: https://reviews.apache.org/r/31624/diff/
Testing (updated)
-------
Manual testing to verify that the HostEntity references to the state entities are cleaned up.
All existing tests pass ...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 33:16 min
[INFO] Finished at: 2015-03-02T10:37:09-05:00
[INFO] Final Memory: 42M/568M
[INFO] ------------------------------------------------------------------------
Thanks,
Tom Beerbower