You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Alejandro Fernandez <af...@hortonworks.com> on 2015/04/01 19:41:54 UTC
Re: Review Request 32557: Full Delete of Host : Switch host-related
tables
to use host_id instead of host_name column for hoststate, hostcomponentstate,
hostcomponentdesiredstate
> On March 30, 2015, 8:42 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentStateEntity.java, lines 105-106
> > <https://reviews.apache.org/r/32557/diff/2/?file=908680#file908680line105>
> >
> > Just a thought; although this should never happen, auto boxing could cause NPEs around this. Should we return 0L instead?
I think it ought to be fine. I only call getHostId() if hostEntity is not null, and this returns a "L"ong type that can be null.
> On March 30, 2015, 8:42 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog150.java, line 793
> > <https://reviews.apache.org/r/32557/diff/2/?file=908686#file908686line793>
> >
> > This might be a problem since the actual DDL hasn't changed yet (that happens in Upgrade210). You're using relationship classes that assume the database structure has already changed.
For now, this explicitly requires upgrading to 2.0.0 first, and then using the 2.1.0 jar to perform the next upgrade. There may be a bug in UpgradeCatalog210 because it uses the newer HostEntity while upgrading the schema that still exists in 2.0.0
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32557/#review78252
-----------------------------------------------------------
On March 30, 2015, 11:43 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32557/
> -----------------------------------------------------------
>
> (Updated March 30, 2015, 11:43 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sid Wagle.
>
>
> Bugs: AMBARI-10168
> https://issues.apache.org/jira/browse/AMBARI-10168
>
>
> Repository: ambari
>
>
> Description
> -------
>
> See p;arent Jira AMBARI-10167
>
> Deleting a host from the UI only removes it from the mapping to a cluster.
> The host still exists at the hosts endpoint, and all of the historic data maintains intact; config overrides, config groups, host state, host version, host role command logs, etc.
>
> This causes two problems.
> 1. If a host is removed from a cluster and its underlying environment changes (OS, IP, network card, etc), and is re-added to the cluster, then it still maintains the old information that is incorrect.
> 2. If host x dies, and is removed from the cluster, there is no way for a new host to take over its hostname.
> Expectation in Ambari 2.1.0
> Deleting a host should be clean, i.e., it should delete all configuration and historic records.
> Because renaming a host requires a more complicated workflow, so it will have to be done manually by the user (see instructions in A1).
> Lastly, Ambari will require ambari-agent to be stopped before the host can be deleted.
>
>
> Tables:
> 10 tables have foreign keys to the host_name in the hosts table:
> 1. hostcomponentstate (fixed in this patch)
> 2. hostcomponentdesiredstate (fixed in this patch)
> 3. hoststate (fixed in this patch)
> 4. host_version (future patch)
> 5. host_role_command (future patch)
> 6. request_operation_level (future patch)
> 7. hostconfigmapping (future patch)
> 8. configgrouphostmapping (future patch)
> 9. kerberos_principal_host (future patch)
> 10. clusterhostmapping (already fixed)
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java 8e92877
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostDAO.java 0fb9c59
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentDesiredStateEntity.java e0f1e9e
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentDesiredStateEntityPK.java fc92858
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentStateEntity.java 0a31b90
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentStateEntityPK.java bc103a1
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostEntity.java 4df5f39
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostStateEntity.java 5d5a1e8
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java 1bd2814
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 7975f4e
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java 1f3c09a
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java 3691af2
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog.java fcf4f51
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog150.java dd8e33e
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog151.java afe27b5
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog160.java a1f11d8
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog161.java 4ab0cb6
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog170.java 98ac89f
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog200.java d3b062a
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog210.java ba4dd2d
> ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 26ff9bb
> ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 13e8939
> ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 622fca3
> ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 9fe9cd7
> ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql a722d2a
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAOTest.java 7b34827
> ambari-server/src/test/java/org/apache/ambari/server/state/ServiceComponentTest.java c27e600
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersTest.java ebf742e
> ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java 8ebf2ce
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/SectionDDL.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog150Test.java af6721c
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog170Test.java 6fa3e0a
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog200Test.java 9c39749
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog210Test.java 07b8410
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogHelper.java b5db360
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogTest.java b0c8fd9
>
> Diff: https://reviews.apache.org/r/32557/diff/
>
>
> Testing
> -------
>
> Unit tests passed,
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [4.737s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.031s]
> [INFO] Ambari Web ........................................ SUCCESS [38.862s]
> [INFO] Ambari Views ...................................... SUCCESS [2.078s]
> [INFO] Ambari Admin View ................................. SUCCESS [9.289s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.487s]
> [INFO] Ambari Server ..................................... SUCCESS [34:09.719s]
> [INFO] Ambari Agent ...................................... SUCCESS [21.818s]
> [INFO] Ambari Client ..................................... SUCCESS [0.033s]
> [INFO] Ambari Python Client .............................. SUCCESS [0.254s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [9.834s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.032s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.044s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [6.661s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 35:45.473s
> [INFO] Finished at: Thu Mar 26 17:39:24 PDT 2015
> [INFO] Final Memory: 66M/500M
> [INFO] ------------------------------------------------------------------------
>
> I tested this by making the DB changes, copying my jar, then hitting the dashboard, API, and restarting services on a cluster.
>
>
> -- For Postgres
> --- Fix hoststate
> SELECT COUNT(*) FROM hoststate;
> ALTER TABLE hoststate ADD COLUMN host_id BIGINT NULL;
> UPDATE hoststate t SET host_id = (SELECT host_id FROM hosts h WHERE h.host_name = t.host_name) WHERE t.host_id IS NULL AND t.host_name IS NOT NULL;
> ALTER TABLE hoststate ALTER COLUMN host_id SET NOT NULL;
> ALTER TABLE hoststate ADD CONSTRAINT FK_hoststate_host_id FOREIGN KEY (host_id) REFERENCES hosts (host_id);
> ALTER TABLE hoststate DROP COLUMN host_name;
> ALTER TABLE hoststate ADD CONSTRAINT hoststate_pkey PRIMARY KEY (host_id);
>
> -- Fix hostcomponentstate
> SELECT COUNT(*) FROM hostcomponentstate;
> ALTER TABLE hostcomponentstate ADD COLUMN host_id BIGINT NULL;
> UPDATE hostcomponentstate t SET host_id = (SELECT host_id FROM hosts h WHERE h.host_name = t.host_name) WHERE t.host_id IS NULL AND t.host_name IS NOT NULL;
> ALTER TABLE hostcomponentstate ALTER COLUMN host_id SET NOT NULL;
> ALTER TABLE hostcomponentstate ADD CONSTRAINT FK_hostcomponentstate_host_id FOREIGN KEY (host_id) REFERENCES hosts (host_id);
> ALTER TABLE hostcomponentstate DROP COLUMN host_name;
> ALTER TABLE hostcomponentstate ADD CONSTRAINT hostcomponentstate_pkey PRIMARY KEY (cluster_id, component_name, host_id, service_name);
>
> -- Fix hostcomponentdesiredstate
> SELECT COUNT(*) FROM hostcomponentdesiredstate;
> ALTER TABLE hostcomponentdesiredstate ADD COLUMN host_id BIGINT NULL;
> UPDATE hostcomponentdesiredstate t SET host_id = (SELECT host_id FROM hosts h WHERE h.host_name = t.host_name) WHERE t.host_id IS NULL AND t.host_name IS NOT NULL;
> ALTER TABLE hostcomponentdesiredstate ALTER COLUMN host_id SET NOT NULL;
> ALTER TABLE hostcomponentdesiredstate ADD CONSTRAINT FK_hostcomponentdesiredstate_host_id FOREIGN KEY (host_id) REFERENCES hosts (host_id);
> ALTER TABLE hostcomponentdesiredstate DROP COLUMN host_name;
> ALTER TABLE hostcomponentdesiredstate ADD CONSTRAINT hostcomponentdesiredstate_pkey PRIMARY KEY (cluster_id, component_name, host_id, service_name);
>
>
> Thanks,
>
> Alejandro Fernandez
>
>