You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Marcus Sorensen (JIRA)" <ji...@apache.org> on 2013/09/23 23:09:08 UTC
[jira] [Comment Edited] (CLOUDSTACK-4725) KVM agent fails to join
if local pool is already registered
[ https://issues.apache.org/jira/browse/CLOUDSTACK-4725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13775664#comment-13775664 ]
Marcus Sorensen edited comment on CLOUDSTACK-4725 at 9/23/13 9:08 PM:
----------------------------------------------------------------------
just ran through it again... installed 4.1.1, started a zone, added a host, added local storage. Upgraded to 4.2, agent won't connect, fails on the default storage plugin's initialize call for its local storage as mentioned in the initial post. I had understood that initialize should only be called when adding new storage pools, not whenever host connects.
2013-09-23 15:07:42,593 DEBUG [agent.manager.AgentManagerImpl] (AgentConnectTaskPool-41:null) Sending Connect to listener: LocalStoragePoolListener
2013-09-23 15:07:42,593 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-10:null) Ping from 1
2013-09-23 15:07:42,594 DEBUG [datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl] (AgentConnectTaskPool-41:null) createPool Params @ scheme - Filesystem storageHost - 172.17.10.10 hostPath - /var/lib/libvirt/images port - -1
2013-09-23 15:07:42,595 DEBUG [datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl] (AgentConnectTaskPool-41:null) Another active pool with the same uuid already exists
2013-09-23 15:07:42,595 WARN [cloud.storage.StorageManagerImpl] (AgentConnectTaskPool-41:null) Unable to setup the local storage pool for Host[-1-Routing]
com.cloud.utils.exception.CloudRuntimeException: Another active pool with the same uuid already exists
at org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.initialize(CloudStackPrimaryDataStoreLifeCycleImpl.java:319)
at com.cloud.storage.StorageManagerImpl.createLocalStorage(StorageManagerImpl.java:647)
at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at com.cloud.storage.LocalStoragePoolListener.processConnect(LocalStoragePoolListener.java:86)
at com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:587)
at com.cloud.agent.manager.AgentManagerImpl.handleConnectedAgent(AgentManagerImpl.java:1085)
at com.cloud.agent.manager.AgentManagerImpl.access$100(AgentManagerImpl.java:144)
at com.cloud.agent.manager.AgentManagerImpl$HandleAgentConnectTask.run(AgentManagerImpl.java:1160)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
4.1.1:
mysql> select id,name,uuid,version,status,data_center_id,pod_id,cluster_id,removed from host;
+----+-----------------------------------+--------------------------------------+---------+--------+----------------+--------+------------+---------+
| id | name | uuid | version | status | data_center_id | pod_id | cluster_id | removed |
+----+-----------------------------------+--------------------------------------+---------+--------+----------------+--------+------------+---------+
| 1 | devcloud-kvm | 6084fccf-e669-4aad-866b-1e364fa63bd3 | 4.1.1 | Up | 1 | 1 | 1 | NULL |
| 2 | nfs://172.17.10.10:/nfs/secondary | 734fc093-0ea5-4813-b395-acadd2d9c4ae | 4.1.1 | Alert | 1 | NULL | NULL | NULL |
| 3 | s-1-VM | 35027be8-9748-4cd0-8541-f2849d445de4 | 4.1.1 | Up | 1 | 1 | NULL | NULL |
| 4 | v-2-VM | 6c67f10a-cfa6-4f0c-bc36-b3a8be8672b9 | 4.1.1 | Up | 1 | 1 | NULL | NULL |
+----+-----------------------------------+--------------------------------------+---------+--------+----------------+--------+------------+---------+
4 rows in set (0.00 sec)
mysql> select id,name,uuid,pool_type,data_center_id,pod_id,cluster_id,status from storage_pool;
+-----+--------------+--------------------------------------+------------+----------------+--------+------------+--------+
| id | name | uuid | pool_type | data_center_id | pod_id | cluster_id | status |
+-----+--------------+--------------------------------------+------------+----------------+--------+------------+--------+
| 200 | devcloud-kvm | 430249f8-646d-46bb-a0b9-0f723e7908d7 | Filesystem | 1 | 1 | 1 | Up |
+-----+--------------+--------------------------------------+------------+----------------+--------+------------+--------+
1 row in set (0.00 sec)
4.2:
mysql> select id,name,uuid,version,status,data_center_id,pod_id,cluster_id,removed from host;
+----+-----------------------------------+--------------------------------------+---------+--------------+----------------+--------+------------+---------+
| id | name | uuid | version | status | data_center_id | pod_id | cluster_id | removed |
+----+-----------------------------------+--------------------------------------+---------+--------------+----------------+--------+------------+---------+
| 1 | devcloud-kvm | 6084fccf-e669-4aad-866b-1e364fa63bd3 | 4.2.0 | Alert | 1 | 1 | 1 | NULL |
| 2 | nfs://172.17.10.10:/nfs/secondary | 734fc093-0ea5-4813-b395-acadd2d9c4ae | 4.1.1 | Alert | 1 | NULL | NULL | NULL |
| 3 | s-1-VM | 35027be8-9748-4cd0-8541-f2849d445de4 | 4.1.1 | Disconnected | 1 | 1 | NULL | NULL |
| 4 | v-2-VM | 6c67f10a-cfa6-4f0c-bc36-b3a8be8672b9 | 4.1.1 | Disconnected | 1 | 1 | NULL | NULL |
+----+-----------------------------------+--------------------------------------+---------+--------------+----------------+--------+------------+---------+
4 rows in set (0.00 sec)
mysql> select id,name,uuid,pool_type,data_center_id,pod_id,cluster_id,status from storage_pool;
+-----+--------------+--------------------------------------+------------+----------------+--------+------------+--------+
| id | name | uuid | pool_type | data_center_id | pod_id | cluster_id | status |
+-----+--------------+--------------------------------------+------------+----------------+--------+------------+--------+
| 200 | devcloud-kvm | 430249f8-646d-46bb-a0b9-0f723e7908d7 | Filesystem | 1 | 1 | 1 | Up |
+-----+--------------+--------------------------------------+------------+----------------+--------+------------+--------+
1 row in set (0.00 sec)
was (Author: mlsorensen):
just ran through it again... installed 4.1.1, started a zone, added a host, added local storage. Upgraded to 4.2, agent won't connect, fails on the default storage plugin's initialize call for its local storage as mentioned in the initial post. I had understood that initialize should only be called when adding new storage pools, not whenever host connects.
4.1.1:
mysql> select id,name,uuid,version,status,data_center_id,pod_id,cluster_id,removed from host;
+----+-----------------------------------+--------------------------------------+---------+--------+----------------+--------+------------+---------+
| id | name | uuid | version | status | data_center_id | pod_id | cluster_id | removed |
+----+-----------------------------------+--------------------------------------+---------+--------+----------------+--------+------------+---------+
| 1 | devcloud-kvm | 6084fccf-e669-4aad-866b-1e364fa63bd3 | 4.1.1 | Up | 1 | 1 | 1 | NULL |
| 2 | nfs://172.17.10.10:/nfs/secondary | 734fc093-0ea5-4813-b395-acadd2d9c4ae | 4.1.1 | Alert | 1 | NULL | NULL | NULL |
| 3 | s-1-VM | 35027be8-9748-4cd0-8541-f2849d445de4 | 4.1.1 | Up | 1 | 1 | NULL | NULL |
| 4 | v-2-VM | 6c67f10a-cfa6-4f0c-bc36-b3a8be8672b9 | 4.1.1 | Up | 1 | 1 | NULL | NULL |
+----+-----------------------------------+--------------------------------------+---------+--------+----------------+--------+------------+---------+
4 rows in set (0.00 sec)
mysql> select id,name,uuid,pool_type,data_center_id,pod_id,cluster_id,status from storage_pool;
+-----+--------------+--------------------------------------+------------+----------------+--------+------------+--------+
| id | name | uuid | pool_type | data_center_id | pod_id | cluster_id | status |
+-----+--------------+--------------------------------------+------------+----------------+--------+------------+--------+
| 200 | devcloud-kvm | 430249f8-646d-46bb-a0b9-0f723e7908d7 | Filesystem | 1 | 1 | 1 | Up |
+-----+--------------+--------------------------------------+------------+----------------+--------+------------+--------+
1 row in set (0.00 sec)
4.2:
mysql> select id,name,uuid,version,status,data_center_id,pod_id,cluster_id,removed from host;
+----+-----------------------------------+--------------------------------------+---------+--------------+----------------+--------+------------+---------+
| id | name | uuid | version | status | data_center_id | pod_id | cluster_id | removed |
+----+-----------------------------------+--------------------------------------+---------+--------------+----------------+--------+------------+---------+
| 1 | devcloud-kvm | 6084fccf-e669-4aad-866b-1e364fa63bd3 | 4.2.0 | Alert | 1 | 1 | 1 | NULL |
| 2 | nfs://172.17.10.10:/nfs/secondary | 734fc093-0ea5-4813-b395-acadd2d9c4ae | 4.1.1 | Alert | 1 | NULL | NULL | NULL |
| 3 | s-1-VM | 35027be8-9748-4cd0-8541-f2849d445de4 | 4.1.1 | Disconnected | 1 | 1 | NULL | NULL |
| 4 | v-2-VM | 6c67f10a-cfa6-4f0c-bc36-b3a8be8672b9 | 4.1.1 | Disconnected | 1 | 1 | NULL | NULL |
+----+-----------------------------------+--------------------------------------+---------+--------------+----------------+--------+------------+---------+
4 rows in set (0.00 sec)
mysql> select id,name,uuid,pool_type,data_center_id,pod_id,cluster_id,status from storage_pool;
+-----+--------------+--------------------------------------+------------+----------------+--------+------------+--------+
| id | name | uuid | pool_type | data_center_id | pod_id | cluster_id | status |
+-----+--------------+--------------------------------------+------------+----------------+--------+------------+--------+
| 200 | devcloud-kvm | 430249f8-646d-46bb-a0b9-0f723e7908d7 | Filesystem | 1 | 1 | 1 | Up |
+-----+--------------+--------------------------------------+------------+----------------+--------+------------+--------+
1 row in set (0.00 sec)
> KVM agent fails to join if local pool is already registered
> -----------------------------------------------------------
>
> Key: CLOUDSTACK-4725
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4725
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the default.)
> Components: KVM
> Affects Versions: 4.2.0
> Reporter: Marcus Sorensen
> Assignee: edison su
> Priority: Critical
> Fix For: 4.2.1
>
>
> "Another active pool with the same uuid already exists"
> This might be a regression, I believe we used to catch this case and/or check libvirt to see if pool was already established and just return if it is already there. Or maybe we only do that for non-local.
> 2013-09-22 23:42:35,265 DEBUG [agent.manager.AgentManagerImpl] (AgentConnectTaskPool-396:null) Sending Connect to listener: LocalStoragePoolListener
> 2013-09-22 23:42:35,266 DEBUG [datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl] (AgentConnectTaskPool-396:null) createPool Params @ scheme - Filesystem storageHost - 172.17.10.10 hostPath - /var/lib/libvirt/images port - -1
> 2013-09-22 23:42:35,267 DEBUG [datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl] (AgentConnectTaskPool-396:null) Another active pool with the same uuid already exists
> 2013-09-22 23:42:35,267 WARN [cloud.storage.StorageManagerImpl] (AgentConnectTaskPool-396:null) Unable to setup the local storage pool for Host[-1-Routing]
> com.cloud.utils.exception.CloudRuntimeException: Another active pool with the same uuid already exists
> at org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.initialize(CloudStackPrimaryDataStoreLifeCycleImpl.java:319)
> at com.cloud.storage.StorageManagerImpl.createLocalStorage(StorageManagerImpl.java:647)
> at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at com.cloud.storage.LocalStoragePoolListener.processConnect(LocalStoragePoolListener.java:86)
> at com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:587)
> at com.cloud.agent.manager.AgentManagerImpl.handleConnectedAgent(AgentManagerImpl.java:1085)
> at com.cloud.agent.manager.AgentManagerImpl.access$100(AgentManagerImpl.java:144)
> at com.cloud.agent.manager.AgentManagerImpl$HandleAgentConnectTask.run(AgentManagerImpl.java:1160)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira