You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Franky Hall <fr...@cartcrafter.com> on 2015/03/04 00:55:56 UTC
Unable to add new hosts to existing cluster
Hello,
I have a 4-node cluster using VMware 5.5 and CS 4.4.1. It works great, but I need to expand it. I have two more identical servers already added to the HA-cluster in vCenter (in maintenance mode). I’m trying to add them into CS via the “Add Host” button in Infrastructure->Hosts. I immediately get an error with some correlating log entries. The host has four networks configured; three of them for iSCSI and one for Management. Other than the networking I added the iSCSI data stores. I’m not sure what I’m missing or what setting I need to tweak to make these hosts add-able. Any help is greatly appreciated.
Warm regards,
Franky Hall
Here’s the relevant log snippet:
2015-03-03 15:48:31,978 DEBUG [c.c.a.ApiServlet] (catalina-exec-15:ctx-c2c30a12) ===START=== 192.168.5.115 -- POST command=addHost&response=json&sessionkey=2Pq%2BiiJSbkYbYnCSJUHRhkyO%2B%2FU%3D
2015-03-03 15:48:31,983 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Received unknown parameters for command addHost. Unknown parameters : clustertype
2015-03-03 15:48:31,985 INFO [c.c.r.ResourceManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Trying to add a new host at http://hq-core-vcenter02.blurb.com/HQ/Cluster1/hq-core-vmware07.blurb.com in data center 1
2015-03-03 15:48:31,985 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Discover host. dc: 1, pod: 1, cluster: 1, uri host: hq-core-vcenter02.blurb.com
2015-03-03 15:48:32,006 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Detected private network label : vSwitch0,140
2015-03-03 15:48:32,006 DEBUG [c.c.h.v.r.VmwareContextFactory] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) initialize VmwareContext. url: https://hq-core-vcenter02.blurb.com/sdk/vimService, username: root, password: w********
2015-03-03 15:48:32,094 INFO [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) New VmwareContext object, current outstanding count: 6
2015-03-03 15:48:32,107 ERROR [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Path element points to an un-existing inventory entity
2015-03-03 15:48:32,107 ERROR [c.c.h.v.m.VmwareManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find host from inventory path: /HQ/Cluster1/hq-core-vmware07.blurb.com
2015-03-03 15:48:32,107 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Found 0 hosts.
2015-03-03 15:48:32,107 ERROR [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find host or cluster based on url: /HQ/Cluster1/hq-core-vmware07.blurb.com
2015-03-03 15:48:32,107 INFO [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Disconnecting VMware session
2015-03-03 15:48:32,110 WARN [c.c.r.ResourceManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find the server resources at http://hq-core-vcenter02.blurb.com/HQ/Cluster1/hq-core-vmware07.blurb.com
2015-03-03 15:48:32,111 INFO [c.c.u.e.CSExceptionErrorCode] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Could not find exception: com.cloud.exception.DiscoveryException in error code list for exceptions
2015-03-03 15:48:32,111 WARN [o.a.c.a.c.a.h.AddHostCmd] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Exception:
com.cloud.exception.DiscoveryException: Unable to add the host
at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:791)
at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:586)
<some backtrace lines removed>
at java.lang.Thread.run(Thread.java:744)
2015-03-03 15:48:32,112 INFO [c.c.a.ApiServer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to add the host
2015-03-03 15:48:32,112 DEBUG [c.c.a.ApiServlet] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) ===END=== 192.168.5.115 -- POST command=addHost&response=json&sessionkey=2Pq%2BiiJSbkYbYnCSJUHRhkyO%2B%2FU%3D
Re: Unable to add new hosts to existing cluster
Posted by Franky Hall <fr...@cartcrafter.com>.
I found the problem. I doubt anyone else will run into this, but just in case..
Inside vsphere, the cluster was inside of a folder. I do not recall if I put it there before or after I added the original four hosts to cloudstack. Moving the cluster out of the folder and directly under the datacenter in vsphere seemed to do the trick. I was able to add the new hosts to cloudstack after that.
Thanks for your help!
On Mar 4, 2015, at 6:21 PM, ilya musayev <il...@gmail.com> wrote:
> I'm running CS 4.3 with vSphere 5.5 (very sizable infrastructure). I dont see these issues in 4.3
>
> 4.5 will come out hopefully this week, you can consider upgrading to see if that issue will be addressed.
>
> In the meantime, please note the time and try the addition again, then post a complete log on pastebin/google drive, usually there are multiple pointers any issue, perhaps what you posted was not complete.
>
> regards
> ilya
> On 3/4/15 9:29 AM, Franky Hall wrote:
>> Oh, I tried that originally. They are in a DRS cluster so vmware starts migrating machines to them if they’re not in maintenance mode. That’s not a good thing since they aren’t in cloudstack yet. I got the same error when they were not in maintenance mode though…
>>
>> Thanks!
>> Franky
>>
>>
>> On Mar 4, 2015, at 9:27 AM, ilya musayev <il...@gmail.com> wrote:
>>
>>> Franky,
>>>
>>> MM - Maintenance Mode.
>>>
>>> Take the hosts out of Maintenance mode and try again.
>>>
>>> Regards,
>>> ilya
>>> On 3/4/15 8:59 AM, Franky Hall wrote:
>>>> Hi Ilya,
>>>>
>>>> Thanks for your reply.
>>>>
>>>> I’m not sure what MM stands for, sorry. Please elaborate.
>>>>
>>>> As for the URL, I’m not putting it in. I just type in a hostname for the new server under add host “hq-core-vmware07.blurb.com” - the URL is generated in CS and shows in the management log.
>>>>
>>>> I did not remove anything from the log snippet except part of the back trace. I work for blurb :) The only other sensitive info is the first letter of my root password, which isn’t a big deal since that password doesn’t exist anywhere except on this cloud, and the cloud is not accessible from the Internet. Thank you for your concern.
>>>>
>>>> Regards,
>>>> Franky Hall
>>>>
>>>> On Mar 3, 2015, at 7:52 PM, ilya musayev <il...@gmail.com> wrote:
>>>>
>>>>> Frank,
>>>>>
>>>>> Take them out of MM and try again.
>>>>>
>>>>> The URL should be using HTTPS and not HTTP from what i see. Have you sanitized the for private info or this is what you get?
>>>>>
>>>>> Regards
>>>>> ilya
>>>>>
>>>>>
>>>>>
>>>>> On 3/3/15 3:55 PM, Franky Hall wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I have a 4-node cluster using VMware 5.5 and CS 4.4.1. It works great, but I need to expand it. I have two more identical servers already added to the HA-cluster in vCenter (in maintenance mode). I’m trying to add them into CS via the “Add Host” button in Infrastructure->Hosts. I immediately get an error with some correlating log entries. The host has four networks configured; three of them for iSCSI and one for Management. Other than the networking I added the iSCSI data stores. I’m not sure what I’m missing or what setting I need to tweak to make these hosts add-able. Any help is greatly appreciated.
>>>>>>
>>>>>> Warm regards,
>>>>>> Franky Hall
>>>>>>
>>>>>> Here’s the relevant log snippet:
>>>>>> 2015-03-03 15:48:31,978 DEBUG [c.c.a.ApiServlet] (catalina-exec-15:ctx-c2c30a12) ===START=== 192.168.5.115 -- POST command=addHost&response=json&sessionkey=2Pq%2BiiJSbkYbYnCSJUHRhkyO%2B%2FU%3D
>>>>>> 2015-03-03 15:48:31,983 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Received unknown parameters for command addHost. Unknown parameters : clustertype
>>>>>> 2015-03-03 15:48:31,985 INFO [c.c.r.ResourceManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Trying to add a new host at http://hq-core-vcenter02.blurb.com/HQ/Cluster1/hq-core-vmware07.blurb.com in data center 1
>>>>>> 2015-03-03 15:48:31,985 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Discover host. dc: 1, pod: 1, cluster: 1, uri host: hq-core-vcenter02.blurb.com
>>>>>> 2015-03-03 15:48:32,006 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Detected private network label : vSwitch0,140
>>>>>> 2015-03-03 15:48:32,006 DEBUG [c.c.h.v.r.VmwareContextFactory] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) initialize VmwareContext. url: https://hq-core-vcenter02.blurb.com/sdk/vimService, username: root, password: w********
>>>>>> 2015-03-03 15:48:32,094 INFO [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) New VmwareContext object, current outstanding count: 6
>>>>>> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Path element points to an un-existing inventory entity
>>>>>> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.m.VmwareManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find host from inventory path: /HQ/Cluster1/hq-core-vmware07.blurb.com
>>>>>> 2015-03-03 15:48:32,107 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Found 0 hosts.
>>>>>> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find host or cluster based on url: /HQ/Cluster1/hq-core-vmware07.blurb.com
>>>>>> 2015-03-03 15:48:32,107 INFO [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Disconnecting VMware session
>>>>>> 2015-03-03 15:48:32,110 WARN [c.c.r.ResourceManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find the server resources at http://hq-core-vcenter02.blurb.com/HQ/Cluster1/hq-core-vmware07.blurb.com
>>>>>> 2015-03-03 15:48:32,111 INFO [c.c.u.e.CSExceptionErrorCode] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Could not find exception: com.cloud.exception.DiscoveryException in error code list for exceptions
>>>>>> 2015-03-03 15:48:32,111 WARN [o.a.c.a.c.a.h.AddHostCmd] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Exception:
>>>>>> com.cloud.exception.DiscoveryException: Unable to add the host
>>>>>> at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:791)
>>>>>> at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:586)
>>>>>> <some backtrace lines removed>
>>>>>> at java.lang.Thread.run(Thread.java:744)
>>>>>> 2015-03-03 15:48:32,112 INFO [c.c.a.ApiServer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to add the host
>>>>>> 2015-03-03 15:48:32,112 DEBUG [c.c.a.ApiServlet] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) ===END=== 192.168.5.115 -- POST command=addHost&response=json&sessionkey=2Pq%2BiiJSbkYbYnCSJUHRhkyO%2B%2FU%3D
>>
>
Re: Unable to add new hosts to existing cluster
Posted by ilya musayev <il...@gmail.com>.
I'm running CS 4.3 with vSphere 5.5 (very sizable infrastructure). I
dont see these issues in 4.3
4.5 will come out hopefully this week, you can consider upgrading to see
if that issue will be addressed.
In the meantime, please note the time and try the addition again, then
post a complete log on pastebin/google drive, usually there are multiple
pointers any issue, perhaps what you posted was not complete.
regards
ilya
On 3/4/15 9:29 AM, Franky Hall wrote:
> Oh, I tried that originally. They are in a DRS cluster so vmware starts migrating machines to them if they’re not in maintenance mode. That’s not a good thing since they aren’t in cloudstack yet. I got the same error when they were not in maintenance mode though…
>
> Thanks!
> Franky
>
>
> On Mar 4, 2015, at 9:27 AM, ilya musayev <il...@gmail.com> wrote:
>
>> Franky,
>>
>> MM - Maintenance Mode.
>>
>> Take the hosts out of Maintenance mode and try again.
>>
>> Regards,
>> ilya
>> On 3/4/15 8:59 AM, Franky Hall wrote:
>>> Hi Ilya,
>>>
>>> Thanks for your reply.
>>>
>>> I’m not sure what MM stands for, sorry. Please elaborate.
>>>
>>> As for the URL, I’m not putting it in. I just type in a hostname for the new server under add host “hq-core-vmware07.blurb.com” - the URL is generated in CS and shows in the management log.
>>>
>>> I did not remove anything from the log snippet except part of the back trace. I work for blurb :) The only other sensitive info is the first letter of my root password, which isn’t a big deal since that password doesn’t exist anywhere except on this cloud, and the cloud is not accessible from the Internet. Thank you for your concern.
>>>
>>> Regards,
>>> Franky Hall
>>>
>>> On Mar 3, 2015, at 7:52 PM, ilya musayev <il...@gmail.com> wrote:
>>>
>>>> Frank,
>>>>
>>>> Take them out of MM and try again.
>>>>
>>>> The URL should be using HTTPS and not HTTP from what i see. Have you sanitized the for private info or this is what you get?
>>>>
>>>> Regards
>>>> ilya
>>>>
>>>>
>>>>
>>>> On 3/3/15 3:55 PM, Franky Hall wrote:
>>>>> Hello,
>>>>>
>>>>> I have a 4-node cluster using VMware 5.5 and CS 4.4.1. It works great, but I need to expand it. I have two more identical servers already added to the HA-cluster in vCenter (in maintenance mode). I’m trying to add them into CS via the “Add Host” button in Infrastructure->Hosts. I immediately get an error with some correlating log entries. The host has four networks configured; three of them for iSCSI and one for Management. Other than the networking I added the iSCSI data stores. I’m not sure what I’m missing or what setting I need to tweak to make these hosts add-able. Any help is greatly appreciated.
>>>>>
>>>>> Warm regards,
>>>>> Franky Hall
>>>>>
>>>>> Here’s the relevant log snippet:
>>>>> 2015-03-03 15:48:31,978 DEBUG [c.c.a.ApiServlet] (catalina-exec-15:ctx-c2c30a12) ===START=== 192.168.5.115 -- POST command=addHost&response=json&sessionkey=2Pq%2BiiJSbkYbYnCSJUHRhkyO%2B%2FU%3D
>>>>> 2015-03-03 15:48:31,983 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Received unknown parameters for command addHost. Unknown parameters : clustertype
>>>>> 2015-03-03 15:48:31,985 INFO [c.c.r.ResourceManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Trying to add a new host at http://hq-core-vcenter02.blurb.com/HQ/Cluster1/hq-core-vmware07.blurb.com in data center 1
>>>>> 2015-03-03 15:48:31,985 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Discover host. dc: 1, pod: 1, cluster: 1, uri host: hq-core-vcenter02.blurb.com
>>>>> 2015-03-03 15:48:32,006 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Detected private network label : vSwitch0,140
>>>>> 2015-03-03 15:48:32,006 DEBUG [c.c.h.v.r.VmwareContextFactory] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) initialize VmwareContext. url: https://hq-core-vcenter02.blurb.com/sdk/vimService, username: root, password: w********
>>>>> 2015-03-03 15:48:32,094 INFO [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) New VmwareContext object, current outstanding count: 6
>>>>> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Path element points to an un-existing inventory entity
>>>>> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.m.VmwareManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find host from inventory path: /HQ/Cluster1/hq-core-vmware07.blurb.com
>>>>> 2015-03-03 15:48:32,107 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Found 0 hosts.
>>>>> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find host or cluster based on url: /HQ/Cluster1/hq-core-vmware07.blurb.com
>>>>> 2015-03-03 15:48:32,107 INFO [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Disconnecting VMware session
>>>>> 2015-03-03 15:48:32,110 WARN [c.c.r.ResourceManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find the server resources at http://hq-core-vcenter02.blurb.com/HQ/Cluster1/hq-core-vmware07.blurb.com
>>>>> 2015-03-03 15:48:32,111 INFO [c.c.u.e.CSExceptionErrorCode] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Could not find exception: com.cloud.exception.DiscoveryException in error code list for exceptions
>>>>> 2015-03-03 15:48:32,111 WARN [o.a.c.a.c.a.h.AddHostCmd] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Exception:
>>>>> com.cloud.exception.DiscoveryException: Unable to add the host
>>>>> at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:791)
>>>>> at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:586)
>>>>> <some backtrace lines removed>
>>>>> at java.lang.Thread.run(Thread.java:744)
>>>>> 2015-03-03 15:48:32,112 INFO [c.c.a.ApiServer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to add the host
>>>>> 2015-03-03 15:48:32,112 DEBUG [c.c.a.ApiServlet] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) ===END=== 192.168.5.115 -- POST command=addHost&response=json&sessionkey=2Pq%2BiiJSbkYbYnCSJUHRhkyO%2B%2FU%3D
>
Re: Unable to add new hosts to existing cluster
Posted by Franky Hall <fr...@cartcrafter.com>.
Oh, I tried that originally. They are in a DRS cluster so vmware starts migrating machines to them if they’re not in maintenance mode. That’s not a good thing since they aren’t in cloudstack yet. I got the same error when they were not in maintenance mode though…
Thanks!
Franky
On Mar 4, 2015, at 9:27 AM, ilya musayev <il...@gmail.com> wrote:
> Franky,
>
> MM - Maintenance Mode.
>
> Take the hosts out of Maintenance mode and try again.
>
> Regards,
> ilya
> On 3/4/15 8:59 AM, Franky Hall wrote:
>> Hi Ilya,
>>
>> Thanks for your reply.
>>
>> I’m not sure what MM stands for, sorry. Please elaborate.
>>
>> As for the URL, I’m not putting it in. I just type in a hostname for the new server under add host “hq-core-vmware07.blurb.com” - the URL is generated in CS and shows in the management log.
>>
>> I did not remove anything from the log snippet except part of the back trace. I work for blurb :) The only other sensitive info is the first letter of my root password, which isn’t a big deal since that password doesn’t exist anywhere except on this cloud, and the cloud is not accessible from the Internet. Thank you for your concern.
>>
>> Regards,
>> Franky Hall
>>
>> On Mar 3, 2015, at 7:52 PM, ilya musayev <il...@gmail.com> wrote:
>>
>>> Frank,
>>>
>>> Take them out of MM and try again.
>>>
>>> The URL should be using HTTPS and not HTTP from what i see. Have you sanitized the for private info or this is what you get?
>>>
>>> Regards
>>> ilya
>>>
>>>
>>>
>>> On 3/3/15 3:55 PM, Franky Hall wrote:
>>>> Hello,
>>>>
>>>> I have a 4-node cluster using VMware 5.5 and CS 4.4.1. It works great, but I need to expand it. I have two more identical servers already added to the HA-cluster in vCenter (in maintenance mode). I’m trying to add them into CS via the “Add Host” button in Infrastructure->Hosts. I immediately get an error with some correlating log entries. The host has four networks configured; three of them for iSCSI and one for Management. Other than the networking I added the iSCSI data stores. I’m not sure what I’m missing or what setting I need to tweak to make these hosts add-able. Any help is greatly appreciated.
>>>>
>>>> Warm regards,
>>>> Franky Hall
>>>>
>>>> Here’s the relevant log snippet:
>>>> 2015-03-03 15:48:31,978 DEBUG [c.c.a.ApiServlet] (catalina-exec-15:ctx-c2c30a12) ===START=== 192.168.5.115 -- POST command=addHost&response=json&sessionkey=2Pq%2BiiJSbkYbYnCSJUHRhkyO%2B%2FU%3D
>>>> 2015-03-03 15:48:31,983 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Received unknown parameters for command addHost. Unknown parameters : clustertype
>>>> 2015-03-03 15:48:31,985 INFO [c.c.r.ResourceManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Trying to add a new host at http://hq-core-vcenter02.blurb.com/HQ/Cluster1/hq-core-vmware07.blurb.com in data center 1
>>>> 2015-03-03 15:48:31,985 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Discover host. dc: 1, pod: 1, cluster: 1, uri host: hq-core-vcenter02.blurb.com
>>>> 2015-03-03 15:48:32,006 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Detected private network label : vSwitch0,140
>>>> 2015-03-03 15:48:32,006 DEBUG [c.c.h.v.r.VmwareContextFactory] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) initialize VmwareContext. url: https://hq-core-vcenter02.blurb.com/sdk/vimService, username: root, password: w********
>>>> 2015-03-03 15:48:32,094 INFO [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) New VmwareContext object, current outstanding count: 6
>>>> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Path element points to an un-existing inventory entity
>>>> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.m.VmwareManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find host from inventory path: /HQ/Cluster1/hq-core-vmware07.blurb.com
>>>> 2015-03-03 15:48:32,107 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Found 0 hosts.
>>>> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find host or cluster based on url: /HQ/Cluster1/hq-core-vmware07.blurb.com
>>>> 2015-03-03 15:48:32,107 INFO [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Disconnecting VMware session
>>>> 2015-03-03 15:48:32,110 WARN [c.c.r.ResourceManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find the server resources at http://hq-core-vcenter02.blurb.com/HQ/Cluster1/hq-core-vmware07.blurb.com
>>>> 2015-03-03 15:48:32,111 INFO [c.c.u.e.CSExceptionErrorCode] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Could not find exception: com.cloud.exception.DiscoveryException in error code list for exceptions
>>>> 2015-03-03 15:48:32,111 WARN [o.a.c.a.c.a.h.AddHostCmd] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Exception:
>>>> com.cloud.exception.DiscoveryException: Unable to add the host
>>>> at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:791)
>>>> at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:586)
>>>> <some backtrace lines removed>
>>>> at java.lang.Thread.run(Thread.java:744)
>>>> 2015-03-03 15:48:32,112 INFO [c.c.a.ApiServer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to add the host
>>>> 2015-03-03 15:48:32,112 DEBUG [c.c.a.ApiServlet] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) ===END=== 192.168.5.115 -- POST command=addHost&response=json&sessionkey=2Pq%2BiiJSbkYbYnCSJUHRhkyO%2B%2FU%3D
>>
>
Re: Unable to add new hosts to existing cluster
Posted by ilya musayev <il...@gmail.com>.
Franky,
MM - Maintenance Mode.
Take the hosts out of Maintenance mode and try again.
Regards,
ilya
On 3/4/15 8:59 AM, Franky Hall wrote:
> Hi Ilya,
>
> Thanks for your reply.
>
> I’m not sure what MM stands for, sorry. Please elaborate.
>
> As for the URL, I’m not putting it in. I just type in a hostname for the new server under add host “hq-core-vmware07.blurb.com” - the URL is generated in CS and shows in the management log.
>
> I did not remove anything from the log snippet except part of the back trace. I work for blurb :) The only other sensitive info is the first letter of my root password, which isn’t a big deal since that password doesn’t exist anywhere except on this cloud, and the cloud is not accessible from the Internet. Thank you for your concern.
>
> Regards,
> Franky Hall
>
> On Mar 3, 2015, at 7:52 PM, ilya musayev <il...@gmail.com> wrote:
>
>> Frank,
>>
>> Take them out of MM and try again.
>>
>> The URL should be using HTTPS and not HTTP from what i see. Have you sanitized the for private info or this is what you get?
>>
>> Regards
>> ilya
>>
>>
>>
>> On 3/3/15 3:55 PM, Franky Hall wrote:
>>> Hello,
>>>
>>> I have a 4-node cluster using VMware 5.5 and CS 4.4.1. It works great, but I need to expand it. I have two more identical servers already added to the HA-cluster in vCenter (in maintenance mode). I’m trying to add them into CS via the “Add Host” button in Infrastructure->Hosts. I immediately get an error with some correlating log entries. The host has four networks configured; three of them for iSCSI and one for Management. Other than the networking I added the iSCSI data stores. I’m not sure what I’m missing or what setting I need to tweak to make these hosts add-able. Any help is greatly appreciated.
>>>
>>> Warm regards,
>>> Franky Hall
>>>
>>> Here’s the relevant log snippet:
>>> 2015-03-03 15:48:31,978 DEBUG [c.c.a.ApiServlet] (catalina-exec-15:ctx-c2c30a12) ===START=== 192.168.5.115 -- POST command=addHost&response=json&sessionkey=2Pq%2BiiJSbkYbYnCSJUHRhkyO%2B%2FU%3D
>>> 2015-03-03 15:48:31,983 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Received unknown parameters for command addHost. Unknown parameters : clustertype
>>> 2015-03-03 15:48:31,985 INFO [c.c.r.ResourceManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Trying to add a new host at http://hq-core-vcenter02.blurb.com/HQ/Cluster1/hq-core-vmware07.blurb.com in data center 1
>>> 2015-03-03 15:48:31,985 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Discover host. dc: 1, pod: 1, cluster: 1, uri host: hq-core-vcenter02.blurb.com
>>> 2015-03-03 15:48:32,006 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Detected private network label : vSwitch0,140
>>> 2015-03-03 15:48:32,006 DEBUG [c.c.h.v.r.VmwareContextFactory] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) initialize VmwareContext. url: https://hq-core-vcenter02.blurb.com/sdk/vimService, username: root, password: w********
>>> 2015-03-03 15:48:32,094 INFO [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) New VmwareContext object, current outstanding count: 6
>>> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Path element points to an un-existing inventory entity
>>> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.m.VmwareManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find host from inventory path: /HQ/Cluster1/hq-core-vmware07.blurb.com
>>> 2015-03-03 15:48:32,107 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Found 0 hosts.
>>> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find host or cluster based on url: /HQ/Cluster1/hq-core-vmware07.blurb.com
>>> 2015-03-03 15:48:32,107 INFO [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Disconnecting VMware session
>>> 2015-03-03 15:48:32,110 WARN [c.c.r.ResourceManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find the server resources at http://hq-core-vcenter02.blurb.com/HQ/Cluster1/hq-core-vmware07.blurb.com
>>> 2015-03-03 15:48:32,111 INFO [c.c.u.e.CSExceptionErrorCode] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Could not find exception: com.cloud.exception.DiscoveryException in error code list for exceptions
>>> 2015-03-03 15:48:32,111 WARN [o.a.c.a.c.a.h.AddHostCmd] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Exception:
>>> com.cloud.exception.DiscoveryException: Unable to add the host
>>> at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:791)
>>> at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:586)
>>> <some backtrace lines removed>
>>> at java.lang.Thread.run(Thread.java:744)
>>> 2015-03-03 15:48:32,112 INFO [c.c.a.ApiServer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to add the host
>>> 2015-03-03 15:48:32,112 DEBUG [c.c.a.ApiServlet] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) ===END=== 192.168.5.115 -- POST command=addHost&response=json&sessionkey=2Pq%2BiiJSbkYbYnCSJUHRhkyO%2B%2FU%3D
>
Re: Unable to add new hosts to existing cluster
Posted by Franky Hall <fr...@cartcrafter.com>.
Hi Ilya,
Thanks for your reply.
I’m not sure what MM stands for, sorry. Please elaborate.
As for the URL, I’m not putting it in. I just type in a hostname for the new server under add host “hq-core-vmware07.blurb.com” - the URL is generated in CS and shows in the management log.
I did not remove anything from the log snippet except part of the back trace. I work for blurb :) The only other sensitive info is the first letter of my root password, which isn’t a big deal since that password doesn’t exist anywhere except on this cloud, and the cloud is not accessible from the Internet. Thank you for your concern.
Regards,
Franky Hall
On Mar 3, 2015, at 7:52 PM, ilya musayev <il...@gmail.com> wrote:
> Frank,
>
> Take them out of MM and try again.
>
> The URL should be using HTTPS and not HTTP from what i see. Have you sanitized the for private info or this is what you get?
>
> Regards
> ilya
>
>
>
> On 3/3/15 3:55 PM, Franky Hall wrote:
>> Hello,
>>
>> I have a 4-node cluster using VMware 5.5 and CS 4.4.1. It works great, but I need to expand it. I have two more identical servers already added to the HA-cluster in vCenter (in maintenance mode). I’m trying to add them into CS via the “Add Host” button in Infrastructure->Hosts. I immediately get an error with some correlating log entries. The host has four networks configured; three of them for iSCSI and one for Management. Other than the networking I added the iSCSI data stores. I’m not sure what I’m missing or what setting I need to tweak to make these hosts add-able. Any help is greatly appreciated.
>>
>> Warm regards,
>> Franky Hall
>>
>> Here’s the relevant log snippet:
>> 2015-03-03 15:48:31,978 DEBUG [c.c.a.ApiServlet] (catalina-exec-15:ctx-c2c30a12) ===START=== 192.168.5.115 -- POST command=addHost&response=json&sessionkey=2Pq%2BiiJSbkYbYnCSJUHRhkyO%2B%2FU%3D
>> 2015-03-03 15:48:31,983 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Received unknown parameters for command addHost. Unknown parameters : clustertype
>> 2015-03-03 15:48:31,985 INFO [c.c.r.ResourceManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Trying to add a new host at http://hq-core-vcenter02.blurb.com/HQ/Cluster1/hq-core-vmware07.blurb.com in data center 1
>> 2015-03-03 15:48:31,985 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Discover host. dc: 1, pod: 1, cluster: 1, uri host: hq-core-vcenter02.blurb.com
>> 2015-03-03 15:48:32,006 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Detected private network label : vSwitch0,140
>> 2015-03-03 15:48:32,006 DEBUG [c.c.h.v.r.VmwareContextFactory] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) initialize VmwareContext. url: https://hq-core-vcenter02.blurb.com/sdk/vimService, username: root, password: w********
>> 2015-03-03 15:48:32,094 INFO [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) New VmwareContext object, current outstanding count: 6
>> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Path element points to an un-existing inventory entity
>> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.m.VmwareManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find host from inventory path: /HQ/Cluster1/hq-core-vmware07.blurb.com
>> 2015-03-03 15:48:32,107 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Found 0 hosts.
>> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find host or cluster based on url: /HQ/Cluster1/hq-core-vmware07.blurb.com
>> 2015-03-03 15:48:32,107 INFO [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Disconnecting VMware session
>> 2015-03-03 15:48:32,110 WARN [c.c.r.ResourceManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find the server resources at http://hq-core-vcenter02.blurb.com/HQ/Cluster1/hq-core-vmware07.blurb.com
>> 2015-03-03 15:48:32,111 INFO [c.c.u.e.CSExceptionErrorCode] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Could not find exception: com.cloud.exception.DiscoveryException in error code list for exceptions
>> 2015-03-03 15:48:32,111 WARN [o.a.c.a.c.a.h.AddHostCmd] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Exception:
>> com.cloud.exception.DiscoveryException: Unable to add the host
>> at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:791)
>> at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:586)
>> <some backtrace lines removed>
>> at java.lang.Thread.run(Thread.java:744)
>> 2015-03-03 15:48:32,112 INFO [c.c.a.ApiServer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to add the host
>> 2015-03-03 15:48:32,112 DEBUG [c.c.a.ApiServlet] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) ===END=== 192.168.5.115 -- POST command=addHost&response=json&sessionkey=2Pq%2BiiJSbkYbYnCSJUHRhkyO%2B%2FU%3D
>
Re: Unable to add new hosts to existing cluster
Posted by ilya musayev <il...@gmail.com>.
Frank,
Take them out of MM and try again.
The URL should be using HTTPS and not HTTP from what i see. Have you
sanitized the for private info or this is what you get?
Regards
ilya
On 3/3/15 3:55 PM, Franky Hall wrote:
> Hello,
>
> I have a 4-node cluster using VMware 5.5 and CS 4.4.1. It works great, but I need to expand it. I have two more identical servers already added to the HA-cluster in vCenter (in maintenance mode). I’m trying to add them into CS via the “Add Host” button in Infrastructure->Hosts. I immediately get an error with some correlating log entries. The host has four networks configured; three of them for iSCSI and one for Management. Other than the networking I added the iSCSI data stores. I’m not sure what I’m missing or what setting I need to tweak to make these hosts add-able. Any help is greatly appreciated.
>
> Warm regards,
> Franky Hall
>
> Here’s the relevant log snippet:
> 2015-03-03 15:48:31,978 DEBUG [c.c.a.ApiServlet] (catalina-exec-15:ctx-c2c30a12) ===START=== 192.168.5.115 -- POST command=addHost&response=json&sessionkey=2Pq%2BiiJSbkYbYnCSJUHRhkyO%2B%2FU%3D
> 2015-03-03 15:48:31,983 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Received unknown parameters for command addHost. Unknown parameters : clustertype
> 2015-03-03 15:48:31,985 INFO [c.c.r.ResourceManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Trying to add a new host at http://hq-core-vcenter02.blurb.com/HQ/Cluster1/hq-core-vmware07.blurb.com in data center 1
> 2015-03-03 15:48:31,985 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Discover host. dc: 1, pod: 1, cluster: 1, uri host: hq-core-vcenter02.blurb.com
> 2015-03-03 15:48:32,006 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Detected private network label : vSwitch0,140
> 2015-03-03 15:48:32,006 DEBUG [c.c.h.v.r.VmwareContextFactory] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) initialize VmwareContext. url: https://hq-core-vcenter02.blurb.com/sdk/vimService, username: root, password: w********
> 2015-03-03 15:48:32,094 INFO [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) New VmwareContext object, current outstanding count: 6
> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Path element points to an un-existing inventory entity
> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.m.VmwareManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find host from inventory path: /HQ/Cluster1/hq-core-vmware07.blurb.com
> 2015-03-03 15:48:32,107 INFO [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Found 0 hosts.
> 2015-03-03 15:48:32,107 ERROR [c.c.h.v.VmwareServerDiscoverer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find host or cluster based on url: /HQ/Cluster1/hq-core-vmware07.blurb.com
> 2015-03-03 15:48:32,107 INFO [c.c.h.v.u.VmwareContext] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Disconnecting VMware session
> 2015-03-03 15:48:32,110 WARN [c.c.r.ResourceManagerImpl] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to find the server resources at http://hq-core-vcenter02.blurb.com/HQ/Cluster1/hq-core-vmware07.blurb.com
> 2015-03-03 15:48:32,111 INFO [c.c.u.e.CSExceptionErrorCode] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Could not find exception: com.cloud.exception.DiscoveryException in error code list for exceptions
> 2015-03-03 15:48:32,111 WARN [o.a.c.a.c.a.h.AddHostCmd] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Exception:
> com.cloud.exception.DiscoveryException: Unable to add the host
> at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:791)
> at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:586)
> <some backtrace lines removed>
> at java.lang.Thread.run(Thread.java:744)
> 2015-03-03 15:48:32,112 INFO [c.c.a.ApiServer] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) Unable to add the host
> 2015-03-03 15:48:32,112 DEBUG [c.c.a.ApiServlet] (catalina-exec-15:ctx-c2c30a12 ctx-55597ee3) ===END=== 192.168.5.115 -- POST command=addHost&response=json&sessionkey=2Pq%2BiiJSbkYbYnCSJUHRhkyO%2B%2FU%3D