You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Ciobanu Cristian <cr...@istream.today> on 2017/10/11 12:26:16 UTC

Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Hello,

 

   In order to upgrade cloudstack to 4.9 I have followed this doecumentation
:
http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.9.2
.0/upgrade/upgrade-4.5.html  and ended with this : 

 

management-server.log:2017-10-11 08:05:35,612 WARN
[o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:) Exception
encountered during context initialization - cancelling refresh attempt:
org.springframework.context.ApplicationContextException: Failed to start
bean 'cloudStackLifeCycle'; nested exception is
com.cloud.utils.exception.CloudRuntimeException: Unable to upgrade the
database

 

Is there a valid upgrade procedure or script for : Cloudstack 4.5.2 to
CloudStack 4.9 ( VMware with Local Storage ) ?

 

(cloudstack was installed from shapeblue repo)

 

 

Thank you.

Cristian

 

 

 


Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Daan Hoogland <da...@gmail.com>.
did you see any stack trace around the place you saw that error; above or
just below?
did you see any warning messages or suspicious debug statements above the
error message?
can you double check the package installation of the cloud* packages on
your machine and give us the versions?

​the procedure you have followed is the correct and current one​.

On Wed, Oct 11, 2017 at 2:51 PM, Ciobanu Cristian <cr...@istream.today>
wrote:

> I have used the following info in my repo: baseurl=http://packages.
> shapeblue.com/cloudstack/upstream/centos/4.9 , the 4.9.3 was installed.
>
>
> Regards,
> Cristian
>
> -----Original Message-----
> From: Rene Moser [mailto:mail@renemoser.net]
> Sent: Wednesday, October 11, 2017 3:38 PM
> To: users@cloudstack.apache.org
> Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database
>
> did you upgrade to 4.9.2 or 4.9.3? centos?
>
> Regards
> René
>
>


-- 
Daan

RE: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Ciobanu Cristian <cr...@istream.today>.
I have used the following info in my repo: baseurl=http://packages.shapeblue.com/cloudstack/upstream/centos/4.9 , the 4.9.3 was installed.


Regards,
Cristian

-----Original Message-----
From: Rene Moser [mailto:mail@renemoser.net] 
Sent: Wednesday, October 11, 2017 3:38 PM
To: users@cloudstack.apache.org
Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

did you upgrade to 4.9.2 or 4.9.3? centos?

Regards
René


Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Rene Moser <ma...@renemoser.net>.
did you upgrade to 4.9.2 or 4.9.3? centos?

Regards
René

RE: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Ciobanu Cristian <cr...@istream.today>.
Just opened the ticket, sorry for my misunderstood.

Regards,
Cristian

-----Original Message-----
From: Daan Hoogland [mailto:daan.hoogland@gmail.com] 
Sent: Friday, October 13, 2017 2:55 PM
To: users@cloudstack.apache.org; dev <de...@cloudstack.apache.org>
Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

sorry Cristian, I wasn't clear. Can you add a jira ticket at issues.apache.org/jira?

@devs I have heard this before but am not sure if it was exactly the same.
Anybody else? Was this fixed in 4.10 or master yet?
tl;dr setting password doesn't work for Cristian due to the firewall on the VR.

On Fri, Oct 13, 2017 at 1:44 PM, Ciobanu Cristian <cr...@istream.today>
wrote:

> Daan,
>
>     Can I track this issue somewhere?
>
>
> Thank you!
> Cristian
>
> -----Original Message-----
> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> Sent: Thursday, October 12, 2017 9:46 AM
> To: users@cloudstack.apache.org
> Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the 
> database
>
> Cristian,
>
> I am cross-posting this to dev@coudstack.a.o, sorry for the noise. I 
> have heard this before but am not sure if it was exactly the same. It 
> sounds like a bug to me. Can you enter one in jira?
> Anybody else? Was this fixed in 4.10 or master yet?
>
> On Wed, Oct 11, 2017 at 11:23 PM, Ciobanu Cristian 
> <cristian.c@istream.today
> > wrote:
>
> > Hello,
> >
> >  After I continued the investigation I found that router is not 
> > allowing the VM to fetch the password (port 8080) not sure what is 
> > wrong with this version but in order to fix I have added this rule 
> > "-A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT"  on virtual router..
> >
> >   Any idea how to fix this, permanent?  In case reinstall the router 
> > for sure I will have to do this again.
> >
> >
> > Thank you.
> > Cristian
> >
> > -----Original Message-----
> > From: Ciobanu Cristian [mailto:cristian.c@istream.today]
> > Sent: Wednesday, October 11, 2017 9:45 PM
> > To: 'users@cloudstack.apache.org' <us...@cloudstack.apache.org>
> > Subject: RE: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the 
> > database
> >
> > I was able to update the virtual router only after I have restarted 
> > the iptables-persistend form virtual router side, not sure what is 
> > wrong with this router version but I don't get the right status if I 
> > don't restart the iptables from virtual router, I have did multiple
> tests :
> >
> > Example :
> >  -  Reboot router
> > - getting this error : job-3247/job-3248, cmd: CheckSshCommand)
> > (logid:6b3de9fa) Trying to
> > > connect to 149.56.xxx.xx  ( I see this error in management log )
> > - restart iptables from router side
> > - works
> >
> >   In case I destroy/restart the router I have to repeat the same 
> > thing, restart iptables...
> >
> >   I also tested a new deploy with a template used before on this 
> > setup, looks like the password set for new deploy is not working...
> >
> >
> > Any idea why I have those issue?
> >
> > Upgraded from 4.5.2 to 4.9.3 ( shapeblue packages )
> >
> >
> > Regards,
> > Cristian
> >
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > Sent: Wednesday, October 11, 2017 6:10 PM
> > To: users@cloudstack.apache.org
> > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the 
> > database
> >
> > that is a tricky one. You can destroy the vm from VmWare and set the 
> > state to stopped in the DB. It should then start a new one.
> > check if r-4-VM/ROOT-4-000001.vmdk has gone from the datastore.
> >
> > the cause is probably some synchronisation issue. We've seen and 
> > solved it before but are yet to find the root cause.
> >
> >
> > On Wed, Oct 11, 2017 at 4:23 PM, Ciobanu Cristian 
> > <cristian.c@istream.today
> > >
> > wrote:
> >
> > > Hello,
> > >
> > >    Was my mistake, forgot to add the system template from 
> > > cloudstack interface before starting the upgrade, now I have other 
> > > issue, looks like after I have enabled one of the zone and click 
> > > on upgrade virtual router I still see the state "starting" if I 
> > > check the router from vSphere side is up.
> > >
> > >
> > >   This is what is see :
> > >
> > > tting
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Chose disk 
> > > controller for vol ROOT -> osdefault, based on root disk 
> > > controller settings at global configuration setting.
> > > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Folder r-4-VM exists on datastore 
> > > catalina.out:INFO [c.c.h.v.m.DatastoreMO] 
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] 
> > > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on 
> > > datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Search file ROOT-4-000001-flat.vmdk on 
> > > [datastore1] catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001-flat.vmdk exists 
> > > on datastore catalina.out:INFO  
> > > [c.c.s.r.VmwareStorageLayoutHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync 
> > > [datastore1] ROOT-4-000001-flat.vmdk->[datastore1]
> > > r-4-VM/ROOT-4-000001-flat.vmdk catalina.out:INFO 
> > > [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 
> > > ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Search file ROOT-4-000001-delta.vmdk on 
> > > [datastore1] catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001-delta.vmdk does 
> > > not exist on datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] 
> > > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on 
> > > datastore catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync 
> > > [datastore1] ROOT-4-000001.vmdk->[datastore1] 
> > > r-4-VM/ROOT-4-000001.vmdk catalina.out:INFO  
> > > [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] 
> > > r-4-VM catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) File [datastore1] r-4-VM/ROOT-4-000001.vmdk 
> > > exists on datastore catalina.out:INFO  
> > > [c.c.u.n.n.p.NiciraNvpApiVersion]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) NSX API VERSION:
> > > NOT PRESENT
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC 
> > > device based on NicTO: {"deviceId":0,"networkRateMbps":200,"
> > > defaultNic":true,"pxeDisable":true,"nicUuid":"fc405400-6839-
> > > 4eea-9b30-e5e14b375322","uuid":"41f42309-8eaa-425a-a4c5-
> > > c8b5a88f92a2","ip":"149.56.xxx.xx","netmask":"255.255.
> > > 255.224","gateway":"149.56.164.62","mac":"06:76:62:00:00:
> > > 4b","dns1":"172.20.0.3","dns2":"8.8.8.8","broadcastType":"
> > > Native","type":"Guest","broadcastUri":"vlan://untagged","
> > > isSecurityGroupEnabled":false,"name":"vSwitch1"}
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare 
> > > network on vmwaresvs vSwitch1 with name prefix: cloud.guest 
> > > catalina.out:INFO [c.c.h.v.m.HypervisorHostHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 
> > > catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 
> > > catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing 
> > > NIC device on network cloud.guest.untagged.200.1-vSwitch1
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC 
> > > device based on NicTO: {"deviceId":1,"networkRateMbps":-1,"
> > > defaultNic":false,"pxeDisable":true,"nicUuid":"fe3f7db4-
> > > 9ff5-4b63-883b-e08bb6f7568b","uuid":"8ef0b547-f569-4987-
> > > ac80-9ebf72f44fdd","ip":"0.0.0.0","netmask":"0.0.0.0","
> > > gateway":"0.0.0.0","mac":"02:00:5c:9c:00:0a","
> > broadcastType":"LinkLocal","
> > > type":"Control","isSecurityGroupEnabled":false,"name":"vSwitch1"}
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare 
> > > network on vmwaresvs vSwitch1 with name prefix: cloud.private 
> > > catalina.out:WARN  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) 
> > > Unrecognized broadcast type in VmwareResource, type: LinkLocal. 
> > > Use vlan info from
> > > labeling: untagged
> > > catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > > cloud.private.untagged.0.1-vSwitch1 is ready on vSwitch vSwitch1 
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing 
> > > NIC device on network cloud.private.untagged.0.1-vSwitch1
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Configure 
> > > VNC port for VM r-4-VM, port: 5939, host: 
> > > ns53xxxx.ip-149-56-xx.net catalina.out:INFO  
> > > [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Found 
> > > existing disk info from volume path: ROOT-4-000001 
> > > catalina.out:INFO [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying 
> > > to connect to 149.56.xxx.xx catalina.out:WARN 
> > > [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-8baef180)
> > > (logid:c6f5e75e) Task (job-3247) has been pending for 129 seconds 
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could 
> > > not connect to 149.56.xxx.xx due to java.net.ConnectException:
> > > Connection timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying 
> > > to connect to 149.56.xxx.xx catalina.out:WARN 
> > > [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-3026cb48)
> > > (logid:9ea6ca9e) Task (job-3247) has been pending for 189 seconds 
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could 
> > > not connect to 149.56.xxx.xx due to java.net.ConnectException:
> > > Connection timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying 
> > > to connect to 149.56.xxx.xx
> > >
> > >
> > > Any suggestion? Please.
> > >
> > > Regards,
> > > Cristian
> > >
> > > -----Original Message-----
> > > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > > Sent: Wednesday, October 11, 2017 3:41 PM
> > > To: users@cloudstack.apache.org
> > > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the 
> > > database
> > >
> > > Christian,
> > >
> > > The chances are I have not done the exact same upgrade as you are 
> > > attempting. the generic upgrade of the database (assuming the 
> > > right template and other things) is quite straight forward. You 
> > > will need to give us at least the stacktrace and any related 
> > > information. from the log above it, if this is a reproducible error.
> > >
> > > On Wed, Oct 11, 2017 at 2:34 PM, Ciobanu Cristian 
> > > <cristian.c@istream.today
> > > >
> > > wrote:
> > >
> > > > I have restored back to 4.5.2, but if I upgrade again using the 
> > > > steps provide in documentation I will end with the same error..
> > > > did you tested this? With tha configuration (acs 4.5.2, vmware, 
> > > > local
> > storage) ?
> > > >
> > > > Regards,
> > > > Cristian
> > > >
> > > > -----Original Message-----
> > > > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > > > Sent: Wednesday, October 11, 2017 3:29 PM
> > > > To: users@cloudstack.apache.org
> > > > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade 
> > > > the database
> > > >
> > > > Hi the management server on starting up should take care of the
> > upgrade.
> > > > if you have a clustered MS make sure to start only one. In your 
> > > > case after the failure you should probably restore the latest 
> > > > backup before
> > > continuing.
> > > >
> > > > On Wed, Oct 11, 2017 at 2:26 PM, Ciobanu Cristian 
> > > > <cristian.c@istream.today
> > > > >
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > >
> > > > >
> > > > >    In order to upgrade cloudstack to 4.9 I have followed this 
> > > > > doecumentation
> > > > > :
> > > > > http://docs.cloudstack.apache.org/projects/cloudstack-
> > > > > release-notes/en/4.9.2
> > > > > .0/upgrade/upgrade-4.5.html  and ended with this :
> > > > >
> > > > >
> > > > >
> > > > > management-server.log:2017-10-11 08:05:35,612 WARN 
> > > > > [o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:) 
> > > > > Exception encountered during context initialization - 
> > > > > cancelling refresh
> > > > attempt:
> > > > > org.springframework.context.ApplicationContextException: 
> > > > > Failed to start bean 'cloudStackLifeCycle'; nested exception 
> > > > > is
> > > > > com.cloud.utils.exception.CloudRuntimeException: Unable to 
> > > > > upgrade the database
> > > > >
> > > > >
> > > > >
> > > > > Is there a valid upgrade procedure or script for : Cloudstack
> > > > > 4.5.2 to CloudStack 4.9 ( VMware with Local Storage ) ?
> > > > >
> > > > >
> > > > >
> > > > > (cloudstack was installed from shapeblue repo)
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Thank you.
> > > > >
> > > > > Cristian
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Daan
> > > >
> > > >
> > >
> > >
> > > --
> > > Daan
> > >
> > >
> >
> >
> > --
> > Daan
> >
> >
>
>
> --
> Daan
>
>


--
Daan


Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Daan Hoogland <da...@gmail.com>.
sorry Cristian, I wasn't clear. Can you add a jira ticket at
issues.apache.org/jira?

@devs I have heard this before but am not sure if it was exactly the same.
Anybody else? Was this fixed in 4.10 or master yet?
tl;dr setting password doesn't work for Cristian due to the firewall on the
VR.

On Fri, Oct 13, 2017 at 1:44 PM, Ciobanu Cristian <cr...@istream.today>
wrote:

> Daan,
>
>     Can I track this issue somewhere?
>
>
> Thank you!
> Cristian
>
> -----Original Message-----
> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> Sent: Thursday, October 12, 2017 9:46 AM
> To: users@cloudstack.apache.org
> Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database
>
> Cristian,
>
> I am cross-posting this to dev@coudstack.a.o, sorry for the noise. I have
> heard this before but am not sure if it was exactly the same. It sounds
> like a bug to me. Can you enter one in jira?
> Anybody else? Was this fixed in 4.10 or master yet?
>
> On Wed, Oct 11, 2017 at 11:23 PM, Ciobanu Cristian
> <cristian.c@istream.today
> > wrote:
>
> > Hello,
> >
> >  After I continued the investigation I found that router is not
> > allowing the VM to fetch the password (port 8080) not sure what is
> > wrong with this version but in order to fix I have added this rule "-A
> > INPUT -p tcp -m tcp --dport 8080 -j ACCEPT"  on virtual router..
> >
> >   Any idea how to fix this, permanent?  In case reinstall the router
> > for sure I will have to do this again.
> >
> >
> > Thank you.
> > Cristian
> >
> > -----Original Message-----
> > From: Ciobanu Cristian [mailto:cristian.c@istream.today]
> > Sent: Wednesday, October 11, 2017 9:45 PM
> > To: 'users@cloudstack.apache.org' <us...@cloudstack.apache.org>
> > Subject: RE: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the
> > database
> >
> > I was able to update the virtual router only after I have restarted
> > the iptables-persistend form virtual router side, not sure what is
> > wrong with this router version but I don't get the right status if I
> > don't restart the iptables from virtual router, I have did multiple
> tests :
> >
> > Example :
> >  -  Reboot router
> > - getting this error : job-3247/job-3248, cmd: CheckSshCommand)
> > (logid:6b3de9fa) Trying to
> > > connect to 149.56.xxx.xx  ( I see this error in management log )
> > - restart iptables from router side
> > - works
> >
> >   In case I destroy/restart the router I have to repeat the same
> > thing, restart iptables...
> >
> >   I also tested a new deploy with a template used before on this
> > setup, looks like the password set for new deploy is not working...
> >
> >
> > Any idea why I have those issue?
> >
> > Upgraded from 4.5.2 to 4.9.3 ( shapeblue packages )
> >
> >
> > Regards,
> > Cristian
> >
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > Sent: Wednesday, October 11, 2017 6:10 PM
> > To: users@cloudstack.apache.org
> > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the
> > database
> >
> > that is a tricky one. You can destroy the vm from VmWare and set the
> > state to stopped in the DB. It should then start a new one.
> > check if r-4-VM/ROOT-4-000001.vmdk has gone from the datastore.
> >
> > the cause is probably some synchronisation issue. We've seen and
> > solved it before but are yet to find the root cause.
> >
> >
> > On Wed, Oct 11, 2017 at 4:23 PM, Ciobanu Cristian
> > <cristian.c@istream.today
> > >
> > wrote:
> >
> > > Hello,
> > >
> > >    Was my mistake, forgot to add the system template from cloudstack
> > > interface before starting the upgrade, now I have other issue, looks
> > > like after I have enabled one of the zone and click on upgrade
> > > virtual router I still see the state "starting" if I check the
> > > router from vSphere side is up.
> > >
> > >
> > >   This is what is see :
> > >
> > > tting
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Chose disk
> > > controller for vol ROOT -> osdefault, based on root disk controller
> > > settings at global configuration setting.
> > > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Folder r-4-VM exists on datastore catalina.out:INFO
> > > [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> > > ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1]
> > > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on
> > > datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Search file ROOT-4-000001-flat.vmdk on [datastore1]
> > > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001-flat.vmdk exists on
> > > datastore catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync
> > > [datastore1] ROOT-4-000001-flat.vmdk->[datastore1]
> > > r-4-VM/ROOT-4-000001-flat.vmdk catalina.out:INFO
> > > [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> > > ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Search file ROOT-4-000001-delta.vmdk on
> > > [datastore1] catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001-delta.vmdk does not
> > > exist on datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1]
> > > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on
> > > datastore catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync
> > > [datastore1] ROOT-4-000001.vmdk->[datastore1]
> > > r-4-VM/ROOT-4-000001.vmdk catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1]
> > > r-4-VM catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) File [datastore1] r-4-VM/ROOT-4-000001.vmdk exists
> > > on datastore catalina.out:INFO  [c.c.u.n.n.p.NiciraNvpApiVersion]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) NSX API VERSION:
> > > NOT PRESENT
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC
> > > device based on NicTO: {"deviceId":0,"networkRateMbps":200,"
> > > defaultNic":true,"pxeDisable":true,"nicUuid":"fc405400-6839-
> > > 4eea-9b30-e5e14b375322","uuid":"41f42309-8eaa-425a-a4c5-
> > > c8b5a88f92a2","ip":"149.56.xxx.xx","netmask":"255.255.
> > > 255.224","gateway":"149.56.164.62","mac":"06:76:62:00:00:
> > > 4b","dns1":"172.20.0.3","dns2":"8.8.8.8","broadcastType":"
> > > Native","type":"Guest","broadcastUri":"vlan://untagged","
> > > isSecurityGroupEnabled":false,"name":"vSwitch1"}
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare
> > > network on vmwaresvs vSwitch1 with name prefix: cloud.guest
> > > catalina.out:INFO [c.c.h.v.m.HypervisorHostHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
> > > catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
> > > catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC
> > > device on network cloud.guest.untagged.200.1-vSwitch1
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC
> > > device based on NicTO: {"deviceId":1,"networkRateMbps":-1,"
> > > defaultNic":false,"pxeDisable":true,"nicUuid":"fe3f7db4-
> > > 9ff5-4b63-883b-e08bb6f7568b","uuid":"8ef0b547-f569-4987-
> > > ac80-9ebf72f44fdd","ip":"0.0.0.0","netmask":"0.0.0.0","
> > > gateway":"0.0.0.0","mac":"02:00:5c:9c:00:0a","
> > broadcastType":"LinkLocal","
> > > type":"Control","isSecurityGroupEnabled":false,"name":"vSwitch1"}
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare
> > > network on vmwaresvs vSwitch1 with name prefix: cloud.private
> > > catalina.out:WARN  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Unrecognized
> > > broadcast type in VmwareResource, type: LinkLocal. Use vlan info
> > > from
> > > labeling: untagged
> > > catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > > cloud.private.untagged.0.1-vSwitch1 is ready on vSwitch vSwitch1
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC
> > > device on network cloud.private.untagged.0.1-vSwitch1
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Configure VNC
> > > port for VM r-4-VM, port: 5939, host: ns53xxxx.ip-149-56-xx.net
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Found
> > > existing disk info from volume path: ROOT-4-000001 catalina.out:INFO
> > > [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to
> > > connect to 149.56.xxx.xx catalina.out:WARN
> > > [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-8baef180)
> > > (logid:c6f5e75e) Task (job-3247) has been pending for 129 seconds
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not
> > > connect to 149.56.xxx.xx due to java.net.ConnectException:
> > > Connection timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to
> > > connect to 149.56.xxx.xx catalina.out:WARN
> > > [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-3026cb48)
> > > (logid:9ea6ca9e) Task (job-3247) has been pending for 189 seconds
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not
> > > connect to 149.56.xxx.xx due to java.net.ConnectException:
> > > Connection timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to
> > > connect to 149.56.xxx.xx
> > >
> > >
> > > Any suggestion? Please.
> > >
> > > Regards,
> > > Cristian
> > >
> > > -----Original Message-----
> > > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > > Sent: Wednesday, October 11, 2017 3:41 PM
> > > To: users@cloudstack.apache.org
> > > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the
> > > database
> > >
> > > Christian,
> > >
> > > The chances are I have not done the exact same upgrade as you are
> > > attempting. the generic upgrade of the database (assuming the right
> > > template and other things) is quite straight forward. You will need
> > > to give us at least the stacktrace and any related information. from
> > > the log above it, if this is a reproducible error.
> > >
> > > On Wed, Oct 11, 2017 at 2:34 PM, Ciobanu Cristian
> > > <cristian.c@istream.today
> > > >
> > > wrote:
> > >
> > > > I have restored back to 4.5.2, but if I upgrade again using the
> > > > steps provide in documentation I will end with the same error..
> > > > did you tested this? With tha configuration (acs 4.5.2, vmware,
> > > > local
> > storage) ?
> > > >
> > > > Regards,
> > > > Cristian
> > > >
> > > > -----Original Message-----
> > > > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > > > Sent: Wednesday, October 11, 2017 3:29 PM
> > > > To: users@cloudstack.apache.org
> > > > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the
> > > > database
> > > >
> > > > Hi the management server on starting up should take care of the
> > upgrade.
> > > > if you have a clustered MS make sure to start only one. In your
> > > > case after the failure you should probably restore the latest
> > > > backup before
> > > continuing.
> > > >
> > > > On Wed, Oct 11, 2017 at 2:26 PM, Ciobanu Cristian
> > > > <cristian.c@istream.today
> > > > >
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > >
> > > > >
> > > > >    In order to upgrade cloudstack to 4.9 I have followed this
> > > > > doecumentation
> > > > > :
> > > > > http://docs.cloudstack.apache.org/projects/cloudstack-
> > > > > release-notes/en/4.9.2
> > > > > .0/upgrade/upgrade-4.5.html  and ended with this :
> > > > >
> > > > >
> > > > >
> > > > > management-server.log:2017-10-11 08:05:35,612 WARN
> > > > > [o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:)
> > > > > Exception encountered during context initialization - cancelling
> > > > > refresh
> > > > attempt:
> > > > > org.springframework.context.ApplicationContextException: Failed
> > > > > to start bean 'cloudStackLifeCycle'; nested exception is
> > > > > com.cloud.utils.exception.CloudRuntimeException: Unable to
> > > > > upgrade the database
> > > > >
> > > > >
> > > > >
> > > > > Is there a valid upgrade procedure or script for : Cloudstack
> > > > > 4.5.2 to CloudStack 4.9 ( VMware with Local Storage ) ?
> > > > >
> > > > >
> > > > >
> > > > > (cloudstack was installed from shapeblue repo)
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Thank you.
> > > > >
> > > > > Cristian
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Daan
> > > >
> > > >
> > >
> > >
> > > --
> > > Daan
> > >
> > >
> >
> >
> > --
> > Daan
> >
> >
>
>
> --
> Daan
>
>


-- 
Daan

Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Daan Hoogland <da...@gmail.com>.
sorry Cristian, I wasn't clear. Can you add a jira ticket at
issues.apache.org/jira?

@devs I have heard this before but am not sure if it was exactly the same.
Anybody else? Was this fixed in 4.10 or master yet?
tl;dr setting password doesn't work for Cristian due to the firewall on the
VR.

On Fri, Oct 13, 2017 at 1:44 PM, Ciobanu Cristian <cr...@istream.today>
wrote:

> Daan,
>
>     Can I track this issue somewhere?
>
>
> Thank you!
> Cristian
>
> -----Original Message-----
> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> Sent: Thursday, October 12, 2017 9:46 AM
> To: users@cloudstack.apache.org
> Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database
>
> Cristian,
>
> I am cross-posting this to dev@coudstack.a.o, sorry for the noise. I have
> heard this before but am not sure if it was exactly the same. It sounds
> like a bug to me. Can you enter one in jira?
> Anybody else? Was this fixed in 4.10 or master yet?
>
> On Wed, Oct 11, 2017 at 11:23 PM, Ciobanu Cristian
> <cristian.c@istream.today
> > wrote:
>
> > Hello,
> >
> >  After I continued the investigation I found that router is not
> > allowing the VM to fetch the password (port 8080) not sure what is
> > wrong with this version but in order to fix I have added this rule "-A
> > INPUT -p tcp -m tcp --dport 8080 -j ACCEPT"  on virtual router..
> >
> >   Any idea how to fix this, permanent?  In case reinstall the router
> > for sure I will have to do this again.
> >
> >
> > Thank you.
> > Cristian
> >
> > -----Original Message-----
> > From: Ciobanu Cristian [mailto:cristian.c@istream.today]
> > Sent: Wednesday, October 11, 2017 9:45 PM
> > To: 'users@cloudstack.apache.org' <us...@cloudstack.apache.org>
> > Subject: RE: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the
> > database
> >
> > I was able to update the virtual router only after I have restarted
> > the iptables-persistend form virtual router side, not sure what is
> > wrong with this router version but I don't get the right status if I
> > don't restart the iptables from virtual router, I have did multiple
> tests :
> >
> > Example :
> >  -  Reboot router
> > - getting this error : job-3247/job-3248, cmd: CheckSshCommand)
> > (logid:6b3de9fa) Trying to
> > > connect to 149.56.xxx.xx  ( I see this error in management log )
> > - restart iptables from router side
> > - works
> >
> >   In case I destroy/restart the router I have to repeat the same
> > thing, restart iptables...
> >
> >   I also tested a new deploy with a template used before on this
> > setup, looks like the password set for new deploy is not working...
> >
> >
> > Any idea why I have those issue?
> >
> > Upgraded from 4.5.2 to 4.9.3 ( shapeblue packages )
> >
> >
> > Regards,
> > Cristian
> >
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > Sent: Wednesday, October 11, 2017 6:10 PM
> > To: users@cloudstack.apache.org
> > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the
> > database
> >
> > that is a tricky one. You can destroy the vm from VmWare and set the
> > state to stopped in the DB. It should then start a new one.
> > check if r-4-VM/ROOT-4-000001.vmdk has gone from the datastore.
> >
> > the cause is probably some synchronisation issue. We've seen and
> > solved it before but are yet to find the root cause.
> >
> >
> > On Wed, Oct 11, 2017 at 4:23 PM, Ciobanu Cristian
> > <cristian.c@istream.today
> > >
> > wrote:
> >
> > > Hello,
> > >
> > >    Was my mistake, forgot to add the system template from cloudstack
> > > interface before starting the upgrade, now I have other issue, looks
> > > like after I have enabled one of the zone and click on upgrade
> > > virtual router I still see the state "starting" if I check the
> > > router from vSphere side is up.
> > >
> > >
> > >   This is what is see :
> > >
> > > tting
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Chose disk
> > > controller for vol ROOT -> osdefault, based on root disk controller
> > > settings at global configuration setting.
> > > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Folder r-4-VM exists on datastore catalina.out:INFO
> > > [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> > > ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1]
> > > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on
> > > datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Search file ROOT-4-000001-flat.vmdk on [datastore1]
> > > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001-flat.vmdk exists on
> > > datastore catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync
> > > [datastore1] ROOT-4-000001-flat.vmdk->[datastore1]
> > > r-4-VM/ROOT-4-000001-flat.vmdk catalina.out:INFO
> > > [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> > > ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Search file ROOT-4-000001-delta.vmdk on
> > > [datastore1] catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001-delta.vmdk does not
> > > exist on datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1]
> > > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on
> > > datastore catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync
> > > [datastore1] ROOT-4-000001.vmdk->[datastore1]
> > > r-4-VM/ROOT-4-000001.vmdk catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1]
> > > r-4-VM catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand)
> > > (logid:6b3de9fa) File [datastore1] r-4-VM/ROOT-4-000001.vmdk exists
> > > on datastore catalina.out:INFO  [c.c.u.n.n.p.NiciraNvpApiVersion]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) NSX API VERSION:
> > > NOT PRESENT
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC
> > > device based on NicTO: {"deviceId":0,"networkRateMbps":200,"
> > > defaultNic":true,"pxeDisable":true,"nicUuid":"fc405400-6839-
> > > 4eea-9b30-e5e14b375322","uuid":"41f42309-8eaa-425a-a4c5-
> > > c8b5a88f92a2","ip":"149.56.xxx.xx","netmask":"255.255.
> > > 255.224","gateway":"149.56.164.62","mac":"06:76:62:00:00:
> > > 4b","dns1":"172.20.0.3","dns2":"8.8.8.8","broadcastType":"
> > > Native","type":"Guest","broadcastUri":"vlan://untagged","
> > > isSecurityGroupEnabled":false,"name":"vSwitch1"}
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare
> > > network on vmwaresvs vSwitch1 with name prefix: cloud.guest
> > > catalina.out:INFO [c.c.h.v.m.HypervisorHostHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
> > > catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
> > > catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC
> > > device on network cloud.guest.untagged.200.1-vSwitch1
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC
> > > device based on NicTO: {"deviceId":1,"networkRateMbps":-1,"
> > > defaultNic":false,"pxeDisable":true,"nicUuid":"fe3f7db4-
> > > 9ff5-4b63-883b-e08bb6f7568b","uuid":"8ef0b547-f569-4987-
> > > ac80-9ebf72f44fdd","ip":"0.0.0.0","netmask":"0.0.0.0","
> > > gateway":"0.0.0.0","mac":"02:00:5c:9c:00:0a","
> > broadcastType":"LinkLocal","
> > > type":"Control","isSecurityGroupEnabled":false,"name":"vSwitch1"}
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare
> > > network on vmwaresvs vSwitch1 with name prefix: cloud.private
> > > catalina.out:WARN  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Unrecognized
> > > broadcast type in VmwareResource, type: LinkLocal. Use vlan info
> > > from
> > > labeling: untagged
> > > catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > > cloud.private.untagged.0.1-vSwitch1 is ready on vSwitch vSwitch1
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC
> > > device on network cloud.private.untagged.0.1-vSwitch1
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Configure VNC
> > > port for VM r-4-VM, port: 5939, host: ns53xxxx.ip-149-56-xx.net
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Found
> > > existing disk info from volume path: ROOT-4-000001 catalina.out:INFO
> > > [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to
> > > connect to 149.56.xxx.xx catalina.out:WARN
> > > [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-8baef180)
> > > (logid:c6f5e75e) Task (job-3247) has been pending for 129 seconds
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not
> > > connect to 149.56.xxx.xx due to java.net.ConnectException:
> > > Connection timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to
> > > connect to 149.56.xxx.xx catalina.out:WARN
> > > [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-3026cb48)
> > > (logid:9ea6ca9e) Task (job-3247) has been pending for 189 seconds
> > > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not
> > > connect to 149.56.xxx.xx due to java.net.ConnectException:
> > > Connection timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to
> > > connect to 149.56.xxx.xx
> > >
> > >
> > > Any suggestion? Please.
> > >
> > > Regards,
> > > Cristian
> > >
> > > -----Original Message-----
> > > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > > Sent: Wednesday, October 11, 2017 3:41 PM
> > > To: users@cloudstack.apache.org
> > > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the
> > > database
> > >
> > > Christian,
> > >
> > > The chances are I have not done the exact same upgrade as you are
> > > attempting. the generic upgrade of the database (assuming the right
> > > template and other things) is quite straight forward. You will need
> > > to give us at least the stacktrace and any related information. from
> > > the log above it, if this is a reproducible error.
> > >
> > > On Wed, Oct 11, 2017 at 2:34 PM, Ciobanu Cristian
> > > <cristian.c@istream.today
> > > >
> > > wrote:
> > >
> > > > I have restored back to 4.5.2, but if I upgrade again using the
> > > > steps provide in documentation I will end with the same error..
> > > > did you tested this? With tha configuration (acs 4.5.2, vmware,
> > > > local
> > storage) ?
> > > >
> > > > Regards,
> > > > Cristian
> > > >
> > > > -----Original Message-----
> > > > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > > > Sent: Wednesday, October 11, 2017 3:29 PM
> > > > To: users@cloudstack.apache.org
> > > > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the
> > > > database
> > > >
> > > > Hi the management server on starting up should take care of the
> > upgrade.
> > > > if you have a clustered MS make sure to start only one. In your
> > > > case after the failure you should probably restore the latest
> > > > backup before
> > > continuing.
> > > >
> > > > On Wed, Oct 11, 2017 at 2:26 PM, Ciobanu Cristian
> > > > <cristian.c@istream.today
> > > > >
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > >
> > > > >
> > > > >    In order to upgrade cloudstack to 4.9 I have followed this
> > > > > doecumentation
> > > > > :
> > > > > http://docs.cloudstack.apache.org/projects/cloudstack-
> > > > > release-notes/en/4.9.2
> > > > > .0/upgrade/upgrade-4.5.html  and ended with this :
> > > > >
> > > > >
> > > > >
> > > > > management-server.log:2017-10-11 08:05:35,612 WARN
> > > > > [o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:)
> > > > > Exception encountered during context initialization - cancelling
> > > > > refresh
> > > > attempt:
> > > > > org.springframework.context.ApplicationContextException: Failed
> > > > > to start bean 'cloudStackLifeCycle'; nested exception is
> > > > > com.cloud.utils.exception.CloudRuntimeException: Unable to
> > > > > upgrade the database
> > > > >
> > > > >
> > > > >
> > > > > Is there a valid upgrade procedure or script for : Cloudstack
> > > > > 4.5.2 to CloudStack 4.9 ( VMware with Local Storage ) ?
> > > > >
> > > > >
> > > > >
> > > > > (cloudstack was installed from shapeblue repo)
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Thank you.
> > > > >
> > > > > Cristian
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Daan
> > > >
> > > >
> > >
> > >
> > > --
> > > Daan
> > >
> > >
> >
> >
> > --
> > Daan
> >
> >
>
>
> --
> Daan
>
>


-- 
Daan

RE: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Ciobanu Cristian <cr...@istream.today>.
Daan,

    Can I track this issue somewhere?


Thank you!
Cristian

-----Original Message-----
From: Daan Hoogland [mailto:daan.hoogland@gmail.com] 
Sent: Thursday, October 12, 2017 9:46 AM
To: users@cloudstack.apache.org
Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Cristian,

I am cross-posting this to dev@coudstack.a.o, sorry for the noise. I have heard this before but am not sure if it was exactly the same. It sounds like a bug to me. Can you enter one in jira?
Anybody else? Was this fixed in 4.10 or master yet?

On Wed, Oct 11, 2017 at 11:23 PM, Ciobanu Cristian <cristian.c@istream.today
> wrote:

> Hello,
>
>  After I continued the investigation I found that router is not 
> allowing the VM to fetch the password (port 8080) not sure what is 
> wrong with this version but in order to fix I have added this rule "-A 
> INPUT -p tcp -m tcp --dport 8080 -j ACCEPT"  on virtual router..
>
>   Any idea how to fix this, permanent?  In case reinstall the router 
> for sure I will have to do this again.
>
>
> Thank you.
> Cristian
>
> -----Original Message-----
> From: Ciobanu Cristian [mailto:cristian.c@istream.today]
> Sent: Wednesday, October 11, 2017 9:45 PM
> To: 'users@cloudstack.apache.org' <us...@cloudstack.apache.org>
> Subject: RE: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the 
> database
>
> I was able to update the virtual router only after I have restarted 
> the iptables-persistend form virtual router side, not sure what is 
> wrong with this router version but I don't get the right status if I 
> don't restart the iptables from virtual router, I have did multiple tests :
>
> Example :
>  -  Reboot router
> - getting this error : job-3247/job-3248, cmd: CheckSshCommand)
> (logid:6b3de9fa) Trying to
> > connect to 149.56.xxx.xx  ( I see this error in management log )
> - restart iptables from router side
> - works
>
>   In case I destroy/restart the router I have to repeat the same 
> thing, restart iptables...
>
>   I also tested a new deploy with a template used before on this 
> setup, looks like the password set for new deploy is not working...
>
>
> Any idea why I have those issue?
>
> Upgraded from 4.5.2 to 4.9.3 ( shapeblue packages )
>
>
> Regards,
> Cristian
>
>
> -----Original Message-----
> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> Sent: Wednesday, October 11, 2017 6:10 PM
> To: users@cloudstack.apache.org
> Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the 
> database
>
> that is a tricky one. You can destroy the vm from VmWare and set the 
> state to stopped in the DB. It should then start a new one.
> check if r-4-VM/ROOT-4-000001.vmdk has gone from the datastore.
>
> the cause is probably some synchronisation issue. We've seen and 
> solved it before but are yet to find the root cause.
>
>
> On Wed, Oct 11, 2017 at 4:23 PM, Ciobanu Cristian 
> <cristian.c@istream.today
> >
> wrote:
>
> > Hello,
> >
> >    Was my mistake, forgot to add the system template from cloudstack 
> > interface before starting the upgrade, now I have other issue, looks 
> > like after I have enabled one of the zone and click on upgrade 
> > virtual router I still see the state "starting" if I check the 
> > router from vSphere side is up.
> >
> >
> >   This is what is see :
> >
> > tting
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Chose disk 
> > controller for vol ROOT -> osdefault, based on root disk controller 
> > settings at global configuration setting.
> > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) Folder r-4-VM exists on datastore catalina.out:INFO 
> > [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 
> > ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] 
> > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on 
> > datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) Search file ROOT-4-000001-flat.vmdk on [datastore1] 
> > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) File [datastore1] ROOT-4-000001-flat.vmdk exists on 
> > datastore catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync 
> > [datastore1] ROOT-4-000001-flat.vmdk->[datastore1]
> > r-4-VM/ROOT-4-000001-flat.vmdk catalina.out:INFO 
> > [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 
> > ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) Search file ROOT-4-000001-delta.vmdk on 
> > [datastore1] catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) File [datastore1] ROOT-4-000001-delta.vmdk does not 
> > exist on datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] 
> > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on 
> > datastore catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync 
> > [datastore1] ROOT-4-000001.vmdk->[datastore1] 
> > r-4-VM/ROOT-4-000001.vmdk catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] 
> > r-4-VM catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) File [datastore1] r-4-VM/ROOT-4-000001.vmdk exists 
> > on datastore catalina.out:INFO  [c.c.u.n.n.p.NiciraNvpApiVersion]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) NSX API VERSION:
> > NOT PRESENT
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC 
> > device based on NicTO: {"deviceId":0,"networkRateMbps":200,"
> > defaultNic":true,"pxeDisable":true,"nicUuid":"fc405400-6839-
> > 4eea-9b30-e5e14b375322","uuid":"41f42309-8eaa-425a-a4c5-
> > c8b5a88f92a2","ip":"149.56.xxx.xx","netmask":"255.255.
> > 255.224","gateway":"149.56.164.62","mac":"06:76:62:00:00:
> > 4b","dns1":"172.20.0.3","dns2":"8.8.8.8","broadcastType":"
> > Native","type":"Guest","broadcastUri":"vlan://untagged","
> > isSecurityGroupEnabled":false,"name":"vSwitch1"}
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare 
> > network on vmwaresvs vSwitch1 with name prefix: cloud.guest 
> > catalina.out:INFO [c.c.h.v.m.HypervisorHostHelper]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 
> > catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 
> > catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC 
> > device on network cloud.guest.untagged.200.1-vSwitch1
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC 
> > device based on NicTO: {"deviceId":1,"networkRateMbps":-1,"
> > defaultNic":false,"pxeDisable":true,"nicUuid":"fe3f7db4-
> > 9ff5-4b63-883b-e08bb6f7568b","uuid":"8ef0b547-f569-4987-
> > ac80-9ebf72f44fdd","ip":"0.0.0.0","netmask":"0.0.0.0","
> > gateway":"0.0.0.0","mac":"02:00:5c:9c:00:0a","
> broadcastType":"LinkLocal","
> > type":"Control","isSecurityGroupEnabled":false,"name":"vSwitch1"}
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare 
> > network on vmwaresvs vSwitch1 with name prefix: cloud.private 
> > catalina.out:WARN  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Unrecognized 
> > broadcast type in VmwareResource, type: LinkLocal. Use vlan info 
> > from
> > labeling: untagged
> > catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > cloud.private.untagged.0.1-vSwitch1 is ready on vSwitch vSwitch1 
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC 
> > device on network cloud.private.untagged.0.1-vSwitch1
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Configure VNC 
> > port for VM r-4-VM, port: 5939, host: ns53xxxx.ip-149-56-xx.net 
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Found 
> > existing disk info from volume path: ROOT-4-000001 catalina.out:INFO 
> > [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to 
> > connect to 149.56.xxx.xx catalina.out:WARN 
> > [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-8baef180)
> > (logid:c6f5e75e) Task (job-3247) has been pending for 129 seconds 
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not 
> > connect to 149.56.xxx.xx due to java.net.ConnectException: 
> > Connection timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to 
> > connect to 149.56.xxx.xx catalina.out:WARN 
> > [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-3026cb48)
> > (logid:9ea6ca9e) Task (job-3247) has been pending for 189 seconds 
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not 
> > connect to 149.56.xxx.xx due to java.net.ConnectException: 
> > Connection timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to 
> > connect to 149.56.xxx.xx
> >
> >
> > Any suggestion? Please.
> >
> > Regards,
> > Cristian
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > Sent: Wednesday, October 11, 2017 3:41 PM
> > To: users@cloudstack.apache.org
> > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the 
> > database
> >
> > Christian,
> >
> > The chances are I have not done the exact same upgrade as you are 
> > attempting. the generic upgrade of the database (assuming the right 
> > template and other things) is quite straight forward. You will need 
> > to give us at least the stacktrace and any related information. from 
> > the log above it, if this is a reproducible error.
> >
> > On Wed, Oct 11, 2017 at 2:34 PM, Ciobanu Cristian 
> > <cristian.c@istream.today
> > >
> > wrote:
> >
> > > I have restored back to 4.5.2, but if I upgrade again using the 
> > > steps provide in documentation I will end with the same error..  
> > > did you tested this? With tha configuration (acs 4.5.2, vmware, 
> > > local
> storage) ?
> > >
> > > Regards,
> > > Cristian
> > >
> > > -----Original Message-----
> > > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > > Sent: Wednesday, October 11, 2017 3:29 PM
> > > To: users@cloudstack.apache.org
> > > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the 
> > > database
> > >
> > > Hi the management server on starting up should take care of the
> upgrade.
> > > if you have a clustered MS make sure to start only one. In your 
> > > case after the failure you should probably restore the latest 
> > > backup before
> > continuing.
> > >
> > > On Wed, Oct 11, 2017 at 2:26 PM, Ciobanu Cristian 
> > > <cristian.c@istream.today
> > > >
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > >
> > > >
> > > >    In order to upgrade cloudstack to 4.9 I have followed this 
> > > > doecumentation
> > > > :
> > > > http://docs.cloudstack.apache.org/projects/cloudstack-
> > > > release-notes/en/4.9.2
> > > > .0/upgrade/upgrade-4.5.html  and ended with this :
> > > >
> > > >
> > > >
> > > > management-server.log:2017-10-11 08:05:35,612 WARN 
> > > > [o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:) 
> > > > Exception encountered during context initialization - cancelling 
> > > > refresh
> > > attempt:
> > > > org.springframework.context.ApplicationContextException: Failed 
> > > > to start bean 'cloudStackLifeCycle'; nested exception is
> > > > com.cloud.utils.exception.CloudRuntimeException: Unable to 
> > > > upgrade the database
> > > >
> > > >
> > > >
> > > > Is there a valid upgrade procedure or script for : Cloudstack
> > > > 4.5.2 to CloudStack 4.9 ( VMware with Local Storage ) ?
> > > >
> > > >
> > > >
> > > > (cloudstack was installed from shapeblue repo)
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Thank you.
> > > >
> > > > Cristian
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Daan
> > >
> > >
> >
> >
> > --
> > Daan
> >
> >
>
>
> --
> Daan
>
>


--
Daan


Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Daan Hoogland <da...@gmail.com>.
Cristian,

I am cross-posting this to dev@coudstack.a.o, sorry for the noise. I have
heard this before but am not sure if it was exactly the same. It sounds
like a bug to me. Can you enter one in jira?
Anybody else? Was this fixed in 4.10 or master yet?

On Wed, Oct 11, 2017 at 11:23 PM, Ciobanu Cristian <cristian.c@istream.today
> wrote:

> Hello,
>
>  After I continued the investigation I found that router is not allowing
> the VM to fetch the password (port 8080) not sure what is wrong with this
> version but in order to fix I have added this rule "-A INPUT -p tcp -m tcp
> --dport 8080 -j ACCEPT"  on virtual router..
>
>   Any idea how to fix this, permanent?  In case reinstall the router for
> sure I will have to do this again.
>
>
> Thank you.
> Cristian
>
> -----Original Message-----
> From: Ciobanu Cristian [mailto:cristian.c@istream.today]
> Sent: Wednesday, October 11, 2017 9:45 PM
> To: 'users@cloudstack.apache.org' <us...@cloudstack.apache.org>
> Subject: RE: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database
>
> I was able to update the virtual router only after I have restarted the
> iptables-persistend form virtual router side, not sure what is wrong with
> this router version but I don't get the right status if I don't restart the
> iptables from virtual router, I have did multiple tests :
>
> Example :
>  -  Reboot router
> - getting this error : job-3247/job-3248, cmd: CheckSshCommand)
> (logid:6b3de9fa) Trying to
> > connect to 149.56.xxx.xx  ( I see this error in management log )
> - restart iptables from router side
> - works
>
>   In case I destroy/restart the router I have to repeat the same thing,
> restart iptables...
>
>   I also tested a new deploy with a template used before on this setup,
> looks like the password set for new deploy is not working...
>
>
> Any idea why I have those issue?
>
> Upgraded from 4.5.2 to 4.9.3 ( shapeblue packages )
>
>
> Regards,
> Cristian
>
>
> -----Original Message-----
> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> Sent: Wednesday, October 11, 2017 6:10 PM
> To: users@cloudstack.apache.org
> Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database
>
> that is a tricky one. You can destroy the vm from VmWare and set the state
> to stopped in the DB. It should then start a new one.
> check if r-4-VM/ROOT-4-000001.vmdk has gone from the datastore.
>
> the cause is probably some synchronisation issue. We've seen and solved it
> before but are yet to find the root cause.
>
>
> On Wed, Oct 11, 2017 at 4:23 PM, Ciobanu Cristian <cristian.c@istream.today
> >
> wrote:
>
> > Hello,
> >
> >    Was my mistake, forgot to add the system template from cloudstack
> > interface before starting the upgrade, now I have other issue, looks
> > like after I have enabled one of the zone and click on upgrade virtual
> > router I still see the state "starting" if I check the router from
> > vSphere side is up.
> >
> >
> >   This is what is see :
> >
> > tting
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Chose disk
> > controller for vol ROOT -> osdefault, based on root disk controller
> > settings at global configuration setting.
> > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) Folder r-4-VM exists on datastore catalina.out:INFO
> > [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> > ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1]
> > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on
> > datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) Search file ROOT-4-000001-flat.vmdk on [datastore1]
> > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) File [datastore1] ROOT-4-000001-flat.vmdk exists on
> > datastore catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync
> > [datastore1] ROOT-4-000001-flat.vmdk->[datastore1]
> > r-4-VM/ROOT-4-000001-flat.vmdk catalina.out:INFO
> > [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> > ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) Search file ROOT-4-000001-delta.vmdk on [datastore1]
> > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) File [datastore1] ROOT-4-000001-delta.vmdk does not
> > exist on datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1]
> > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on
> > datastore catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync
> > [datastore1] ROOT-4-000001.vmdk->[datastore1]
> > r-4-VM/ROOT-4-000001.vmdk catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] r-4-VM
> > catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand)
> > (logid:6b3de9fa) File [datastore1] r-4-VM/ROOT-4-000001.vmdk exists on
> > datastore catalina.out:INFO  [c.c.u.n.n.p.NiciraNvpApiVersion]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) NSX API VERSION:
> > NOT PRESENT
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC
> > device based on NicTO: {"deviceId":0,"networkRateMbps":200,"
> > defaultNic":true,"pxeDisable":true,"nicUuid":"fc405400-6839-
> > 4eea-9b30-e5e14b375322","uuid":"41f42309-8eaa-425a-a4c5-
> > c8b5a88f92a2","ip":"149.56.xxx.xx","netmask":"255.255.
> > 255.224","gateway":"149.56.164.62","mac":"06:76:62:00:00:
> > 4b","dns1":"172.20.0.3","dns2":"8.8.8.8","broadcastType":"
> > Native","type":"Guest","broadcastUri":"vlan://untagged","
> > isSecurityGroupEnabled":false,"name":"vSwitch1"}
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare network
> > on vmwaresvs vSwitch1 with name prefix: cloud.guest catalina.out:INFO
> > [c.c.h.v.m.HypervisorHostHelper]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
> > catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
> > catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC
> > device on network cloud.guest.untagged.200.1-vSwitch1
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC
> > device based on NicTO: {"deviceId":1,"networkRateMbps":-1,"
> > defaultNic":false,"pxeDisable":true,"nicUuid":"fe3f7db4-
> > 9ff5-4b63-883b-e08bb6f7568b","uuid":"8ef0b547-f569-4987-
> > ac80-9ebf72f44fdd","ip":"0.0.0.0","netmask":"0.0.0.0","
> > gateway":"0.0.0.0","mac":"02:00:5c:9c:00:0a","
> broadcastType":"LinkLocal","
> > type":"Control","isSecurityGroupEnabled":false,"name":"vSwitch1"}
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare network
> > on vmwaresvs vSwitch1 with name prefix: cloud.private
> > catalina.out:WARN  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Unrecognized
> > broadcast type in VmwareResource, type: LinkLocal. Use vlan info from
> > labeling: untagged
> > catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> > cloud.private.untagged.0.1-vSwitch1 is ready on vSwitch vSwitch1
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC
> > device on network cloud.private.untagged.0.1-vSwitch1
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Configure VNC
> > port for VM r-4-VM, port: 5939, host: ns53xxxx.ip-149-56-xx.net
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Found existing
> > disk info from volume path: ROOT-4-000001 catalina.out:INFO
> > [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to
> > connect to 149.56.xxx.xx catalina.out:WARN
> > [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-8baef180)
> > (logid:c6f5e75e) Task (job-3247) has been pending for 129 seconds
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not
> > connect to 149.56.xxx.xx due to java.net.ConnectException: Connection
> > timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to
> > connect to 149.56.xxx.xx catalina.out:WARN
> > [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-3026cb48)
> > (logid:9ea6ca9e) Task (job-3247) has been pending for 189 seconds
> > catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not
> > connect to 149.56.xxx.xx due to java.net.ConnectException: Connection
> > timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> > (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> > job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to
> > connect to 149.56.xxx.xx
> >
> >
> > Any suggestion? Please.
> >
> > Regards,
> > Cristian
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > Sent: Wednesday, October 11, 2017 3:41 PM
> > To: users@cloudstack.apache.org
> > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the
> > database
> >
> > Christian,
> >
> > The chances are I have not done the exact same upgrade as you are
> > attempting. the generic upgrade of the database (assuming the right
> > template and other things) is quite straight forward. You will need to
> > give us at least the stacktrace and any related information. from the
> > log above it, if this is a reproducible error.
> >
> > On Wed, Oct 11, 2017 at 2:34 PM, Ciobanu Cristian
> > <cristian.c@istream.today
> > >
> > wrote:
> >
> > > I have restored back to 4.5.2, but if I upgrade again using the
> > > steps provide in documentation I will end with the same error..  did
> > > you tested this? With tha configuration (acs 4.5.2, vmware, local
> storage) ?
> > >
> > > Regards,
> > > Cristian
> > >
> > > -----Original Message-----
> > > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > > Sent: Wednesday, October 11, 2017 3:29 PM
> > > To: users@cloudstack.apache.org
> > > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the
> > > database
> > >
> > > Hi the management server on starting up should take care of the
> upgrade.
> > > if you have a clustered MS make sure to start only one. In your case
> > > after the failure you should probably restore the latest backup
> > > before
> > continuing.
> > >
> > > On Wed, Oct 11, 2017 at 2:26 PM, Ciobanu Cristian
> > > <cristian.c@istream.today
> > > >
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > >
> > > >
> > > >    In order to upgrade cloudstack to 4.9 I have followed this
> > > > doecumentation
> > > > :
> > > > http://docs.cloudstack.apache.org/projects/cloudstack-
> > > > release-notes/en/4.9.2
> > > > .0/upgrade/upgrade-4.5.html  and ended with this :
> > > >
> > > >
> > > >
> > > > management-server.log:2017-10-11 08:05:35,612 WARN
> > > > [o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:)
> > > > Exception encountered during context initialization - cancelling
> > > > refresh
> > > attempt:
> > > > org.springframework.context.ApplicationContextException: Failed to
> > > > start bean 'cloudStackLifeCycle'; nested exception is
> > > > com.cloud.utils.exception.CloudRuntimeException: Unable to upgrade
> > > > the database
> > > >
> > > >
> > > >
> > > > Is there a valid upgrade procedure or script for : Cloudstack
> > > > 4.5.2 to CloudStack 4.9 ( VMware with Local Storage ) ?
> > > >
> > > >
> > > >
> > > > (cloudstack was installed from shapeblue repo)
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Thank you.
> > > >
> > > > Cristian
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Daan
> > >
> > >
> >
> >
> > --
> > Daan
> >
> >
>
>
> --
> Daan
>
>


-- 
Daan

RE: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Ciobanu Cristian <cr...@istream.today>.
Hello,

 After I continued the investigation I found that router is not allowing the VM to fetch the password (port 8080) not sure what is wrong with this version but in order to fix I have added this rule "-A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT"  on virtual router..

  Any idea how to fix this, permanent?  In case reinstall the router for sure I will have to do this again.


Thank you.
Cristian

-----Original Message-----
From: Ciobanu Cristian [mailto:cristian.c@istream.today] 
Sent: Wednesday, October 11, 2017 9:45 PM
To: 'users@cloudstack.apache.org' <us...@cloudstack.apache.org>
Subject: RE: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

I was able to update the virtual router only after I have restarted the iptables-persistend form virtual router side, not sure what is wrong with this router version but I don't get the right status if I don't restart the iptables from virtual router, I have did multiple tests :

Example :
 -  Reboot router
- getting this error : job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to 
> connect to 149.56.xxx.xx  ( I see this error in management log )
- restart iptables from router side
- works

  In case I destroy/restart the router I have to repeat the same thing, restart iptables...

  I also tested a new deploy with a template used before on this setup, looks like the password set for new deploy is not working...


Any idea why I have those issue?

Upgraded from 4.5.2 to 4.9.3 ( shapeblue packages )


Regards,
Cristian


-----Original Message-----
From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
Sent: Wednesday, October 11, 2017 6:10 PM
To: users@cloudstack.apache.org
Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

that is a tricky one. You can destroy the vm from VmWare and set the state to stopped in the DB. It should then start a new one.
check if r-4-VM/ROOT-4-000001.vmdk has gone from the datastore.

the cause is probably some synchronisation issue. We've seen and solved it before but are yet to find the root cause.


On Wed, Oct 11, 2017 at 4:23 PM, Ciobanu Cristian <cr...@istream.today>
wrote:

> Hello,
>
>    Was my mistake, forgot to add the system template from cloudstack 
> interface before starting the upgrade, now I have other issue, looks 
> like after I have enabled one of the zone and click on upgrade virtual 
> router I still see the state "starting" if I check the router from 
> vSphere side is up.
>
>
>   This is what is see :
>
> tting
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Chose disk 
> controller for vol ROOT -> osdefault, based on root disk controller 
> settings at global configuration setting.
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Folder r-4-VM exists on datastore catalina.out:INFO 
> [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] 
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on 
> datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001-flat.vmdk on [datastore1] 
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001-flat.vmdk exists on 
> datastore catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync 
> [datastore1] ROOT-4-000001-flat.vmdk->[datastore1]
> r-4-VM/ROOT-4-000001-flat.vmdk catalina.out:INFO 
> [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001-delta.vmdk on [datastore1] 
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001-delta.vmdk does not 
> exist on datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] 
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on 
> datastore catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync 
> [datastore1] ROOT-4-000001.vmdk->[datastore1] 
> r-4-VM/ROOT-4-000001.vmdk catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] r-4-VM 
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] r-4-VM/ROOT-4-000001.vmdk exists on 
> datastore catalina.out:INFO  [c.c.u.n.n.p.NiciraNvpApiVersion]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) NSX API VERSION:
> NOT PRESENT
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC 
> device based on NicTO: {"deviceId":0,"networkRateMbps":200,"
> defaultNic":true,"pxeDisable":true,"nicUuid":"fc405400-6839-
> 4eea-9b30-e5e14b375322","uuid":"41f42309-8eaa-425a-a4c5-
> c8b5a88f92a2","ip":"149.56.xxx.xx","netmask":"255.255.
> 255.224","gateway":"149.56.164.62","mac":"06:76:62:00:00:
> 4b","dns1":"172.20.0.3","dns2":"8.8.8.8","broadcastType":"
> Native","type":"Guest","broadcastUri":"vlan://untagged","
> isSecurityGroupEnabled":false,"name":"vSwitch1"}
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare network 
> on vmwaresvs vSwitch1 with name prefix: cloud.guest catalina.out:INFO 
> [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 
> catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 
> catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC 
> device on network cloud.guest.untagged.200.1-vSwitch1
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC 
> device based on NicTO: {"deviceId":1,"networkRateMbps":-1,"
> defaultNic":false,"pxeDisable":true,"nicUuid":"fe3f7db4-
> 9ff5-4b63-883b-e08bb6f7568b","uuid":"8ef0b547-f569-4987-
> ac80-9ebf72f44fdd","ip":"0.0.0.0","netmask":"0.0.0.0","
> gateway":"0.0.0.0","mac":"02:00:5c:9c:00:0a","broadcastType":"LinkLocal","
> type":"Control","isSecurityGroupEnabled":false,"name":"vSwitch1"}
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare network 
> on vmwaresvs vSwitch1 with name prefix: cloud.private 
> catalina.out:WARN  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Unrecognized 
> broadcast type in VmwareResource, type: LinkLocal. Use vlan info from
> labeling: untagged
> catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.private.untagged.0.1-vSwitch1 is ready on vSwitch vSwitch1 
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC 
> device on network cloud.private.untagged.0.1-vSwitch1
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Configure VNC 
> port for VM r-4-VM, port: 5939, host: ns53xxxx.ip-149-56-xx.net 
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Found existing 
> disk info from volume path: ROOT-4-000001 catalina.out:INFO 
> [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to 
> connect to 149.56.xxx.xx catalina.out:WARN 
> [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-8baef180)
> (logid:c6f5e75e) Task (job-3247) has been pending for 129 seconds 
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not 
> connect to 149.56.xxx.xx due to java.net.ConnectException: Connection 
> timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to 
> connect to 149.56.xxx.xx catalina.out:WARN 
> [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-3026cb48)
> (logid:9ea6ca9e) Task (job-3247) has been pending for 189 seconds 
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not 
> connect to 149.56.xxx.xx due to java.net.ConnectException: Connection 
> timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to 
> connect to 149.56.xxx.xx
>
>
> Any suggestion? Please.
>
> Regards,
> Cristian
>
> -----Original Message-----
> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> Sent: Wednesday, October 11, 2017 3:41 PM
> To: users@cloudstack.apache.org
> Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the 
> database
>
> Christian,
>
> The chances are I have not done the exact same upgrade as you are 
> attempting. the generic upgrade of the database (assuming the right 
> template and other things) is quite straight forward. You will need to 
> give us at least the stacktrace and any related information. from the 
> log above it, if this is a reproducible error.
>
> On Wed, Oct 11, 2017 at 2:34 PM, Ciobanu Cristian 
> <cristian.c@istream.today
> >
> wrote:
>
> > I have restored back to 4.5.2, but if I upgrade again using the 
> > steps provide in documentation I will end with the same error..  did 
> > you tested this? With tha configuration (acs 4.5.2, vmware, local storage) ?
> >
> > Regards,
> > Cristian
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > Sent: Wednesday, October 11, 2017 3:29 PM
> > To: users@cloudstack.apache.org
> > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the 
> > database
> >
> > Hi the management server on starting up should take care of the upgrade.
> > if you have a clustered MS make sure to start only one. In your case 
> > after the failure you should probably restore the latest backup 
> > before
> continuing.
> >
> > On Wed, Oct 11, 2017 at 2:26 PM, Ciobanu Cristian 
> > <cristian.c@istream.today
> > >
> > wrote:
> >
> > > Hello,
> > >
> > >
> > >
> > >    In order to upgrade cloudstack to 4.9 I have followed this 
> > > doecumentation
> > > :
> > > http://docs.cloudstack.apache.org/projects/cloudstack-
> > > release-notes/en/4.9.2
> > > .0/upgrade/upgrade-4.5.html  and ended with this :
> > >
> > >
> > >
> > > management-server.log:2017-10-11 08:05:35,612 WARN 
> > > [o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:) 
> > > Exception encountered during context initialization - cancelling 
> > > refresh
> > attempt:
> > > org.springframework.context.ApplicationContextException: Failed to 
> > > start bean 'cloudStackLifeCycle'; nested exception is
> > > com.cloud.utils.exception.CloudRuntimeException: Unable to upgrade 
> > > the database
> > >
> > >
> > >
> > > Is there a valid upgrade procedure or script for : Cloudstack
> > > 4.5.2 to CloudStack 4.9 ( VMware with Local Storage ) ?
> > >
> > >
> > >
> > > (cloudstack was installed from shapeblue repo)
> > >
> > >
> > >
> > >
> > >
> > > Thank you.
> > >
> > > Cristian
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Daan
> >
> >
>
>
> --
> Daan
>
>


--
Daan


RE: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Ciobanu Cristian <cr...@istream.today>.
Hello,

     I fixed by doing this, ssvm and virtual router started but not able to deploy vms because of this :

catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) Search file ROOT-241-000001.vmdk on [datastore1 (2)] r-241-VM
catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) File [datastore1 (2)] r-241-VM/ROOT-241-000001.vmdk exists on datastore
catalina.out:INFO  [c.c.u.n.n.p.NiciraNvpApiVersion] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) NSX API VERSION:  NOT PRESENT
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) Prepare NIC device based on NicTO: {"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":true,"nicUuid":"0a03c87b-f378-4459-93cd-724187d2f9fe","uuid":"41f42309-8eaa-425a-a4c5-c8b5a88f92a2","ip":"149.56.xx.xx","netmask":"255.255.255.224","gateway":"149.56.164.62","mac":"06:64:1c:00:00:4b","dns1":"172.20.0.3","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":false,"name":"vSwitch1"}
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) Prepare network on vmwaresvs vSwitch1 with name prefix: cloud.guest
catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) Network cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) Network cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) Network cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) Preparing NIC device on network cloud.guest.untagged.200.1-vSwitch1
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) Prepare NIC device based on NicTO: {"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"b0b3c108-5d4c-4db8-8a02-f49d0263e111","uuid":"8ef0b547-f569-4987-ac80-9ebf72f44fdd","ip":"0.0.0.0","netmask":"0.0.0.0","gateway":"0.0.0.0","mac":"02:00:21:e5:00:0d","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false,"name":"vSwitch1"}
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) Prepare network on vmwaresvs vSwitch1 with name prefix: cloud.private
catalina.out:WARN  [c.c.h.v.r.VmwareResource] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) Unrecognized broadcast type in VmwareResource, type: LinkLocal. Use vlan info from labeling: untagged
catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) Network cloud.private.untagged.0.1-vSwitch1 is ready on vSwitch vSwitch1
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) Preparing NIC device on network cloud.private.untagged.0.1-vSwitch1
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) Configure VNC port for VM r-241-VM, port: 5903, host: ns539xxx.ip-144-217-xx.net
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: StartCommand) (logid:2b54f0bf) Found existing disk info from volume path: ROOT-241-000001
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: CheckSshCommand) (logid:2b54f0bf) Trying to connect to 149.56.xx.xx
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: CheckSshCommand) (logid:2b54f0bf) Could not connect to 149.56.xx.xx due to java.net.ConnectException: Connection timed out
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-301:ctx-27221d86 ns539xxx.ip-144-217-xx.net, job-3272/job-3273, cmd: CheckSshCommand) (logid:2b54f0bf) Trying to connect to 149.56.xx.xx
catalina.out:WARN  [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-8903c55c) (logid:7ce4cc9b) Task (job-3272) has been pending for 99 seconds


Any idea what is wrong?

Thank you!
Cristian

-----Original Message-----
From: Daan Hoogland [mailto:daan.hoogland@gmail.com] 
Sent: Wednesday, October 11, 2017 6:10 PM
To: users@cloudstack.apache.org
Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

that is a tricky one. You can destroy the vm from VmWare and set the state to stopped in the DB. It should then start a new one.
check if r-4-VM/ROOT-4-000001.vmdk has gone from the datastore.

the cause is probably some synchronisation issue. We've seen and solved it before but are yet to find the root cause.


On Wed, Oct 11, 2017 at 4:23 PM, Ciobanu Cristian <cr...@istream.today>
wrote:

> Hello,
>
>    Was my mistake, forgot to add the system template from cloudstack 
> interface before starting the upgrade, now I have other issue, looks 
> like after I have enabled one of the zone and click on upgrade virtual 
> router I still see the state "starting" if I check the router from 
> vSphere side is up.
>
>
>   This is what is see :
>
> tting
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Chose disk 
> controller for vol ROOT -> osdefault, based on root disk controller 
> settings at global configuration setting.
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Folder r-4-VM exists on datastore catalina.out:INFO  
> [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] 
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on 
> datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001-flat.vmdk on [datastore1] 
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001-flat.vmdk exists on 
> datastore catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync 
> [datastore1] ROOT-4-000001-flat.vmdk->[datastore1] 
> r-4-VM/ROOT-4-000001-flat.vmdk catalina.out:INFO  
> [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001-delta.vmdk on [datastore1] 
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001-delta.vmdk does not 
> exist on datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] 
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on 
> datastore catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync 
> [datastore1] ROOT-4-000001.vmdk->[datastore1] 
> r-4-VM/ROOT-4-000001.vmdk catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] r-4-VM 
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] r-4-VM/ROOT-4-000001.vmdk exists on 
> datastore catalina.out:INFO  [c.c.u.n.n.p.NiciraNvpApiVersion]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) NSX API VERSION:
> NOT PRESENT
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC 
> device based on NicTO: {"deviceId":0,"networkRateMbps":200,"
> defaultNic":true,"pxeDisable":true,"nicUuid":"fc405400-6839-
> 4eea-9b30-e5e14b375322","uuid":"41f42309-8eaa-425a-a4c5-
> c8b5a88f92a2","ip":"149.56.xxx.xx","netmask":"255.255.
> 255.224","gateway":"149.56.164.62","mac":"06:76:62:00:00:
> 4b","dns1":"172.20.0.3","dns2":"8.8.8.8","broadcastType":"
> Native","type":"Guest","broadcastUri":"vlan://untagged","
> isSecurityGroupEnabled":false,"name":"vSwitch1"}
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare network 
> on vmwaresvs vSwitch1 with name prefix: cloud.guest catalina.out:INFO  
> [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 
> catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 
> catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC 
> device on network cloud.guest.untagged.200.1-vSwitch1
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC 
> device based on NicTO: {"deviceId":1,"networkRateMbps":-1,"
> defaultNic":false,"pxeDisable":true,"nicUuid":"fe3f7db4-
> 9ff5-4b63-883b-e08bb6f7568b","uuid":"8ef0b547-f569-4987-
> ac80-9ebf72f44fdd","ip":"0.0.0.0","netmask":"0.0.0.0","
> gateway":"0.0.0.0","mac":"02:00:5c:9c:00:0a","broadcastType":"LinkLocal","
> type":"Control","isSecurityGroupEnabled":false,"name":"vSwitch1"}
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare network 
> on vmwaresvs vSwitch1 with name prefix: cloud.private 
> catalina.out:WARN  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Unrecognized 
> broadcast type in VmwareResource, type: LinkLocal. Use vlan info from
> labeling: untagged
> catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.private.untagged.0.1-vSwitch1 is ready on vSwitch vSwitch1 
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC 
> device on network cloud.private.untagged.0.1-vSwitch1
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Configure VNC 
> port for VM r-4-VM, port: 5939, host: ns53xxxx.ip-149-56-xx.net 
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Found existing 
> disk info from volume path: ROOT-4-000001 catalina.out:INFO  
> [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to 
> connect to 149.56.xxx.xx catalina.out:WARN  
> [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-8baef180)
> (logid:c6f5e75e) Task (job-3247) has been pending for 129 seconds 
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not 
> connect to 149.56.xxx.xx due to java.net.ConnectException: Connection 
> timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to 
> connect to 149.56.xxx.xx catalina.out:WARN  
> [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-3026cb48)
> (logid:9ea6ca9e) Task (job-3247) has been pending for 189 seconds 
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not 
> connect to 149.56.xxx.xx due to java.net.ConnectException: Connection 
> timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to 
> connect to 149.56.xxx.xx
>
>
> Any suggestion? Please.
>
> Regards,
> Cristian
>
> -----Original Message-----
> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> Sent: Wednesday, October 11, 2017 3:41 PM
> To: users@cloudstack.apache.org
> Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the 
> database
>
> Christian,
>
> The chances are I have not done the exact same upgrade as you are 
> attempting. the generic upgrade of the database (assuming the right 
> template and other things) is quite straight forward. You will need to 
> give us at least the stacktrace and any related information. from the 
> log above it, if this is a reproducible error.
>
> On Wed, Oct 11, 2017 at 2:34 PM, Ciobanu Cristian 
> <cristian.c@istream.today
> >
> wrote:
>
> > I have restored back to 4.5.2, but if I upgrade again using the 
> > steps provide in documentation I will end with the same error..  did 
> > you tested this? With tha configuration (acs 4.5.2, vmware, local storage) ?
> >
> > Regards,
> > Cristian
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > Sent: Wednesday, October 11, 2017 3:29 PM
> > To: users@cloudstack.apache.org
> > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the 
> > database
> >
> > Hi the management server on starting up should take care of the upgrade.
> > if you have a clustered MS make sure to start only one. In your case 
> > after the failure you should probably restore the latest backup 
> > before
> continuing.
> >
> > On Wed, Oct 11, 2017 at 2:26 PM, Ciobanu Cristian 
> > <cristian.c@istream.today
> > >
> > wrote:
> >
> > > Hello,
> > >
> > >
> > >
> > >    In order to upgrade cloudstack to 4.9 I have followed this 
> > > doecumentation
> > > :
> > > http://docs.cloudstack.apache.org/projects/cloudstack-
> > > release-notes/en/4.9.2
> > > .0/upgrade/upgrade-4.5.html  and ended with this :
> > >
> > >
> > >
> > > management-server.log:2017-10-11 08:05:35,612 WARN 
> > > [o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:) 
> > > Exception encountered during context initialization - cancelling 
> > > refresh
> > attempt:
> > > org.springframework.context.ApplicationContextException: Failed to 
> > > start bean 'cloudStackLifeCycle'; nested exception is
> > > com.cloud.utils.exception.CloudRuntimeException: Unable to upgrade 
> > > the database
> > >
> > >
> > >
> > > Is there a valid upgrade procedure or script for : Cloudstack 
> > > 4.5.2 to CloudStack 4.9 ( VMware with Local Storage ) ?
> > >
> > >
> > >
> > > (cloudstack was installed from shapeblue repo)
> > >
> > >
> > >
> > >
> > >
> > > Thank you.
> > >
> > > Cristian
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Daan
> >
> >
>
>
> --
> Daan
>
>


--
Daan


RE: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Ciobanu Cristian <cr...@istream.today>.
I was able to update the virtual router only after I have restarted the iptables-persistend form virtual router side, not sure what is wrong with this router version but I don't get the right status if I don't restart the iptables from virtual router, I have did multiple tests :

Example :
 -  Reboot router 
- getting this error : job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to 
> connect to 149.56.xxx.xx  ( I see this error in management log )
- restart iptables from router side
- works

  In case I destroy/restart the router I have to repeat the same thing, restart iptables...

  I also tested a new deploy with a template used before on this setup, looks like the password set for new deploy is not working...


Any idea why I have those issue?

Upgraded from 4.5.2 to 4.9.3 ( shapeblue packages )


Regards,
Cristian


-----Original Message-----
From: Daan Hoogland [mailto:daan.hoogland@gmail.com] 
Sent: Wednesday, October 11, 2017 6:10 PM
To: users@cloudstack.apache.org
Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

that is a tricky one. You can destroy the vm from VmWare and set the state to stopped in the DB. It should then start a new one.
check if r-4-VM/ROOT-4-000001.vmdk has gone from the datastore.

the cause is probably some synchronisation issue. We've seen and solved it before but are yet to find the root cause.


On Wed, Oct 11, 2017 at 4:23 PM, Ciobanu Cristian <cr...@istream.today>
wrote:

> Hello,
>
>    Was my mistake, forgot to add the system template from cloudstack 
> interface before starting the upgrade, now I have other issue, looks 
> like after I have enabled one of the zone and click on upgrade virtual 
> router I still see the state "starting" if I check the router from 
> vSphere side is up.
>
>
>   This is what is see :
>
> tting
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Chose disk 
> controller for vol ROOT -> osdefault, based on root disk controller 
> settings at global configuration setting.
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Folder r-4-VM exists on datastore catalina.out:INFO  
> [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] 
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on 
> datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001-flat.vmdk on [datastore1] 
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001-flat.vmdk exists on 
> datastore catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync 
> [datastore1] ROOT-4-000001-flat.vmdk->[datastore1] 
> r-4-VM/ROOT-4-000001-flat.vmdk catalina.out:INFO  
> [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001-delta.vmdk on [datastore1] 
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001-delta.vmdk does not 
> exist on datastore catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] 
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on 
> datastore catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync 
> [datastore1] ROOT-4-000001.vmdk->[datastore1] 
> r-4-VM/ROOT-4-000001.vmdk catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] r-4-VM 
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] 
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] r-4-VM/ROOT-4-000001.vmdk exists on 
> datastore catalina.out:INFO  [c.c.u.n.n.p.NiciraNvpApiVersion]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) NSX API VERSION:
> NOT PRESENT
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC 
> device based on NicTO: {"deviceId":0,"networkRateMbps":200,"
> defaultNic":true,"pxeDisable":true,"nicUuid":"fc405400-6839-
> 4eea-9b30-e5e14b375322","uuid":"41f42309-8eaa-425a-a4c5-
> c8b5a88f92a2","ip":"149.56.xxx.xx","netmask":"255.255.
> 255.224","gateway":"149.56.164.62","mac":"06:76:62:00:00:
> 4b","dns1":"172.20.0.3","dns2":"8.8.8.8","broadcastType":"
> Native","type":"Guest","broadcastUri":"vlan://untagged","
> isSecurityGroupEnabled":false,"name":"vSwitch1"}
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare network 
> on vmwaresvs vSwitch1 with name prefix: cloud.guest catalina.out:INFO  
> [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 
> catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 
> catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1 
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC 
> device on network cloud.guest.untagged.200.1-vSwitch1
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC 
> device based on NicTO: {"deviceId":1,"networkRateMbps":-1,"
> defaultNic":false,"pxeDisable":true,"nicUuid":"fe3f7db4-
> 9ff5-4b63-883b-e08bb6f7568b","uuid":"8ef0b547-f569-4987-
> ac80-9ebf72f44fdd","ip":"0.0.0.0","netmask":"0.0.0.0","
> gateway":"0.0.0.0","mac":"02:00:5c:9c:00:0a","broadcastType":"LinkLocal","
> type":"Control","isSecurityGroupEnabled":false,"name":"vSwitch1"}
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare network 
> on vmwaresvs vSwitch1 with name prefix: cloud.private 
> catalina.out:WARN  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Unrecognized 
> broadcast type in VmwareResource, type: LinkLocal. Use vlan info from
> labeling: untagged
> catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.private.untagged.0.1-vSwitch1 is ready on vSwitch vSwitch1 
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC 
> device on network cloud.private.untagged.0.1-vSwitch1
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Configure VNC 
> port for VM r-4-VM, port: 5939, host: ns53xxxx.ip-149-56-xx.net 
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Found existing 
> disk info from volume path: ROOT-4-000001 catalina.out:INFO  
> [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to 
> connect to 149.56.xxx.xx catalina.out:WARN  
> [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-8baef180)
> (logid:c6f5e75e) Task (job-3247) has been pending for 129 seconds 
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not 
> connect to 149.56.xxx.xx due to java.net.ConnectException: Connection 
> timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to 
> connect to 149.56.xxx.xx catalina.out:WARN  
> [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-3026cb48)
> (logid:9ea6ca9e) Task (job-3247) has been pending for 189 seconds 
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not 
> connect to 149.56.xxx.xx due to java.net.ConnectException: Connection 
> timed out catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, 
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to 
> connect to 149.56.xxx.xx
>
>
> Any suggestion? Please.
>
> Regards,
> Cristian
>
> -----Original Message-----
> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> Sent: Wednesday, October 11, 2017 3:41 PM
> To: users@cloudstack.apache.org
> Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the 
> database
>
> Christian,
>
> The chances are I have not done the exact same upgrade as you are 
> attempting. the generic upgrade of the database (assuming the right 
> template and other things) is quite straight forward. You will need to 
> give us at least the stacktrace and any related information. from the 
> log above it, if this is a reproducible error.
>
> On Wed, Oct 11, 2017 at 2:34 PM, Ciobanu Cristian 
> <cristian.c@istream.today
> >
> wrote:
>
> > I have restored back to 4.5.2, but if I upgrade again using the 
> > steps provide in documentation I will end with the same error..  did 
> > you tested this? With tha configuration (acs 4.5.2, vmware, local storage) ?
> >
> > Regards,
> > Cristian
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > Sent: Wednesday, October 11, 2017 3:29 PM
> > To: users@cloudstack.apache.org
> > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the 
> > database
> >
> > Hi the management server on starting up should take care of the upgrade.
> > if you have a clustered MS make sure to start only one. In your case 
> > after the failure you should probably restore the latest backup 
> > before
> continuing.
> >
> > On Wed, Oct 11, 2017 at 2:26 PM, Ciobanu Cristian 
> > <cristian.c@istream.today
> > >
> > wrote:
> >
> > > Hello,
> > >
> > >
> > >
> > >    In order to upgrade cloudstack to 4.9 I have followed this 
> > > doecumentation
> > > :
> > > http://docs.cloudstack.apache.org/projects/cloudstack-
> > > release-notes/en/4.9.2
> > > .0/upgrade/upgrade-4.5.html  and ended with this :
> > >
> > >
> > >
> > > management-server.log:2017-10-11 08:05:35,612 WARN 
> > > [o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:) 
> > > Exception encountered during context initialization - cancelling 
> > > refresh
> > attempt:
> > > org.springframework.context.ApplicationContextException: Failed to 
> > > start bean 'cloudStackLifeCycle'; nested exception is
> > > com.cloud.utils.exception.CloudRuntimeException: Unable to upgrade 
> > > the database
> > >
> > >
> > >
> > > Is there a valid upgrade procedure or script for : Cloudstack 
> > > 4.5.2 to CloudStack 4.9 ( VMware with Local Storage ) ?
> > >
> > >
> > >
> > > (cloudstack was installed from shapeblue repo)
> > >
> > >
> > >
> > >
> > >
> > > Thank you.
> > >
> > > Cristian
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Daan
> >
> >
>
>
> --
> Daan
>
>


--
Daan


Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Daan Hoogland <da...@gmail.com>.
that is a tricky one. You can destroy the vm from VmWare and set the state
to stopped in the DB. It should then start a new one.
check if r-4-VM/ROOT-4-000001.vmdk has gone from the datastore.

the cause is probably some synchronisation issue. We've seen and solved it
before but are yet to find the root cause.


On Wed, Oct 11, 2017 at 4:23 PM, Ciobanu Cristian <cr...@istream.today>
wrote:

> Hello,
>
>    Was my mistake, forgot to add the system template from cloudstack
> interface before starting the upgrade, now I have other issue, looks like
> after I have enabled one of the zone and click on upgrade virtual router I
> still see the state "starting" if I check the router from vSphere side is
> up.
>
>
>   This is what is see :
>
> tting
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Chose disk
> controller for vol ROOT -> osdefault, based on root disk controller
> settings at global configuration setting.
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Folder r-4-VM exists on datastore
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1]
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on datastore
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001-flat.vmdk on [datastore1]
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001-flat.vmdk exists on
> datastore
> catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync [datastore1]
> ROOT-4-000001-flat.vmdk->[datastore1] r-4-VM/ROOT-4-000001-flat.vmdk
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001-delta.vmdk on [datastore1]
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001-delta.vmdk does not exist
> on datastore
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1]
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on datastore
> catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync [datastore1]
> ROOT-4-000001.vmdk->[datastore1] r-4-VM/ROOT-4-000001.vmdk
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] r-4-VM
> catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50
> ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand)
> (logid:6b3de9fa) File [datastore1] r-4-VM/ROOT-4-000001.vmdk exists on
> datastore
> catalina.out:INFO  [c.c.u.n.n.p.NiciraNvpApiVersion]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) NSX API VERSION:
> NOT PRESENT
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC device
> based on NicTO: {"deviceId":0,"networkRateMbps":200,"
> defaultNic":true,"pxeDisable":true,"nicUuid":"fc405400-6839-
> 4eea-9b30-e5e14b375322","uuid":"41f42309-8eaa-425a-a4c5-
> c8b5a88f92a2","ip":"149.56.xxx.xx","netmask":"255.255.
> 255.224","gateway":"149.56.164.62","mac":"06:76:62:00:00:
> 4b","dns1":"172.20.0.3","dns2":"8.8.8.8","broadcastType":"
> Native","type":"Guest","broadcastUri":"vlan://untagged","
> isSecurityGroupEnabled":false,"name":"vSwitch1"}
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare network on
> vmwaresvs vSwitch1 with name prefix: cloud.guest
> catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
> catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
> catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC device
> on network cloud.guest.untagged.200.1-vSwitch1
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC device
> based on NicTO: {"deviceId":1,"networkRateMbps":-1,"
> defaultNic":false,"pxeDisable":true,"nicUuid":"fe3f7db4-
> 9ff5-4b63-883b-e08bb6f7568b","uuid":"8ef0b547-f569-4987-
> ac80-9ebf72f44fdd","ip":"0.0.0.0","netmask":"0.0.0.0","
> gateway":"0.0.0.0","mac":"02:00:5c:9c:00:0a","broadcastType":"LinkLocal","
> type":"Control","isSecurityGroupEnabled":false,"name":"vSwitch1"}
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare network on
> vmwaresvs vSwitch1 with name prefix: cloud.private
> catalina.out:WARN  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Unrecognized
> broadcast type in VmwareResource, type: LinkLocal. Use vlan info from
> labeling: untagged
> catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network
> cloud.private.untagged.0.1-vSwitch1 is ready on vSwitch vSwitch1
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC device
> on network cloud.private.untagged.0.1-vSwitch1
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Configure VNC port
> for VM r-4-VM, port: 5939, host: ns53xxxx.ip-149-56-xx.net
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Found existing disk
> info from volume path: ROOT-4-000001
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to connect
> to 149.56.xxx.xx
> catalina.out:WARN  [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-8baef180)
> (logid:c6f5e75e) Task (job-3247) has been pending for 129 seconds
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not connect
> to 149.56.xxx.xx due to java.net.ConnectException: Connection timed out
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to connect
> to 149.56.xxx.xx
> catalina.out:WARN  [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-3026cb48)
> (logid:9ea6ca9e) Task (job-3247) has been pending for 189 seconds
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not connect
> to 149.56.xxx.xx due to java.net.ConnectException: Connection timed out
> catalina.out:INFO  [c.c.h.v.r.VmwareResource]
> (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net,
> job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to connect
> to 149.56.xxx.xx
>
>
> Any suggestion? Please.
>
> Regards,
> Cristian
>
> -----Original Message-----
> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> Sent: Wednesday, October 11, 2017 3:41 PM
> To: users@cloudstack.apache.org
> Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database
>
> Christian,
>
> The chances are I have not done the exact same upgrade as you are
> attempting. the generic upgrade of the database (assuming the right
> template and other things) is quite straight forward. You will need to give
> us at least the stacktrace and any related information. from the log above
> it, if this is a reproducible error.
>
> On Wed, Oct 11, 2017 at 2:34 PM, Ciobanu Cristian <cristian.c@istream.today
> >
> wrote:
>
> > I have restored back to 4.5.2, but if I upgrade again using the steps
> > provide in documentation I will end with the same error..  did you
> > tested this? With tha configuration (acs 4.5.2, vmware, local storage) ?
> >
> > Regards,
> > Cristian
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > Sent: Wednesday, October 11, 2017 3:29 PM
> > To: users@cloudstack.apache.org
> > Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the
> > database
> >
> > Hi the management server on starting up should take care of the upgrade.
> > if you have a clustered MS make sure to start only one. In your case
> > after the failure you should probably restore the latest backup before
> continuing.
> >
> > On Wed, Oct 11, 2017 at 2:26 PM, Ciobanu Cristian
> > <cristian.c@istream.today
> > >
> > wrote:
> >
> > > Hello,
> > >
> > >
> > >
> > >    In order to upgrade cloudstack to 4.9 I have followed this
> > > doecumentation
> > > :
> > > http://docs.cloudstack.apache.org/projects/cloudstack-
> > > release-notes/en/4.9.2
> > > .0/upgrade/upgrade-4.5.html  and ended with this :
> > >
> > >
> > >
> > > management-server.log:2017-10-11 08:05:35,612 WARN
> > > [o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:)
> > > Exception encountered during context initialization - cancelling
> > > refresh
> > attempt:
> > > org.springframework.context.ApplicationContextException: Failed to
> > > start bean 'cloudStackLifeCycle'; nested exception is
> > > com.cloud.utils.exception.CloudRuntimeException: Unable to upgrade
> > > the database
> > >
> > >
> > >
> > > Is there a valid upgrade procedure or script for : Cloudstack 4.5.2
> > > to CloudStack 4.9 ( VMware with Local Storage ) ?
> > >
> > >
> > >
> > > (cloudstack was installed from shapeblue repo)
> > >
> > >
> > >
> > >
> > >
> > > Thank you.
> > >
> > > Cristian
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Daan
> >
> >
>
>
> --
> Daan
>
>


-- 
Daan

RE: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Ciobanu Cristian <cr...@istream.today>.
Hello,

   Was my mistake, forgot to add the system template from cloudstack interface before starting the upgrade, now I have other issue, looks like after I have enabled one of the zone and click on upgrade virtual router I still see the state "starting" if I check the router from vSphere side is up.


  This is what is see :

tting
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Chose disk controller for vol ROOT -> osdefault, based on root disk controller settings at global configuration setting.
catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Folder r-4-VM exists on datastore
catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1]
catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on datastore
catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Search file ROOT-4-000001-flat.vmdk on [datastore1]
catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) File [datastore1] ROOT-4-000001-flat.vmdk exists on datastore
catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync [datastore1] ROOT-4-000001-flat.vmdk->[datastore1] r-4-VM/ROOT-4-000001-flat.vmdk
catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Search file ROOT-4-000001-delta.vmdk on [datastore1]
catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) File [datastore1] ROOT-4-000001-delta.vmdk does not exist on datastore
catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1]
catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) File [datastore1] ROOT-4-000001.vmdk exists on datastore
catalina.out:INFO  [c.c.s.r.VmwareStorageLayoutHelper] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) sync [datastore1] ROOT-4-000001.vmdk->[datastore1] r-4-VM/ROOT-4-000001.vmdk
catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Search file ROOT-4-000001.vmdk on [datastore1] r-4-VM
catalina.out:INFO  [c.c.h.v.m.DatastoreMO] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) File [datastore1] r-4-VM/ROOT-4-000001.vmdk exists on datastore
catalina.out:INFO  [c.c.u.n.n.p.NiciraNvpApiVersion] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) NSX API VERSION:  NOT PRESENT
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC device based on NicTO: {"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":true,"nicUuid":"fc405400-6839-4eea-9b30-e5e14b375322","uuid":"41f42309-8eaa-425a-a4c5-c8b5a88f92a2","ip":"149.56.xxx.xx","netmask":"255.255.255.224","gateway":"149.56.164.62","mac":"06:76:62:00:00:4b","dns1":"172.20.0.3","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":false,"name":"vSwitch1"}
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare network on vmwaresvs vSwitch1 with name prefix: cloud.guest
catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network cloud.guest.untagged.200.1-vSwitch1 is ready on vSwitch vSwitch1
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC device on network cloud.guest.untagged.200.1-vSwitch1
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare NIC device based on NicTO: {"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"fe3f7db4-9ff5-4b63-883b-e08bb6f7568b","uuid":"8ef0b547-f569-4987-ac80-9ebf72f44fdd","ip":"0.0.0.0","netmask":"0.0.0.0","gateway":"0.0.0.0","mac":"02:00:5c:9c:00:0a","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false,"name":"vSwitch1"}
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Prepare network on vmwaresvs vSwitch1 with name prefix: cloud.private
catalina.out:WARN  [c.c.h.v.r.VmwareResource] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Unrecognized broadcast type in VmwareResource, type: LinkLocal. Use vlan info from labeling: untagged
catalina.out:INFO  [c.c.h.v.m.HypervisorHostHelper] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Network cloud.private.untagged.0.1-vSwitch1 is ready on vSwitch vSwitch1
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Preparing NIC device on network cloud.private.untagged.0.1-vSwitch1
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Configure VNC port for VM r-4-VM, port: 5939, host: ns53xxxx.ip-149-56-xx.net
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: StartCommand) (logid:6b3de9fa) Found existing disk info from volume path: ROOT-4-000001
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to connect to 149.56.xxx.xx
catalina.out:WARN  [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-8baef180) (logid:c6f5e75e) Task (job-3247) has been pending for 129 seconds
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not connect to 149.56.xxx.xx due to java.net.ConnectException: Connection timed out
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to connect to 149.56.xxx.xx
catalina.out:WARN  [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-3026cb48) (logid:9ea6ca9e) Task (job-3247) has been pending for 189 seconds
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Could not connect to 149.56.xxx.xx due to java.net.ConnectException: Connection timed out
catalina.out:INFO  [c.c.h.v.r.VmwareResource] (DirectAgent-381:ctx-5f16ff50 ns53xxxx.ip-149-56-xx.net, job-3247/job-3248, cmd: CheckSshCommand) (logid:6b3de9fa) Trying to connect to 149.56.xxx.xx


Any suggestion? Please.

Regards,
Cristian

-----Original Message-----
From: Daan Hoogland [mailto:daan.hoogland@gmail.com] 
Sent: Wednesday, October 11, 2017 3:41 PM
To: users@cloudstack.apache.org
Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Christian,

The chances are I have not done the exact same upgrade as you are attempting. the generic upgrade of the database (assuming the right template and other things) is quite straight forward. You will need to give us at least the stacktrace and any related information. from the log above it, if this is a reproducible error.

On Wed, Oct 11, 2017 at 2:34 PM, Ciobanu Cristian <cr...@istream.today>
wrote:

> I have restored back to 4.5.2, but if I upgrade again using the steps 
> provide in documentation I will end with the same error..  did you 
> tested this? With tha configuration (acs 4.5.2, vmware, local storage) ?
>
> Regards,
> Cristian
>
> -----Original Message-----
> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> Sent: Wednesday, October 11, 2017 3:29 PM
> To: users@cloudstack.apache.org
> Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the 
> database
>
> Hi the management server on starting up should take care of the upgrade.
> if you have a clustered MS make sure to start only one. In your case 
> after the failure you should probably restore the latest backup before continuing.
>
> On Wed, Oct 11, 2017 at 2:26 PM, Ciobanu Cristian 
> <cristian.c@istream.today
> >
> wrote:
>
> > Hello,
> >
> >
> >
> >    In order to upgrade cloudstack to 4.9 I have followed this 
> > doecumentation
> > :
> > http://docs.cloudstack.apache.org/projects/cloudstack-
> > release-notes/en/4.9.2
> > .0/upgrade/upgrade-4.5.html  and ended with this :
> >
> >
> >
> > management-server.log:2017-10-11 08:05:35,612 WARN 
> > [o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:) 
> > Exception encountered during context initialization - cancelling 
> > refresh
> attempt:
> > org.springframework.context.ApplicationContextException: Failed to 
> > start bean 'cloudStackLifeCycle'; nested exception is
> > com.cloud.utils.exception.CloudRuntimeException: Unable to upgrade 
> > the database
> >
> >
> >
> > Is there a valid upgrade procedure or script for : Cloudstack 4.5.2 
> > to CloudStack 4.9 ( VMware with Local Storage ) ?
> >
> >
> >
> > (cloudstack was installed from shapeblue repo)
> >
> >
> >
> >
> >
> > Thank you.
> >
> > Cristian
> >
> >
> >
> >
> >
> >
> >
> >
>
>
> --
> Daan
>
>


--
Daan


Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Daan Hoogland <da...@gmail.com>.
Christian,

The chances are I have not done the exact same upgrade as you are
attempting. the generic upgrade of the database (assuming the right
template and other things) is quite straight forward. You will need to give
us at least the stacktrace and any related information. from the log above
it, if this is a reproducible error.

On Wed, Oct 11, 2017 at 2:34 PM, Ciobanu Cristian <cr...@istream.today>
wrote:

> I have restored back to 4.5.2, but if I upgrade again using the steps
> provide in documentation I will end with the same error..  did you tested
> this? With tha configuration (acs 4.5.2, vmware, local storage) ?
>
> Regards,
> Cristian
>
> -----Original Message-----
> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> Sent: Wednesday, October 11, 2017 3:29 PM
> To: users@cloudstack.apache.org
> Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database
>
> Hi the management server on starting up should take care of the upgrade.
> if you have a clustered MS make sure to start only one. In your case after
> the failure you should probably restore the latest backup before continuing.
>
> On Wed, Oct 11, 2017 at 2:26 PM, Ciobanu Cristian <cristian.c@istream.today
> >
> wrote:
>
> > Hello,
> >
> >
> >
> >    In order to upgrade cloudstack to 4.9 I have followed this
> > doecumentation
> > :
> > http://docs.cloudstack.apache.org/projects/cloudstack-
> > release-notes/en/4.9.2
> > .0/upgrade/upgrade-4.5.html  and ended with this :
> >
> >
> >
> > management-server.log:2017-10-11 08:05:35,612 WARN
> > [o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:)
> > Exception encountered during context initialization - cancelling refresh
> attempt:
> > org.springframework.context.ApplicationContextException: Failed to
> > start bean 'cloudStackLifeCycle'; nested exception is
> > com.cloud.utils.exception.CloudRuntimeException: Unable to upgrade the
> > database
> >
> >
> >
> > Is there a valid upgrade procedure or script for : Cloudstack 4.5.2 to
> > CloudStack 4.9 ( VMware with Local Storage ) ?
> >
> >
> >
> > (cloudstack was installed from shapeblue repo)
> >
> >
> >
> >
> >
> > Thank you.
> >
> > Cristian
> >
> >
> >
> >
> >
> >
> >
> >
>
>
> --
> Daan
>
>


-- 
Daan

RE: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Ciobanu Cristian <cr...@istream.today>.
I have restored back to 4.5.2, but if I upgrade again using the steps provide in documentation I will end with the same error..  did you tested this? With tha configuration (acs 4.5.2, vmware, local storage) ?

Regards,
Cristian

-----Original Message-----
From: Daan Hoogland [mailto:daan.hoogland@gmail.com] 
Sent: Wednesday, October 11, 2017 3:29 PM
To: users@cloudstack.apache.org
Subject: Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Hi the management server on starting up should take care of the upgrade. if you have a clustered MS make sure to start only one. In your case after the failure you should probably restore the latest backup before continuing.

On Wed, Oct 11, 2017 at 2:26 PM, Ciobanu Cristian <cr...@istream.today>
wrote:

> Hello,
>
>
>
>    In order to upgrade cloudstack to 4.9 I have followed this 
> doecumentation
> :
> http://docs.cloudstack.apache.org/projects/cloudstack-
> release-notes/en/4.9.2
> .0/upgrade/upgrade-4.5.html  and ended with this :
>
>
>
> management-server.log:2017-10-11 08:05:35,612 WARN 
> [o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:) 
> Exception encountered during context initialization - cancelling refresh attempt:
> org.springframework.context.ApplicationContextException: Failed to 
> start bean 'cloudStackLifeCycle'; nested exception is
> com.cloud.utils.exception.CloudRuntimeException: Unable to upgrade the 
> database
>
>
>
> Is there a valid upgrade procedure or script for : Cloudstack 4.5.2 to 
> CloudStack 4.9 ( VMware with Local Storage ) ?
>
>
>
> (cloudstack was installed from shapeblue repo)
>
>
>
>
>
> Thank you.
>
> Cristian
>
>
>
>
>
>
>
>


--
Daan


Re: Upgrade Cloudstack 4.5 to 4.9 - Unable to upgrade the database

Posted by Daan Hoogland <da...@gmail.com>.
Hi the management server on starting up should take care of the upgrade. if
you have a clustered MS make sure to start only one. In your case after the
failure you should probably restore the latest backup before continuing.

On Wed, Oct 11, 2017 at 2:26 PM, Ciobanu Cristian <cr...@istream.today>
wrote:

> Hello,
>
>
>
>    In order to upgrade cloudstack to 4.9 I have followed this
> doecumentation
> :
> http://docs.cloudstack.apache.org/projects/cloudstack-
> release-notes/en/4.9.2
> .0/upgrade/upgrade-4.5.html  and ended with this :
>
>
>
> management-server.log:2017-10-11 08:05:35,612 WARN
> [o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:) Exception
> encountered during context initialization - cancelling refresh attempt:
> org.springframework.context.ApplicationContextException: Failed to start
> bean 'cloudStackLifeCycle'; nested exception is
> com.cloud.utils.exception.CloudRuntimeException: Unable to upgrade the
> database
>
>
>
> Is there a valid upgrade procedure or script for : Cloudstack 4.5.2 to
> CloudStack 4.9 ( VMware with Local Storage ) ?
>
>
>
> (cloudstack was installed from shapeblue repo)
>
>
>
>
>
> Thank you.
>
> Cristian
>
>
>
>
>
>
>
>


-- 
Daan