You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Adam <ad...@gmail.com> on 2014/11/03 00:33:22 UTC
How to Upgrade Virtual Router from 4.3 to 4.4.1
I've performed a complete upgrade from 4.3 to 4.4.1 and followed the
instructions perfectly, but my Virtual Router will not start, nor am I able
to "upgrade" it.
I tried destroying the Virtual Router, starting any VM Instance, and
allowing CS to recreate the VR. However that didn't help. The VR still did
not start and it was still saying that it required to be upgraded.
[image: Inline image 1]
[image: Inline image 2]
[image: Inline image 3]
I know there are manual steps to finalize the systemvm upgrade, but I did
those steps:
http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
[root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 174
Server version: 5.1.73-log Source distribution
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input
statement.
mysql> select id,name from cloud.vm_template where name =
'systemvm-kvm-4.4';
+-----+------------------+
| id | name |
+-----+------------------+
| 274 | systemvm-kvm-4.4 |
+-----+------------------+
1 row in set (0.00 sec)
mysql> select type from cloud.vm_template where id='274';
+--------+
| type |
+--------+
| SYSTEM |
+--------+
1 row in set (0.00 sec)
mysql> select distinct(vm_template_id) from cloud.vm_instance where
type='ConsoleProxy' and hypervisor_type = 'KVM';
+----------------+
| vm_template_id |
+----------------+
| 274 |
+----------------+
1 row in set (0.00 sec)
mysql> select distinct(vm_template_id) from cloud.vm_instance where
type='SecondaryStorageVm' and hypervisor_type = 'KVM';
+----------------+
| vm_template_id |
+----------------+
| 274 |
+----------------+
1 row in set (0.03 sec)
mysql> select value from cloud.configuration where name =
'router.template.kvm';
+------------------+
| value |
+------------------+
| systemvm-kvm-4.4 |
+------------------+
1 row in set (0.00 sec)
mysql>
* PLEASE HELP ! *
Best Regards,
Adam Scarcella
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Pierre-Luc Dion <pd...@apache.org>.
Those the systemvm template systemvm-kvm-4.4 is mark as ready?
on xenserver there is a delay when upgrading the first systemVM while the
template is copied to the primary storage. Maybe that's what's happening.
On Sun, Nov 2, 2014 at 6:33 PM, Adam <ad...@gmail.com> wrote:
> I've performed a complete upgrade from 4.3 to 4.4.1 and followed the
> instructions perfectly, but my Virtual Router will not start, nor am I able
> to "upgrade" it.
>
> I tried destroying the Virtual Router, starting any VM Instance, and
> allowing CS to recreate the VR. However that didn't help. The VR still did
> not start and it was still saying that it required to be upgraded.
>
> [image: Inline image 1]
> [image: Inline image 2]
> [image: Inline image 3]
>
> I know there are manual steps to finalize the systemvm upgrade, but I did
> those steps:
>
>
> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
>
> [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
> Enter password:
> Welcome to the MySQL monitor. Commands end with ; or \g.
> Your MySQL connection id is 174
> Server version: 5.1.73-log Source distribution
>
> Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights
> reserved.
>
> Oracle is a registered trademark of Oracle Corporation and/or its
> affiliates. Other names may be trademarks of their respective
> owners.
>
> Type 'help;' or '\h' for help. Type '\c' to clear the current input
> statement.
>
> mysql> select id,name from cloud.vm_template where name =
> 'systemvm-kvm-4.4';
> +-----+------------------+
> | id | name |
> +-----+------------------+
> | 274 | systemvm-kvm-4.4 |
> +-----+------------------+
> 1 row in set (0.00 sec)
>
> mysql> select type from cloud.vm_template where id='274';
> +--------+
> | type |
> +--------+
> | SYSTEM |
> +--------+
> 1 row in set (0.00 sec)
>
> mysql> select distinct(vm_template_id) from cloud.vm_instance where
> type='ConsoleProxy' and hypervisor_type = 'KVM';
> +----------------+
> | vm_template_id |
> +----------------+
> | 274 |
> +----------------+
> 1 row in set (0.00 sec)
>
> mysql> select distinct(vm_template_id) from cloud.vm_instance where
> type='SecondaryStorageVm' and hypervisor_type = 'KVM';
> +----------------+
> | vm_template_id |
> +----------------+
> | 274 |
> +----------------+
> 1 row in set (0.03 sec)
>
> mysql> select value from cloud.configuration where name =
> 'router.template.kvm';
> +------------------+
> | value |
> +------------------+
> | systemvm-kvm-4.4 |
> +------------------+
> 1 row in set (0.00 sec)
>
> mysql>
>
> * PLEASE HELP ! *
>
> Best Regards,
>
> Adam Scarcella
>
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Adam <ad...@gmail.com>.
At the moment I don't have access to that system. I will only be able to
get logs in the morning.
Hypothetically speaking, if someone lost the entire management server, but
had a complete backup of the entire database only, how could one restore
the entire CloudStack ecosystem?
Best Regards,
Adam Scarcella
44 Avery Street
Ipswich, MA 01938
H: (978) 471-4000
M: (781) 367-8015
On Nov 3, 2014 5:58 PM, "Nux!" <nu...@li.nux.ro> wrote:
> Adam,
>
> I would have gone for the yum downgrade option. Please send us some logs.
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
> > From: "Adam" <ad...@gmail.com>
> > To: users@cloudstack.apache.org
> > Sent: Monday, 3 November, 2014 22:42:25
> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>
> > ok, so I've rolled back and now none of my hosts are connecting. My
> system
> > VMs are not running, and the only thing I can see in the logs is that my
> > zone is not ready for either the Console Proxy or the Secondary Storage
> VM
> > to start because none of my hosts are reachable, even though in the UI it
> > says they're all up and running.
> >
> > I'm really losing all patience and confidence with CloudStack. I
> currently
> > have a 10 person team who cannot access any of their VMs because
> CloudStack
> > can't seem to find itself.
> >
> > I stopped all services and erased all 4.4.1 packages on the management
> > server as well as all the KVM hosts.
> >
> > # STOP ALL CLOUDSTACK SERVICES ON MANAGEMENT HOST
> > service cloudstack-agent stop
> > service cloudstack-usage stop
> > service cloudstack-management stop
> >
> > # STOP CLOUDSTACK AGENT ON KVM HOST #N
> > service cloudstack-agent stop
> >
> > # DROP ALL DATABASES
> > # THIS SCRIPT DROPS ALL DATABASES IN THE CLOUDSTACK MySQL DB (cloud,
> > cloud_usage, cloudbridge)
> > /root/cronScripts/mysql_DropAllDatabases.sh
> >
> > # MODIFIED/etc/yum.repos.d/cloudstack.repo to use 4.3 repository
> > [root@cs-east-prod1 /]# cat /etc/yum.repos.d/cloudstack.repo
> > [cloudstack]
> > name=cloudstack
> > baseurl=http://cloudstack.apt-get.eu/rhel/4.3/
> > enabled=1
> > gpgcheck=0
> > [root@cs-east-prod1 /]#
> >
> > # YUM ERASE/FS CLEAN/YUM INSTALL ON MANAGEMENT HOST
> > yum -y erase cloudstack\*
> > rm -Rf /var/log/cloudstack/management
> > rm -Rf /usr/share/cloudstack*
> > rm -Rf /usr/share/doc/cloudstack*
> > rm -Rf /var/log/cloudstack
> > rm -Rf /etc/cloudstack
> > rm -Rf /var/cache/cloudstack
> > rm -Rf /var/cloudstack
> > yum clean all
> > yum -y install cloudstack\*
> >
> > # YUM ERASE/FS CLEAN/YUM INSTALL ON KVM HOST(S)
> > yum -y erase cloudstack\*
> > rm -Rf /usr/share/cloudstack*
> > rm -Rf /usr/share/doc/cloudstack*
> > rm -Rf /var/log/cloudstack
> > rm -Rf /etc/cloudstack
> > yum clean all
> > yum -y install cloudstack-agent
> >
> > # RAN cloudstack-setup-databases
> > cloudstack-setup-databases cloud:<PASSWORD> --deploy-as=root:<PASSWORD>
> -i
> > 10.97.38.10
> >
> > # RESTORED MySQL BACKUPS
> > mysql -u root -p<PASSWORD> cloud <
> > /var/www/html/mysql-backups/cloud_11-01-2014.sql
> > mysql -u root -p<PASSWORD> cloud <
> > /var/www/html/mysql-backups/cloud_usage_11-01-2014.sql
> > mysql -u root -p<PASSWORD> cloud <
> > /var/www/html/mysql-backups/cloud_meta_11-01-2014.sql
> > mysql -u root -p<PASSWORD> cloud <
> > /var/www/html/mysql-backups/cloudbridge_11-01-2014.sql
> >
> > # START ALL CLOUDSTACK SERVICES ON MANAGEMENT HOST
> > service cloudstack-management start
> > service cloudstack-usage start
> > service cloudstack-agent start
> >
> > # START ALL CLOUDSTACK SERVICES ON KVM HOST(S)
> > service cloudstack-agent start
> >
> > Where do I go from here?
> >
> > Best Regards,
> >
> >
> >
> > Adam Scarcella
> > On Nov 3, 2014 2:06 PM, "Nux!" <nu...@li.nux.ro> wrote:
> >
> >> Adam,
> >>
> >> I take it replacing the VR's disk did not work? Well, I'm hoping 4.4.2
> >> will come with a fixed template.
> >>
> >> Re downgrading this is how I did it:
> >> 1 - stop all services (mgmt and agents) and "yum downgrade
> cloudstack-\*"
> >> 2 - drop the DBs and import backups
> >> 3 - restart the downgraded services
> >> 4 - you'll also need to rebuild your VR, possibly "virsh destroy" the
> >> 4.4.1 one if it's still around
> >>
> >> Good luck
> >>
> >> --
> >> Sent from the Delta quadrant using Borg technology!
> >>
> >> Nux!
> >> www.nux.ro
> >>
> >> ----- Original Message -----
> >> > From: "Adam" <ad...@gmail.com>
> >> > To: users@cloudstack.apache.org
> >> > Sent: Monday, 3 November, 2014 18:51:13
> >> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >>
> >> > I've decided to revert back to 4.3.1 since I cannot wait any loner to
> get
> >> > 4.4.1 working.
> >> >
> >> > I have a complete backup of all three databases just before I
> attempted
> >> to
> >> > upgrade so I'm assuming all I would have to do is remove 4.4.1,
> reinstall
> >> > 4.3.1, restore the db and then start everything back up correct?
> >> >
> >> > Are there any official steps to roll back?
> >> >
> >> > Best Regards,
> >> >
> >> >
> >> >
> >> > Adam Scarcella
> >> >
> >> > On Mon, Nov 3, 2014 at 12:13 PM, Nux! <nu...@li.nux.ro> wrote:
> >> >
> >> >> What I am saying - and excuse my poor explanations - is that the HV
> may
> >> >> have already cached the old template, so the VR still uses the old
> >> broken
> >> >> one.
> >> >>
> >> >> What I was suggesting is find out which
> >> /var/lib/libvirt/images/blah.qcow2
> >> >> the VR is using and replace it with my template.
> >> >>
> >> >> --
> >> >> Sent from the Delta quadrant using Borg technology!
> >> >>
> >> >> Nux!
> >> >> www.nux.ro
> >> >>
> >> >> ----- Original Message -----
> >> >> > From: "Adam" <ad...@gmail.com>
> >> >> > To: users@cloudstack.apache.org
> >> >> > Sent: Monday, 3 November, 2014 17:03:01
> >> >> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >> >>
> >> >> >>
> >> >> >> Not sure if this operation that you performed replaces the current
> >> >> >> systemvm template and *especially if it replaces it also on the
> >> >> >> hypervisor where it is running.*
> >> >> >>
> >> >> >
> >> >> > That procedure is the one documented in the official CloudStack
> >> >> > documentation so I hope it actually replaces the systemvm template.
> >> The
> >> >> > final output from that command tells it was successful:
> >> >> >
> >> >> > Successfully installed system VM template to
> >> >> > /export/secondary/template/tmpl/1/274/
> >> >> >
> >> >> > Also, as stated before, the VR is not running on any hypervisor /
> host
> >> >> > because I cannot get it to run, so I'm unsure what you mean when
> you
> >> >> > say "*especially
> >> >> > if it replaces it also on the hypervisor where it is running.*"
> >> >> >
> >> >> > One more thing I would try is to make sure the VR on the HV is also
> >> using
> >> >> >> this image. You can try to compare md5sums of the image on the
> >> secondary
> >> >> >> storage and also the backend file for the VR's disk image
> (qemu-img
> >> info
> >> >> >> will show it to you) on the HV.
> >> >> >> If the sums differ then I would try to replace the VR disk image
> >> with my
> >> >> >> template directly on the HV and see if that does it.
> >> >> >
> >> >> >
> >> >> > I really don't understand what you're asking me to do.
> >> >> >
> >> >> > Best Regards,
> >> >> >
> >> >> >
> >> >> >
> >> >> > Adam Scarcella
> >> >> >
> >> >> > On Mon, Nov 3, 2014 at 11:50 AM, Nux! <nu...@li.nux.ro> wrote:
> >> >> >
> >> >> >> Adam,
> >> >> >>
> >> >> >> Not sure if this operation that you performed replaces the current
> >> >> >> systemvm template and especially if it replaces it also on the
> >> >> hypervisor
> >> >> >> where it is running.
> >> >> >>
> >> >> >> One more thing I would try is to make sure the VR on the HV is
> also
> >> >> using
> >> >> >> this image. You can try to compare md5sums of the image on the
> >> secondary
> >> >> >> storage and also the backend file for the VR's disk image
> (qemu-img
> >> info
> >> >> >> will show it to you) on the HV.
> >> >> >> If the sums differ then I would try to replace the VR disk image
> >> with my
> >> >> >> template directly on the HV and see if that does it.
> >> >> >>
> >> >> >> --
> >> >> >> Sent from the Delta quadrant using Borg technology!
> >> >> >>
> >> >> >> Nux!
> >> >> >> www.nux.ro
> >> >> >>
> >> >> >> ----- Original Message -----
> >> >> >> > From: "Adam" <ad...@gmail.com>
> >> >> >> > To: users@cloudstack.apache.org
> >> >> >> > Sent: Monday, 3 November, 2014 16:08:05
> >> >> >> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >> >> >>
> >> >> >> > Thanks HTH,
> >> >> >> >
> >> >> >> > I decided to try to use your already modified qcow2 file:
> >> >> >> >
> >> >> >> > [root@cs-east-prod1 /]#
> >> >> >> >
> >> >> >>
> >> >>
> >>
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> >> >> >> > -m /export/secondary -u
> >> >> >> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> >> >> >> > -h kvm -F
> >> >> >> > --2014-11-03 10:11:32--
> >> >> >> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> >> >> >> > Resolving tmp.nux.ro... 217.19.15.108
> >> >> >> > Connecting to tmp.nux.ro|217.19.15.108|:80... connected.
> >> >> >> > HTTP request sent, awaiting response... 200 OK
> >> >> >> > Length: 305498799 (291M) [application/x-bzip2]
> >> >> >> > Saving to:
> >> >> >> >
> >> >> >>
> >> >>
> >>
> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
> >> >> >> >
> >> >> >> >
> >> >> >>
> >> >>
> >>
> 100%[===================================================================================================================>]
> >> >> >> > 305,498,799 9.02M/s in 37s
> >> >> >> >
> >> >> >> > 2014-11-03 10:12:10 (7.82 MB/s) -
> >> >> >> >
> >> >> >>
> >> >>
> >>
> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
> >> >> >> > saved [305498799/305498799]
> >> >> >> >
> >> >> >> > Uncompressing to
> >> >> >> >
> >> >> >>
> >> >>
> >>
> /usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2.tmp
> >> >> >> > (type bz2)...could take a long time
> >> >> >> > Moving to
> >> >> >> >
> >> >> >>
> >> >>
> >>
> /export/secondary/template/tmpl/1/274///00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2...could
> >> >> >> > take a while
> >> >> >> > Successfully installed system VM template to
> >> >> >> > /export/secondary/template/tmpl/1/274/
> >> >> >> > [root@cs-east-prod1 /]#
> >> >> >> >
> >> >> >> > Then I went into the Admin GUI and tried to start the VR, but it
> >> >> failed
> >> >> >> > with the exact same message:
> >> >> >> >
> >> >> >> > [image: Inline image 1]
> >> >> >> >
> >> >> >> > So I destroyed the Virtual Router and stopped and started the
> >> >> following
> >> >> >> > services:
> >> >> >> >
> >> >> >> > [root@cs-east-prod1 /]# service iptables stop
> >> >> >> > iptables: Setting chains to policy ACCEPT: nat filter [
> OK ]
> >> >> >> > iptables: Flushing firewall rules: [
> OK ]
> >> >> >> > iptables: Unloading modules: [
> OK ]
> >> >> >> > [root@cs-east-prod1 /]# service cloudstack-management stop
> >> >> >> > Stopping cloudstack-management: [
> OK ]
> >> >> >> > [root@cs-east-prod1 /]# service cloudstack-agent stop
> >> >> >> > Stopping Cloud Agent:
> >> >> >> > [root@cs-east-prod1 /]# service cloudstack-usage stop
> >> >> >> > Stopping CloudStack Usage Monitor cloudstack-usage [
> OK ]
> >> >> >> > [root@cs-east-prod1 /]# service cloudstack-management start
> >> >> >> > Starting cloudstack-management: [
> OK ]
> >> >> >> > [root@cs-east-prod1 /]# service cloudstack-usage start
> >> >> >> > Starting CloudStack Usage Monitor cloudstack-usage [
> OK ]
> >> >> >> > [root@cs-east-prod1 /]# service cloudstack-agent start
> >> >> >> > Starting Cloud Agent:
> >> >> >> > [root@cs-east-prod1 /]#
> >> >> >> >
> >> >> >> > I then tried starting any VM instance to have CS automatically
> >> >> re-create
> >> >> >> > the VR, but that failed as well.
> >> >> >> >
> >> >> >> > Motty, I also looked at trying to increase the RAM and CPU for
> the
> >> >> router
> >> >> >> > but only saw one option
> >> >> >> >
> >> >> >> > [image: Inline image 2]
> >> >> >> >
> >> >> >> > So I created a new Service Offering with twice as much
> resources:
> >> >> >> >
> >> >> >> > [image: Inline image 3]
> >> >> >> >
> >> >> >> > Then upgraded the service offering for the VR
> >> >> >> >
> >> >> >> > [image: Inline image 4]
> >> >> >> >
> >> >> >> > [image: Inline image 6]
> >> >> >> >
> >> >> >> > But it failed still with the exact same error:
> >> >> >> >
> >> >> >> > [image: Inline image 7]
> >> >> >> >
> >> >> >> > Best Regards,
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> > Adam Scarcella
> >> >> >> >
> >> >> >> > On Mon, Nov 3, 2014 at 10:23 AM, motty cruz <
> motty.cruz@gmail.com>
> >> >> >> wrote:
> >> >> >> >
> >> >> >> >> Adam,
> >> >> >> >> what are the specifications of the VR? can you upgrade the RAM
> and
> >> >> CPU
> >> >> >> and
> >> >> >> >> try to restart that VR? I had similar issue and giving the VR
> more
> >> >> RAM
> >> >> >> and
> >> >> >> >> CPU will start the VR.
> >> >> >> >>
> >> >> >> >> Thanks,
> >> >> >> >> Motty
> >> >> >> >>
> >> >> >> >> On Mon, Nov 3, 2014 at 7:04 AM, Nux! <nu...@li.nux.ro> wrote:
> >> >> >> >>
> >> >> >> >> > Adam,
> >> >> >> >> >
> >> >> >> >> > You can modify the qcow2 file that is in your secondary
> storage,
> >> >> i.e.
> >> >> >> you
> >> >> >> >> > should mount that storage somewhere and go to
> >> >> >> >> > /mount/point/template/tmpl/274 and modify the qcow2 file.
> >> >> >> >> >
> >> >> >> >> > Alternatively you could use my already modified copy to
> >> overwrite
> >> >> your
> >> >> >> >> > uuid.qcow2 with (at your own risk of course):
> >> >> >> >> > http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> >> >> >> >> >
> >> >> >> >> > HTH
> >> >> >> >> >
> >> >> >> >> > --
> >> >> >> >> > Sent from the Delta quadrant using Borg technology!
> >> >> >> >> >
> >> >> >> >> > Nux!
> >> >> >> >> > www.nux.ro
> >> >> >> >> >
> >> >> >> >> > ----- Original Message -----
> >> >> >> >> > > From: "Adam" <ad...@gmail.com>
> >> >> >> >> > > To: users@cloudstack.apache.org
> >> >> >> >> > > Sent: Monday, 3 November, 2014 14:58:58
> >> >> >> >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to
> 4.4.1
> >> >> >> >> >
> >> >> >> >> > > [root@cs-east-prod1 management]# netstat -a|grep 9090
> >> >> >> >> > > unix 3 [ ] STREAM CONNECTED 29090
> >> >> >> >> > > [root@cs-east-prod1 management]# nmap localhost
> >> >> >> >> > >
> >> >> >> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47
> >> EST
> >> >> >> >> > > Nmap scan report for localhost (127.0.0.1)
> >> >> >> >> > > Host is up (0.000015s latency).
> >> >> >> >> > > Other addresses for localhost (not scanned): 127.0.0.1
> >> >> >> >> > > Not shown: 990 closed ports
> >> >> >> >> > > PORT STATE SERVICE
> >> >> >> >> > > 22/tcp open ssh
> >> >> >> >> > > 25/tcp open smtp
> >> >> >> >> > > 80/tcp open http
> >> >> >> >> > > 111/tcp open rpcbind
> >> >> >> >> > > 199/tcp open smux
> >> >> >> >> > > 631/tcp open ipp
> >> >> >> >> > > 2049/tcp open nfs
> >> >> >> >> > > 3306/tcp open mysql
> >> >> >> >> > > 8080/tcp open http-proxy
> >> >> >> >> > > 9090/tcp open zeus-admin
> >> >> >> >> > >
> >> >> >> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
> >> >> >> >> > > [root@cs-east-prod1 management]# telnet localhost 9090
> >> >> >> >> > > Trying ::1...
> >> >> >> >> > > Connected to localhost.
> >> >> >> >> > > Escape character is '^]'.
> >> >> >> >> > > Connection closed by foreign host.
> >> >> >> >> > > [root@cs-east-prod1 management]# nmap 10.97.38.10
> >> >> >> >> > >
> >> >> >> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49
> >> EST
> >> >> >> >> > > Nmap scan report for cs-east-prod1.na.tibco.com
> (10.97.38.10)
> >> >> >> >> > > Host is up (0.000015s latency).
> >> >> >> >> > > Not shown: 993 closed ports
> >> >> >> >> > > PORT STATE SERVICE
> >> >> >> >> > > 22/tcp open ssh
> >> >> >> >> > > 80/tcp open http
> >> >> >> >> > > 111/tcp open rpcbind
> >> >> >> >> > > 2049/tcp open nfs
> >> >> >> >> > > 3306/tcp open mysql
> >> >> >> >> > > 8080/tcp open http-proxy
> >> >> >> >> > > 9090/tcp open zeus-admin
> >> >> >> >> > >
> >> >> >> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
> >> >> >> >> > > [root@cs-east-prod1 management]# telnet 10.97.38.10 9090
> >> >> >> >> > > Trying 10.97.38.10...
> >> >> >> >> > > Connected to 10.97.38.10.
> >> >> >> >> > > Escape character is '^]'.
> >> >> >> >> > > Connection closed by foreign host.
> >> >> >> >> > > [root@cs-east-prod1 management]#
> >> >> >> >> > >
> >> >> >> >> > > Also, I checked for the /opt/cloud/bin/ directory on my
> >> >> Management
> >> >> >> >> Host,
> >> >> >> >> > > but it's not there:
> >> >> >> >> > >
> >> >> >> >> > > [root@cs-east-prod1 management]# cd /opt/cloud/bin/
> >> >> >> >> > > -bash: cd: /opt/cloud/bin/: No such file or directory
> >> >> >> >> > > [root@cs-east-prod1 management]# cd /
> >> >> >> >> > > [root@cs-east-prod1 /]# find . -name "createipAlias.sh"
> >> >> >> >> > > [root@cs-east-prod1 /]# find . -name "createIpAlias.sh"
> >> >> >> >> > > [root@cs-east-prod1 /]#
> >> >> >> >> > >
> >> >> >> >> > > Are you saying that the /opt/cloud/bin/createipAlias.sh
> >> script is
> >> >> >> >> inside
> >> >> >> >> > > the Virtual Router VM? If so, how could I possibly change
> that
> >> >> >> since it
> >> >> >> >> > > won't start?
> >> >> >> >> > >
> >> >> >> >> > > Best Regards,
> >> >> >> >> > >
> >> >> >> >> > >
> >> >> >> >> > >
> >> >> >> >> > > Adam Scarcella
> >> >> >> >> > >
> >> >> >> >> > > On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion <
> >> >> >> pdion891@apache.org>
> >> >> >> >> > wrote:
> >> >> >> >> > >
> >> >> >> >> > >> Could it be possible the firewall is blocking port 9090
> from
> >> the
> >> >> >> >> > >> management-server?
> >> >> >> >> > >>
> >> >> >> >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
> >> >> >> (main:null)
> >> >> >> >> > Unable
> >> >> >> >> > >> to ping management server at 10.97.38.10:9090 due to
> >> >> >> ConnectException
> >> >> >> >> > >> java.net.ConnectException: Connection refused
> >> >> >> >> > >>
> >> >> >> >> > >> On Mon, Nov 3, 2014 at 9:02 AM, Nux! <nu...@li.nux.ro>
> wrote:
> >> >> >> >> > >>
> >> >> >> >> > >> > BTW the agent logs should help clarify this.
> >> >> >> >> > >> > The aforementioned changes must be done either in the
> >> current
> >> >> >> >> > (broken) VR
> >> >> >> >> > >> > or in the systemvm (after which you should redeploy the
> VR
> >> to
> >> >> >> pick
> >> >> >> >> up
> >> >> >> >> > the
> >> >> >> >> > >> > change).
> >> >> >> >> > >> >
> >> >> >> >> > >> > HTH
> >> >> >> >> > >> > Lucian
> >> >> >> >> > >> >
> >> >> >> >> > >> > --
> >> >> >> >> > >> > Sent from the Delta quadrant using Borg technology!
> >> >> >> >> > >> >
> >> >> >> >> > >> > Nux!
> >> >> >> >> > >> > www.nux.ro
> >> >> >> >> > >> >
> >> >> >> >> > >> > ----- Original Message -----
> >> >> >> >> > >> > > From: "Nux!" <nu...@li.nux.ro>
> >> >> >> >> > >> > > To: users@cloudstack.apache.org
> >> >> >> >> > >> > > Sent: Monday, 3 November, 2014 13:53:10
> >> >> >> >> > >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to
> >> 4.4.1
> >> >> >> >> > >> >
> >> >> >> >> > >> > > Hello,
> >> >> >> >> > >> > >
> >> >> >> >> > >> > > I think you are hitting a problem I also had:
> >> >> >> >> > >> > > "2014-11-03 08:16:17,024 ERROR
> >> >> >> [c.c.v.VirtualMachineManagerImpl]
> >> >> >> >> > >> > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556
> >> >> >> >> ctx-fcb072b2)
> >> >> >> >> > >> > Failed to
> >> >> >> >> > >> > > start instance VM[DomainRouter|r-4758-VM]
> >> >> >> >> > >> > > com.cloud.utils.exception.ExecutionException: Unable
> to
> >> >> start
> >> >> >> >> > >> > > VM[DomainRouter|r-4758-VM] due to error in
> finalizeStart,
> >> >> not
> >> >> >> >> > retrying"
> >> >> >> >> > >> > >
> >> >> >> >> > >> > > It's
> >> https://issues.apache.org/jira/browse/CLOUDSTACK-7781
> >> >> >> >> > >> > >
> >> >> >> >> > >> > > Basically the systemvm template has some problems, you
> >> need
> >> >> to
> >> >> >> >> > somehow
> >> >> >> >> > >> > copy
> >> >> >> >> > >> > > /opt/cloud/bin/createIpAlias.sh to
> >> >> >> /opt/cloud/bin/createipAlias.sh
> >> >> >> >> > to
> >> >> >> >> > >> > get it
> >> >> >> >> > >> > > working.
> >> >> >> >> > >> > >
> >> >> >> >> > >> > > --
> >> >> >> >> > >> > > Sent from the Delta quadrant using Borg technology!
> >> >> >> >> > >> > >
> >> >> >> >> > >> > > Nux!
> >> >> >> >> > >> > > www.nux.ro
> >> >> >> >> > >> > >
> >> >> >> >> > >> > > ----- Original Message -----
> >> >> >> >> > >> > >> From: "Adam" <ad...@gmail.com>
> >> >> >> >> > >> > >> To: users@cloudstack.apache.org
> >> >> >> >> > >> > >> Sent: Monday, 3 November, 2014 13:32:12
> >> >> >> >> > >> > >> Subject: Re: How to Upgrade Virtual Router from 4.3
> to
> >> >> 4.4.1
> >> >> >> >> > >> > >
> >> >> >> >> > >> > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release
> >> >> >> >> > >> > >> CentOS release 6.6 (Final)
> >> >> >> >> > >> > >> [root@cs-east-prod1 ~]# uname -a
> >> >> >> >> > >> > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed
> >> Oct 15
> >> >> >> >> > 04:27:16
> >> >> >> >> > >> UTC
> >> >> >> >> > >> > >> 2014 x86_64 x86_64 x86_64 GNU/Linux
> >> >> >> >> > >> > >> [root@cs-east-prod1 ~]# virsh --version
> >> >> >> >> > >> > >> 0.10.2
> >> >> >> >> > >> > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
> >> >> >> >> > >> > >> /etc/cloudstack/agent/log4j-cloud.xml
> >> >> >> >> > >> > >> [root@cs-east-prod1 ~]# service
> cloudstack-management
> >> stop
> >> >> >> >> > >> > >> Stopping cloudstack-management:
> >> >> [
> >> >> >> >> OK ]
> >> >> >> >> > >> > >> [root@cs-east-prod1 ~]# mv
> >> >> >> >> > >> > >> /var/log/cloudstack/management/management-server.log
> >> >> >> >> > >> > >> /var/log/cloudstack/management/management-server.bak
> >> >> >> >> > >> > >> [root@cs-east-prod1 ~]# service
> cloudstack-management
> >> >> start
> >> >> >> >> > >> > >> Starting cloudstack-management:
> >> >> [
> >> >> >> >> OK ]
> >> >> >> >> > >> > >> [root@cs-east-prod1 ~]#
> >> >> >> >> > >> > >>
> >> >> >> >> > >> > >> Please see attached ZIP of clean
> management-server.log
> >> in
> >> >> >> DEBUG
> >> >> >> >> > mode.
> >> >> >> >> > >> At
> >> >> >> >> > >> > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the
> >> >> attached
> >> >> >> >> > >> > >> management-server.log*) I tried starting the Virtual
> >> Router
> >> >> >> from
> >> >> >> >> > the
> >> >> >> >> > >> > >> Management UI and received this "Status" message in
> the
> >> UI.
> >> >> >> >> > >> > >>
> >> >> >> >> > >> > >> [image: Inline image 1]
> >> >> >> >> > >> > >>
> >> >> >> >> > >> > >> I can see in the management-server.log that while the
> >> >> >> >> > >> > cloudstack-management
> >> >> >> >> > >> > >> service is initializing, it throws the following
> ERROR
> >> at
> >> >> >> exactly
> >> >> >> >> > >> > (*2014-11-03
> >> >> >> >> > >> > >> 08:14:36,813 on line 342 of the attached
> >> >> >> management-server.log*):
> >> >> >> >> > >> > >>
> >> >> >> >> > >> > >> 2014-11-03 08:14:36,813 ERROR
> [c.c.c.ClusterManagerImpl]
> >> >> >> >> > (main:null)
> >> >> >> >> > >> > Unable
> >> >> >> >> > >> > >> to ping management server at 10.97.38.10:9090 due to
> >> >> >> >> > ConnectException
> >> >> >> >> > >> > >> java.net.ConnectException: Connection refused
> >> >> >> >> > >> > >> at sun.nio.ch.Net.connect0(Native Method)
> >> >> >> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:465)
> >> >> >> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:457)
> >> >> >> >> > >> > >> at
> >> >> >> >> >
> sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >>
> >> >> >> >>
> >> >>
> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> >
> >> >> >> >>
> >> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> >
> >> >>
> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >>
> >> >> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> >
> >> >> >> >>
> >> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >>
> >> >> org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> >
> >> >> >> >>
> >> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> >
> >> >>
> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >>
> >> >> >> >>
> >> >>
> org.apache.catalina.core.StandardService.start(StandardService.java:516)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> >
> >> >>
> org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
> >> >> >> >> > >> > >> at
> >> >> >> >> org.apache.catalina.startup.Catalina.start(Catalina.java:593)
> >> >> >> >> > >> > >> at
> >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> >> >> >> Method)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> >> >> >> > >> > >> at
> >> >> >> >> > >> > >>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> >> >> >> > >> > >> at
> java.lang.reflect.Method.invoke(Method.java:606)
> >> >> >> >> > >> > >> at
> >> >> >> >> >
> org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> >> >> >> >> > >> > >> at
> >> >> >> >> >
> org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> >> >> >> >> > >> > >> 2014-11-03 08:14:36,816 INFO
> [c.c.c.ClusterManagerImpl]
> >> >> >> >> > (main:null)
> >> >> >> >> > >> > >> Detected that another management node with the same
> IP
> >> >> >> >> 10.97.38.10
> >> >> >> >> > is
> >> >> >> >> > >> > >> considered as running in DB, however it is not
> >> pingable, we
> >> >> >> will
> >> >> >> >> > >> > continue
> >> >> >> >> > >> > >> cluster initialization with this management server
> node
> >> >> >> >> > >> > >>
> >> >> >> >> > >> > >> I'm not sure what that error means, but more than
> likely
> >> >> >> that's
> >> >> >> >> the
> >> >> >> >> > >> > reason
> >> >> >> >> > >> > >> why the Virtual Router will not start.
> >> >> >> >> > >> > >>
> >> >> >> >> > >> > >> * PLEASE HELP ! *
> >> >> >> >> > >> > >>
> >> >> >> >> > >> > >> Best Regards,
> >> >> >> >> > >> > >>
> >> >> >> >> > >> > >>
> >> >> >> >> > >> > >>
> >> >> >> >> > >> > >> Adam Scarcella
> >> >> >> >> > >> > >>
> >> >> >> >> > >> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro>
> >> >> wrote:
> >> >> >> >> > >> > >>
> >> >> >> >> > >> > >>> Hi,
> >> >> >> >> > >> > >>>
> >> >> >> >> > >> > >>> More agent and management logs please, make sure
> they
> >> are
> >> >> in
> >> >> >> >> > DEBUG[1]
> >> >> >> >> > >> > mode.
> >> >> >> >> > >> > >>>
> >> >> >> >> > >> > >>> [1] -
> >> >> >> >> > >> > >>>
> >> >> >> >> > >>
> >> >> >> >>
> >> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
> >> >> >> >> > >> > >>>
> >> >> >> >> > >> > >>> --
> >> >> >> >> > >> > >>> Sent from the Delta quadrant using Borg technology!
> >> >> >> >> > >> > >>>
> >> >> >> >> > >> > >>> Nux!
> >> >> >> >> > >> > >>> www.nux.ro
> >> >> >> >> > >> > >>>
> >> >> >> >> > >> > >>> ----- Original Message -----
> >> >> >> >> > >> > >>> > From: "Adam" <ad...@gmail.com>
> >> >> >> >> > >> > >>> > To: users@cloudstack.apache.org
> >> >> >> >> > >> > >>> > Sent: Sunday, 2 November, 2014 23:33:22
> >> >> >> >> > >> > >>> > Subject: How to Upgrade Virtual Router from 4.3 to
> >> 4.4.1
> >> >> >> >> > >> > >>>
> >> >> >> >> > >> > >>> > I've performed a complete upgrade from 4.3 to
> 4.4.1
> >> and
> >> >> >> >> followed
> >> >> >> >> > >> the
> >> >> >> >> > >> > >>> > instructions perfectly, but my Virtual Router will
> >> not
> >> >> >> start,
> >> >> >> >> > nor
> >> >> >> >> > >> am
> >> >> >> >> > >> > I
> >> >> >> >> > >> > >>> able
> >> >> >> >> > >> > >>> > to "upgrade" it.
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> > I tried destroying the Virtual Router, starting
> any
> >> VM
> >> >> >> >> Instance,
> >> >> >> >> > >> and
> >> >> >> >> > >> > >>> > allowing CS to recreate the VR. However that
> didn't
> >> >> help.
> >> >> >> The
> >> >> >> >> VR
> >> >> >> >> > >> > still
> >> >> >> >> > >> > >>> did
> >> >> >> >> > >> > >>> > not start and it was still saying that it required
> >> to be
> >> >> >> >> > upgraded.
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> > [image: Inline image 1]
> >> >> >> >> > >> > >>> > [image: Inline image 2]
> >> >> >> >> > >> > >>> > [image: Inline image 3]
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> > I know there are manual steps to finalize the
> >> systemvm
> >> >> >> >> upgrade,
> >> >> >> >> > but
> >> >> >> >> > >> > I did
> >> >> >> >> > >> > >>> > those steps:
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>>
> >> >> >> >> > >> >
> >> >> >> >> > >>
> >> >> >> >> >
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u
> root
> >> -p
> >> >> >> >> > >> > >>> > Enter password:
> >> >> >> >> > >> > >>> > Welcome to the MySQL monitor. Commands end with
> ; or
> >> >> \g.
> >> >> >> >> > >> > >>> > Your MySQL connection id is 174
> >> >> >> >> > >> > >>> > Server version: 5.1.73-log Source distribution
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> > Copyright (c) 2000, 2013, Oracle and/or its
> >> affiliates.
> >> >> All
> >> >> >> >> > rights
> >> >> >> >> > >> > >>> reserved.
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> > Oracle is a registered trademark of Oracle
> >> Corporation
> >> >> >> and/or
> >> >> >> >> > its
> >> >> >> >> > >> > >>> > affiliates. Other names may be trademarks of their
> >> >> >> respective
> >> >> >> >> > >> > >>> > owners.
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> > Type 'help;' or '\h' for help. Type '\c' to clear
> the
> >> >> >> current
> >> >> >> >> > input
> >> >> >> >> > >> > >>> > statement.
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> > mysql> select id,name from cloud.vm_template where
> >> name
> >> >> =
> >> >> >> >> > >> > >>> > 'systemvm-kvm-4.4';
> >> >> >> >> > >> > >>> > +-----+------------------+
> >> >> >> >> > >> > >>> >| id | name |
> >> >> >> >> > >> > >>> > +-----+------------------+
> >> >> >> >> > >> > >>> >| 274 | systemvm-kvm-4.4 |
> >> >> >> >> > >> > >>> > +-----+------------------+
> >> >> >> >> > >> > >>> > 1 row in set (0.00 sec)
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> > mysql> select type from cloud.vm_template where
> >> >> id='274';
> >> >> >> >> > >> > >>> > +--------+
> >> >> >> >> > >> > >>> >| type |
> >> >> >> >> > >> > >>> > +--------+
> >> >> >> >> > >> > >>> >| SYSTEM |
> >> >> >> >> > >> > >>> > +--------+
> >> >> >> >> > >> > >>> > 1 row in set (0.00 sec)
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> > mysql> select distinct(vm_template_id) from
> >> >> >> cloud.vm_instance
> >> >> >> >> > where
> >> >> >> >> > >> > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
> >> >> >> >> > >> > >>> > +----------------+
> >> >> >> >> > >> > >>> >| vm_template_id |
> >> >> >> >> > >> > >>> > +----------------+
> >> >> >> >> > >> > >>> >| 274 |
> >> >> >> >> > >> > >>> > +----------------+
> >> >> >> >> > >> > >>> > 1 row in set (0.00 sec)
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> > mysql> select distinct(vm_template_id) from
> >> >> >> cloud.vm_instance
> >> >> >> >> > where
> >> >> >> >> > >> > >>> > type='SecondaryStorageVm' and hypervisor_type =
> >> 'KVM';
> >> >> >> >> > >> > >>> > +----------------+
> >> >> >> >> > >> > >>> >| vm_template_id |
> >> >> >> >> > >> > >>> > +----------------+
> >> >> >> >> > >> > >>> >| 274 |
> >> >> >> >> > >> > >>> > +----------------+
> >> >> >> >> > >> > >>> > 1 row in set (0.03 sec)
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> > mysql> select value from cloud.configuration where
> >> name
> >> >> =
> >> >> >> >> > >> > >>> > 'router.template.kvm';
> >> >> >> >> > >> > >>> > +------------------+
> >> >> >> >> > >> > >>> >| value |
> >> >> >> >> > >> > >>> > +------------------+
> >> >> >> >> > >> > >>> >| systemvm-kvm-4.4 |
> >> >> >> >> > >> > >>> > +------------------+
> >> >> >> >> > >> > >>> > 1 row in set (0.00 sec)
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> > mysql>
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> > * PLEASE HELP ! *
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > >>> > Best Regards,
> >> >> >> >> > >> > >>> >
> >> >> >> >> > >> > > >> > Adam Scarcella
> >> >> >> >> > >> >
> >> >> >> >> >
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> --
> >> >> >> >> Thanks for your support,
> >> >> >> >> Motty
>
>
>
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Nux! <nu...@li.nux.ro>.
Adam,
I would have gone for the yum downgrade option. Please send us some logs.
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Adam" <ad...@gmail.com>
> To: users@cloudstack.apache.org
> Sent: Monday, 3 November, 2014 22:42:25
> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> ok, so I've rolled back and now none of my hosts are connecting. My system
> VMs are not running, and the only thing I can see in the logs is that my
> zone is not ready for either the Console Proxy or the Secondary Storage VM
> to start because none of my hosts are reachable, even though in the UI it
> says they're all up and running.
>
> I'm really losing all patience and confidence with CloudStack. I currently
> have a 10 person team who cannot access any of their VMs because CloudStack
> can't seem to find itself.
>
> I stopped all services and erased all 4.4.1 packages on the management
> server as well as all the KVM hosts.
>
> # STOP ALL CLOUDSTACK SERVICES ON MANAGEMENT HOST
> service cloudstack-agent stop
> service cloudstack-usage stop
> service cloudstack-management stop
>
> # STOP CLOUDSTACK AGENT ON KVM HOST #N
> service cloudstack-agent stop
>
> # DROP ALL DATABASES
> # THIS SCRIPT DROPS ALL DATABASES IN THE CLOUDSTACK MySQL DB (cloud,
> cloud_usage, cloudbridge)
> /root/cronScripts/mysql_DropAllDatabases.sh
>
> # MODIFIED/etc/yum.repos.d/cloudstack.repo to use 4.3 repository
> [root@cs-east-prod1 /]# cat /etc/yum.repos.d/cloudstack.repo
> [cloudstack]
> name=cloudstack
> baseurl=http://cloudstack.apt-get.eu/rhel/4.3/
> enabled=1
> gpgcheck=0
> [root@cs-east-prod1 /]#
>
> # YUM ERASE/FS CLEAN/YUM INSTALL ON MANAGEMENT HOST
> yum -y erase cloudstack\*
> rm -Rf /var/log/cloudstack/management
> rm -Rf /usr/share/cloudstack*
> rm -Rf /usr/share/doc/cloudstack*
> rm -Rf /var/log/cloudstack
> rm -Rf /etc/cloudstack
> rm -Rf /var/cache/cloudstack
> rm -Rf /var/cloudstack
> yum clean all
> yum -y install cloudstack\*
>
> # YUM ERASE/FS CLEAN/YUM INSTALL ON KVM HOST(S)
> yum -y erase cloudstack\*
> rm -Rf /usr/share/cloudstack*
> rm -Rf /usr/share/doc/cloudstack*
> rm -Rf /var/log/cloudstack
> rm -Rf /etc/cloudstack
> yum clean all
> yum -y install cloudstack-agent
>
> # RAN cloudstack-setup-databases
> cloudstack-setup-databases cloud:<PASSWORD> --deploy-as=root:<PASSWORD> -i
> 10.97.38.10
>
> # RESTORED MySQL BACKUPS
> mysql -u root -p<PASSWORD> cloud <
> /var/www/html/mysql-backups/cloud_11-01-2014.sql
> mysql -u root -p<PASSWORD> cloud <
> /var/www/html/mysql-backups/cloud_usage_11-01-2014.sql
> mysql -u root -p<PASSWORD> cloud <
> /var/www/html/mysql-backups/cloud_meta_11-01-2014.sql
> mysql -u root -p<PASSWORD> cloud <
> /var/www/html/mysql-backups/cloudbridge_11-01-2014.sql
>
> # START ALL CLOUDSTACK SERVICES ON MANAGEMENT HOST
> service cloudstack-management start
> service cloudstack-usage start
> service cloudstack-agent start
>
> # START ALL CLOUDSTACK SERVICES ON KVM HOST(S)
> service cloudstack-agent start
>
> Where do I go from here?
>
> Best Regards,
>
>
>
> Adam Scarcella
> On Nov 3, 2014 2:06 PM, "Nux!" <nu...@li.nux.ro> wrote:
>
>> Adam,
>>
>> I take it replacing the VR's disk did not work? Well, I'm hoping 4.4.2
>> will come with a fixed template.
>>
>> Re downgrading this is how I did it:
>> 1 - stop all services (mgmt and agents) and "yum downgrade cloudstack-\*"
>> 2 - drop the DBs and import backups
>> 3 - restart the downgraded services
>> 4 - you'll also need to rebuild your VR, possibly "virsh destroy" the
>> 4.4.1 one if it's still around
>>
>> Good luck
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
>>
>> ----- Original Message -----
>> > From: "Adam" <ad...@gmail.com>
>> > To: users@cloudstack.apache.org
>> > Sent: Monday, 3 November, 2014 18:51:13
>> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>>
>> > I've decided to revert back to 4.3.1 since I cannot wait any loner to get
>> > 4.4.1 working.
>> >
>> > I have a complete backup of all three databases just before I attempted
>> to
>> > upgrade so I'm assuming all I would have to do is remove 4.4.1, reinstall
>> > 4.3.1, restore the db and then start everything back up correct?
>> >
>> > Are there any official steps to roll back?
>> >
>> > Best Regards,
>> >
>> >
>> >
>> > Adam Scarcella
>> >
>> > On Mon, Nov 3, 2014 at 12:13 PM, Nux! <nu...@li.nux.ro> wrote:
>> >
>> >> What I am saying - and excuse my poor explanations - is that the HV may
>> >> have already cached the old template, so the VR still uses the old
>> broken
>> >> one.
>> >>
>> >> What I was suggesting is find out which
>> /var/lib/libvirt/images/blah.qcow2
>> >> the VR is using and replace it with my template.
>> >>
>> >> --
>> >> Sent from the Delta quadrant using Borg technology!
>> >>
>> >> Nux!
>> >> www.nux.ro
>> >>
>> >> ----- Original Message -----
>> >> > From: "Adam" <ad...@gmail.com>
>> >> > To: users@cloudstack.apache.org
>> >> > Sent: Monday, 3 November, 2014 17:03:01
>> >> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> >>
>> >> >>
>> >> >> Not sure if this operation that you performed replaces the current
>> >> >> systemvm template and *especially if it replaces it also on the
>> >> >> hypervisor where it is running.*
>> >> >>
>> >> >
>> >> > That procedure is the one documented in the official CloudStack
>> >> > documentation so I hope it actually replaces the systemvm template.
>> The
>> >> > final output from that command tells it was successful:
>> >> >
>> >> > Successfully installed system VM template to
>> >> > /export/secondary/template/tmpl/1/274/
>> >> >
>> >> > Also, as stated before, the VR is not running on any hypervisor / host
>> >> > because I cannot get it to run, so I'm unsure what you mean when you
>> >> > say "*especially
>> >> > if it replaces it also on the hypervisor where it is running.*"
>> >> >
>> >> > One more thing I would try is to make sure the VR on the HV is also
>> using
>> >> >> this image. You can try to compare md5sums of the image on the
>> secondary
>> >> >> storage and also the backend file for the VR's disk image (qemu-img
>> info
>> >> >> will show it to you) on the HV.
>> >> >> If the sums differ then I would try to replace the VR disk image
>> with my
>> >> >> template directly on the HV and see if that does it.
>> >> >
>> >> >
>> >> > I really don't understand what you're asking me to do.
>> >> >
>> >> > Best Regards,
>> >> >
>> >> >
>> >> >
>> >> > Adam Scarcella
>> >> >
>> >> > On Mon, Nov 3, 2014 at 11:50 AM, Nux! <nu...@li.nux.ro> wrote:
>> >> >
>> >> >> Adam,
>> >> >>
>> >> >> Not sure if this operation that you performed replaces the current
>> >> >> systemvm template and especially if it replaces it also on the
>> >> hypervisor
>> >> >> where it is running.
>> >> >>
>> >> >> One more thing I would try is to make sure the VR on the HV is also
>> >> using
>> >> >> this image. You can try to compare md5sums of the image on the
>> secondary
>> >> >> storage and also the backend file for the VR's disk image (qemu-img
>> info
>> >> >> will show it to you) on the HV.
>> >> >> If the sums differ then I would try to replace the VR disk image
>> with my
>> >> >> template directly on the HV and see if that does it.
>> >> >>
>> >> >> --
>> >> >> Sent from the Delta quadrant using Borg technology!
>> >> >>
>> >> >> Nux!
>> >> >> www.nux.ro
>> >> >>
>> >> >> ----- Original Message -----
>> >> >> > From: "Adam" <ad...@gmail.com>
>> >> >> > To: users@cloudstack.apache.org
>> >> >> > Sent: Monday, 3 November, 2014 16:08:05
>> >> >> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> >> >>
>> >> >> > Thanks HTH,
>> >> >> >
>> >> >> > I decided to try to use your already modified qcow2 file:
>> >> >> >
>> >> >> > [root@cs-east-prod1 /]#
>> >> >> >
>> >> >>
>> >>
>> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
>> >> >> > -m /export/secondary -u
>> >> >> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
>> >> >> > -h kvm -F
>> >> >> > --2014-11-03 10:11:32--
>> >> >> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
>> >> >> > Resolving tmp.nux.ro... 217.19.15.108
>> >> >> > Connecting to tmp.nux.ro|217.19.15.108|:80... connected.
>> >> >> > HTTP request sent, awaiting response... 200 OK
>> >> >> > Length: 305498799 (291M) [application/x-bzip2]
>> >> >> > Saving to:
>> >> >> >
>> >> >>
>> >>
>> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
>> >> >> >
>> >> >> >
>> >> >>
>> >>
>> 100%[===================================================================================================================>]
>> >> >> > 305,498,799 9.02M/s in 37s
>> >> >> >
>> >> >> > 2014-11-03 10:12:10 (7.82 MB/s) -
>> >> >> >
>> >> >>
>> >>
>> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
>> >> >> > saved [305498799/305498799]
>> >> >> >
>> >> >> > Uncompressing to
>> >> >> >
>> >> >>
>> >>
>> /usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2.tmp
>> >> >> > (type bz2)...could take a long time
>> >> >> > Moving to
>> >> >> >
>> >> >>
>> >>
>> /export/secondary/template/tmpl/1/274///00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2...could
>> >> >> > take a while
>> >> >> > Successfully installed system VM template to
>> >> >> > /export/secondary/template/tmpl/1/274/
>> >> >> > [root@cs-east-prod1 /]#
>> >> >> >
>> >> >> > Then I went into the Admin GUI and tried to start the VR, but it
>> >> failed
>> >> >> > with the exact same message:
>> >> >> >
>> >> >> > [image: Inline image 1]
>> >> >> >
>> >> >> > So I destroyed the Virtual Router and stopped and started the
>> >> following
>> >> >> > services:
>> >> >> >
>> >> >> > [root@cs-east-prod1 /]# service iptables stop
>> >> >> > iptables: Setting chains to policy ACCEPT: nat filter [ OK ]
>> >> >> > iptables: Flushing firewall rules: [ OK ]
>> >> >> > iptables: Unloading modules: [ OK ]
>> >> >> > [root@cs-east-prod1 /]# service cloudstack-management stop
>> >> >> > Stopping cloudstack-management: [ OK ]
>> >> >> > [root@cs-east-prod1 /]# service cloudstack-agent stop
>> >> >> > Stopping Cloud Agent:
>> >> >> > [root@cs-east-prod1 /]# service cloudstack-usage stop
>> >> >> > Stopping CloudStack Usage Monitor cloudstack-usage [ OK ]
>> >> >> > [root@cs-east-prod1 /]# service cloudstack-management start
>> >> >> > Starting cloudstack-management: [ OK ]
>> >> >> > [root@cs-east-prod1 /]# service cloudstack-usage start
>> >> >> > Starting CloudStack Usage Monitor cloudstack-usage [ OK ]
>> >> >> > [root@cs-east-prod1 /]# service cloudstack-agent start
>> >> >> > Starting Cloud Agent:
>> >> >> > [root@cs-east-prod1 /]#
>> >> >> >
>> >> >> > I then tried starting any VM instance to have CS automatically
>> >> re-create
>> >> >> > the VR, but that failed as well.
>> >> >> >
>> >> >> > Motty, I also looked at trying to increase the RAM and CPU for the
>> >> router
>> >> >> > but only saw one option
>> >> >> >
>> >> >> > [image: Inline image 2]
>> >> >> >
>> >> >> > So I created a new Service Offering with twice as much resources:
>> >> >> >
>> >> >> > [image: Inline image 3]
>> >> >> >
>> >> >> > Then upgraded the service offering for the VR
>> >> >> >
>> >> >> > [image: Inline image 4]
>> >> >> >
>> >> >> > [image: Inline image 6]
>> >> >> >
>> >> >> > But it failed still with the exact same error:
>> >> >> >
>> >> >> > [image: Inline image 7]
>> >> >> >
>> >> >> > Best Regards,
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > Adam Scarcella
>> >> >> >
>> >> >> > On Mon, Nov 3, 2014 at 10:23 AM, motty cruz <mo...@gmail.com>
>> >> >> wrote:
>> >> >> >
>> >> >> >> Adam,
>> >> >> >> what are the specifications of the VR? can you upgrade the RAM and
>> >> CPU
>> >> >> and
>> >> >> >> try to restart that VR? I had similar issue and giving the VR more
>> >> RAM
>> >> >> and
>> >> >> >> CPU will start the VR.
>> >> >> >>
>> >> >> >> Thanks,
>> >> >> >> Motty
>> >> >> >>
>> >> >> >> On Mon, Nov 3, 2014 at 7:04 AM, Nux! <nu...@li.nux.ro> wrote:
>> >> >> >>
>> >> >> >> > Adam,
>> >> >> >> >
>> >> >> >> > You can modify the qcow2 file that is in your secondary storage,
>> >> i.e.
>> >> >> you
>> >> >> >> > should mount that storage somewhere and go to
>> >> >> >> > /mount/point/template/tmpl/274 and modify the qcow2 file.
>> >> >> >> >
>> >> >> >> > Alternatively you could use my already modified copy to
>> overwrite
>> >> your
>> >> >> >> > uuid.qcow2 with (at your own risk of course):
>> >> >> >> > http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
>> >> >> >> >
>> >> >> >> > HTH
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > Sent from the Delta quadrant using Borg technology!
>> >> >> >> >
>> >> >> >> > Nux!
>> >> >> >> > www.nux.ro
>> >> >> >> >
>> >> >> >> > ----- Original Message -----
>> >> >> >> > > From: "Adam" <ad...@gmail.com>
>> >> >> >> > > To: users@cloudstack.apache.org
>> >> >> >> > > Sent: Monday, 3 November, 2014 14:58:58
>> >> >> >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> >> >> >> >
>> >> >> >> > > [root@cs-east-prod1 management]# netstat -a|grep 9090
>> >> >> >> > > unix 3 [ ] STREAM CONNECTED 29090
>> >> >> >> > > [root@cs-east-prod1 management]# nmap localhost
>> >> >> >> > >
>> >> >> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47
>> EST
>> >> >> >> > > Nmap scan report for localhost (127.0.0.1)
>> >> >> >> > > Host is up (0.000015s latency).
>> >> >> >> > > Other addresses for localhost (not scanned): 127.0.0.1
>> >> >> >> > > Not shown: 990 closed ports
>> >> >> >> > > PORT STATE SERVICE
>> >> >> >> > > 22/tcp open ssh
>> >> >> >> > > 25/tcp open smtp
>> >> >> >> > > 80/tcp open http
>> >> >> >> > > 111/tcp open rpcbind
>> >> >> >> > > 199/tcp open smux
>> >> >> >> > > 631/tcp open ipp
>> >> >> >> > > 2049/tcp open nfs
>> >> >> >> > > 3306/tcp open mysql
>> >> >> >> > > 8080/tcp open http-proxy
>> >> >> >> > > 9090/tcp open zeus-admin
>> >> >> >> > >
>> >> >> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
>> >> >> >> > > [root@cs-east-prod1 management]# telnet localhost 9090
>> >> >> >> > > Trying ::1...
>> >> >> >> > > Connected to localhost.
>> >> >> >> > > Escape character is '^]'.
>> >> >> >> > > Connection closed by foreign host.
>> >> >> >> > > [root@cs-east-prod1 management]# nmap 10.97.38.10
>> >> >> >> > >
>> >> >> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49
>> EST
>> >> >> >> > > Nmap scan report for cs-east-prod1.na.tibco.com (10.97.38.10)
>> >> >> >> > > Host is up (0.000015s latency).
>> >> >> >> > > Not shown: 993 closed ports
>> >> >> >> > > PORT STATE SERVICE
>> >> >> >> > > 22/tcp open ssh
>> >> >> >> > > 80/tcp open http
>> >> >> >> > > 111/tcp open rpcbind
>> >> >> >> > > 2049/tcp open nfs
>> >> >> >> > > 3306/tcp open mysql
>> >> >> >> > > 8080/tcp open http-proxy
>> >> >> >> > > 9090/tcp open zeus-admin
>> >> >> >> > >
>> >> >> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
>> >> >> >> > > [root@cs-east-prod1 management]# telnet 10.97.38.10 9090
>> >> >> >> > > Trying 10.97.38.10...
>> >> >> >> > > Connected to 10.97.38.10.
>> >> >> >> > > Escape character is '^]'.
>> >> >> >> > > Connection closed by foreign host.
>> >> >> >> > > [root@cs-east-prod1 management]#
>> >> >> >> > >
>> >> >> >> > > Also, I checked for the /opt/cloud/bin/ directory on my
>> >> Management
>> >> >> >> Host,
>> >> >> >> > > but it's not there:
>> >> >> >> > >
>> >> >> >> > > [root@cs-east-prod1 management]# cd /opt/cloud/bin/
>> >> >> >> > > -bash: cd: /opt/cloud/bin/: No such file or directory
>> >> >> >> > > [root@cs-east-prod1 management]# cd /
>> >> >> >> > > [root@cs-east-prod1 /]# find . -name "createipAlias.sh"
>> >> >> >> > > [root@cs-east-prod1 /]# find . -name "createIpAlias.sh"
>> >> >> >> > > [root@cs-east-prod1 /]#
>> >> >> >> > >
>> >> >> >> > > Are you saying that the /opt/cloud/bin/createipAlias.sh
>> script is
>> >> >> >> inside
>> >> >> >> > > the Virtual Router VM? If so, how could I possibly change that
>> >> >> since it
>> >> >> >> > > won't start?
>> >> >> >> > >
>> >> >> >> > > Best Regards,
>> >> >> >> > >
>> >> >> >> > >
>> >> >> >> > >
>> >> >> >> > > Adam Scarcella
>> >> >> >> > >
>> >> >> >> > > On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion <
>> >> >> pdion891@apache.org>
>> >> >> >> > wrote:
>> >> >> >> > >
>> >> >> >> > >> Could it be possible the firewall is blocking port 9090 from
>> the
>> >> >> >> > >> management-server?
>> >> >> >> > >>
>> >> >> >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
>> >> >> (main:null)
>> >> >> >> > Unable
>> >> >> >> > >> to ping management server at 10.97.38.10:9090 due to
>> >> >> ConnectException
>> >> >> >> > >> java.net.ConnectException: Connection refused
>> >> >> >> > >>
>> >> >> >> > >> On Mon, Nov 3, 2014 at 9:02 AM, Nux! <nu...@li.nux.ro> wrote:
>> >> >> >> > >>
>> >> >> >> > >> > BTW the agent logs should help clarify this.
>> >> >> >> > >> > The aforementioned changes must be done either in the
>> current
>> >> >> >> > (broken) VR
>> >> >> >> > >> > or in the systemvm (after which you should redeploy the VR
>> to
>> >> >> pick
>> >> >> >> up
>> >> >> >> > the
>> >> >> >> > >> > change).
>> >> >> >> > >> >
>> >> >> >> > >> > HTH
>> >> >> >> > >> > Lucian
>> >> >> >> > >> >
>> >> >> >> > >> > --
>> >> >> >> > >> > Sent from the Delta quadrant using Borg technology!
>> >> >> >> > >> >
>> >> >> >> > >> > Nux!
>> >> >> >> > >> > www.nux.ro
>> >> >> >> > >> >
>> >> >> >> > >> > ----- Original Message -----
>> >> >> >> > >> > > From: "Nux!" <nu...@li.nux.ro>
>> >> >> >> > >> > > To: users@cloudstack.apache.org
>> >> >> >> > >> > > Sent: Monday, 3 November, 2014 13:53:10
>> >> >> >> > >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to
>> 4.4.1
>> >> >> >> > >> >
>> >> >> >> > >> > > Hello,
>> >> >> >> > >> > >
>> >> >> >> > >> > > I think you are hitting a problem I also had:
>> >> >> >> > >> > > "2014-11-03 08:16:17,024 ERROR
>> >> >> [c.c.v.VirtualMachineManagerImpl]
>> >> >> >> > >> > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556
>> >> >> >> ctx-fcb072b2)
>> >> >> >> > >> > Failed to
>> >> >> >> > >> > > start instance VM[DomainRouter|r-4758-VM]
>> >> >> >> > >> > > com.cloud.utils.exception.ExecutionException: Unable to
>> >> start
>> >> >> >> > >> > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart,
>> >> not
>> >> >> >> > retrying"
>> >> >> >> > >> > >
>> >> >> >> > >> > > It's
>> https://issues.apache.org/jira/browse/CLOUDSTACK-7781
>> >> >> >> > >> > >
>> >> >> >> > >> > > Basically the systemvm template has some problems, you
>> need
>> >> to
>> >> >> >> > somehow
>> >> >> >> > >> > copy
>> >> >> >> > >> > > /opt/cloud/bin/createIpAlias.sh to
>> >> >> /opt/cloud/bin/createipAlias.sh
>> >> >> >> > to
>> >> >> >> > >> > get it
>> >> >> >> > >> > > working.
>> >> >> >> > >> > >
>> >> >> >> > >> > > --
>> >> >> >> > >> > > Sent from the Delta quadrant using Borg technology!
>> >> >> >> > >> > >
>> >> >> >> > >> > > Nux!
>> >> >> >> > >> > > www.nux.ro
>> >> >> >> > >> > >
>> >> >> >> > >> > > ----- Original Message -----
>> >> >> >> > >> > >> From: "Adam" <ad...@gmail.com>
>> >> >> >> > >> > >> To: users@cloudstack.apache.org
>> >> >> >> > >> > >> Sent: Monday, 3 November, 2014 13:32:12
>> >> >> >> > >> > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to
>> >> 4.4.1
>> >> >> >> > >> > >
>> >> >> >> > >> > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release
>> >> >> >> > >> > >> CentOS release 6.6 (Final)
>> >> >> >> > >> > >> [root@cs-east-prod1 ~]# uname -a
>> >> >> >> > >> > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed
>> Oct 15
>> >> >> >> > 04:27:16
>> >> >> >> > >> UTC
>> >> >> >> > >> > >> 2014 x86_64 x86_64 x86_64 GNU/Linux
>> >> >> >> > >> > >> [root@cs-east-prod1 ~]# virsh --version
>> >> >> >> > >> > >> 0.10.2
>> >> >> >> > >> > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
>> >> >> >> > >> > >> /etc/cloudstack/agent/log4j-cloud.xml
>> >> >> >> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management
>> stop
>> >> >> >> > >> > >> Stopping cloudstack-management:
>> >> [
>> >> >> >> OK ]
>> >> >> >> > >> > >> [root@cs-east-prod1 ~]# mv
>> >> >> >> > >> > >> /var/log/cloudstack/management/management-server.log
>> >> >> >> > >> > >> /var/log/cloudstack/management/management-server.bak
>> >> >> >> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management
>> >> start
>> >> >> >> > >> > >> Starting cloudstack-management:
>> >> [
>> >> >> >> OK ]
>> >> >> >> > >> > >> [root@cs-east-prod1 ~]#
>> >> >> >> > >> > >>
>> >> >> >> > >> > >> Please see attached ZIP of clean management-server.log
>> in
>> >> >> DEBUG
>> >> >> >> > mode.
>> >> >> >> > >> At
>> >> >> >> > >> > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the
>> >> attached
>> >> >> >> > >> > >> management-server.log*) I tried starting the Virtual
>> Router
>> >> >> from
>> >> >> >> > the
>> >> >> >> > >> > >> Management UI and received this "Status" message in the
>> UI.
>> >> >> >> > >> > >>
>> >> >> >> > >> > >> [image: Inline image 1]
>> >> >> >> > >> > >>
>> >> >> >> > >> > >> I can see in the management-server.log that while the
>> >> >> >> > >> > cloudstack-management
>> >> >> >> > >> > >> service is initializing, it throws the following ERROR
>> at
>> >> >> exactly
>> >> >> >> > >> > (*2014-11-03
>> >> >> >> > >> > >> 08:14:36,813 on line 342 of the attached
>> >> >> management-server.log*):
>> >> >> >> > >> > >>
>> >> >> >> > >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
>> >> >> >> > (main:null)
>> >> >> >> > >> > Unable
>> >> >> >> > >> > >> to ping management server at 10.97.38.10:9090 due to
>> >> >> >> > ConnectException
>> >> >> >> > >> > >> java.net.ConnectException: Connection refused
>> >> >> >> > >> > >> at sun.nio.ch.Net.connect0(Native Method)
>> >> >> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:465)
>> >> >> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:457)
>> >> >> >> > >> > >> at
>> >> >> >> > sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> >
>> >> >>
>> >>
>> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >>
>> >> >> >>
>> >> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
>> >> >> >> > >> > >> at
>> >> >> >> > >> >
>> >> >> >>
>> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> >
>> >> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
>> >> >> >> > >> > >> at
>> >> >> >> > >>
>> >> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
>> >> >> >> > >> > >> at
>> >> >> >> > >> >
>> >> >> >>
>> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
>> >> >> >> > >> > >> at
>> >> >> >> > >>
>> >> org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
>> >> >> >> > >> > >> at
>> >> >> >> > >> >
>> >> >> >>
>> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> >
>> >> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >>
>> >> >> >>
>> >> org.apache.catalina.core.StandardService.start(StandardService.java:516)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> >
>> >> org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
>> >> >> >> > >> > >> at
>> >> >> >> org.apache.catalina.startup.Catalina.start(Catalina.java:593)
>> >> >> >> > >> > >> at
>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> >> >> Method)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> >> >> >> > >> > >> at
>> >> >> >> > >> > >>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >> >> >> > >> > >> at java.lang.reflect.Method.invoke(Method.java:606)
>> >> >> >> > >> > >> at
>> >> >> >> > org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
>> >> >> >> > >> > >> at
>> >> >> >> > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
>> >> >> >> > >> > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl]
>> >> >> >> > (main:null)
>> >> >> >> > >> > >> Detected that another management node with the same IP
>> >> >> >> 10.97.38.10
>> >> >> >> > is
>> >> >> >> > >> > >> considered as running in DB, however it is not
>> pingable, we
>> >> >> will
>> >> >> >> > >> > continue
>> >> >> >> > >> > >> cluster initialization with this management server node
>> >> >> >> > >> > >>
>> >> >> >> > >> > >> I'm not sure what that error means, but more than likely
>> >> >> that's
>> >> >> >> the
>> >> >> >> > >> > reason
>> >> >> >> > >> > >> why the Virtual Router will not start.
>> >> >> >> > >> > >>
>> >> >> >> > >> > >> * PLEASE HELP ! *
>> >> >> >> > >> > >>
>> >> >> >> > >> > >> Best Regards,
>> >> >> >> > >> > >>
>> >> >> >> > >> > >>
>> >> >> >> > >> > >>
>> >> >> >> > >> > >> Adam Scarcella
>> >> >> >> > >> > >>
>> >> >> >> > >> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro>
>> >> wrote:
>> >> >> >> > >> > >>
>> >> >> >> > >> > >>> Hi,
>> >> >> >> > >> > >>>
>> >> >> >> > >> > >>> More agent and management logs please, make sure they
>> are
>> >> in
>> >> >> >> > DEBUG[1]
>> >> >> >> > >> > mode.
>> >> >> >> > >> > >>>
>> >> >> >> > >> > >>> [1] -
>> >> >> >> > >> > >>>
>> >> >> >> > >>
>> >> >> >>
>> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
>> >> >> >> > >> > >>>
>> >> >> >> > >> > >>> --
>> >> >> >> > >> > >>> Sent from the Delta quadrant using Borg technology!
>> >> >> >> > >> > >>>
>> >> >> >> > >> > >>> Nux!
>> >> >> >> > >> > >>> www.nux.ro
>> >> >> >> > >> > >>>
>> >> >> >> > >> > >>> ----- Original Message -----
>> >> >> >> > >> > >>> > From: "Adam" <ad...@gmail.com>
>> >> >> >> > >> > >>> > To: users@cloudstack.apache.org
>> >> >> >> > >> > >>> > Sent: Sunday, 2 November, 2014 23:33:22
>> >> >> >> > >> > >>> > Subject: How to Upgrade Virtual Router from 4.3 to
>> 4.4.1
>> >> >> >> > >> > >>>
>> >> >> >> > >> > >>> > I've performed a complete upgrade from 4.3 to 4.4.1
>> and
>> >> >> >> followed
>> >> >> >> > >> the
>> >> >> >> > >> > >>> > instructions perfectly, but my Virtual Router will
>> not
>> >> >> start,
>> >> >> >> > nor
>> >> >> >> > >> am
>> >> >> >> > >> > I
>> >> >> >> > >> > >>> able
>> >> >> >> > >> > >>> > to "upgrade" it.
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> > I tried destroying the Virtual Router, starting any
>> VM
>> >> >> >> Instance,
>> >> >> >> > >> and
>> >> >> >> > >> > >>> > allowing CS to recreate the VR. However that didn't
>> >> help.
>> >> >> The
>> >> >> >> VR
>> >> >> >> > >> > still
>> >> >> >> > >> > >>> did
>> >> >> >> > >> > >>> > not start and it was still saying that it required
>> to be
>> >> >> >> > upgraded.
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> > [image: Inline image 1]
>> >> >> >> > >> > >>> > [image: Inline image 2]
>> >> >> >> > >> > >>> > [image: Inline image 3]
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> > I know there are manual steps to finalize the
>> systemvm
>> >> >> >> upgrade,
>> >> >> >> > but
>> >> >> >> > >> > I did
>> >> >> >> > >> > >>> > those steps:
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>>
>> >> >> >> > >> >
>> >> >> >> > >>
>> >> >> >> >
>> >> >> >>
>> >> >>
>> >>
>> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root
>> -p
>> >> >> >> > >> > >>> > Enter password:
>> >> >> >> > >> > >>> > Welcome to the MySQL monitor. Commands end with ; or
>> >> \g.
>> >> >> >> > >> > >>> > Your MySQL connection id is 174
>> >> >> >> > >> > >>> > Server version: 5.1.73-log Source distribution
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> > Copyright (c) 2000, 2013, Oracle and/or its
>> affiliates.
>> >> All
>> >> >> >> > rights
>> >> >> >> > >> > >>> reserved.
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> > Oracle is a registered trademark of Oracle
>> Corporation
>> >> >> and/or
>> >> >> >> > its
>> >> >> >> > >> > >>> > affiliates. Other names may be trademarks of their
>> >> >> respective
>> >> >> >> > >> > >>> > owners.
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the
>> >> >> current
>> >> >> >> > input
>> >> >> >> > >> > >>> > statement.
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> > mysql> select id,name from cloud.vm_template where
>> name
>> >> =
>> >> >> >> > >> > >>> > 'systemvm-kvm-4.4';
>> >> >> >> > >> > >>> > +-----+------------------+
>> >> >> >> > >> > >>> >| id | name |
>> >> >> >> > >> > >>> > +-----+------------------+
>> >> >> >> > >> > >>> >| 274 | systemvm-kvm-4.4 |
>> >> >> >> > >> > >>> > +-----+------------------+
>> >> >> >> > >> > >>> > 1 row in set (0.00 sec)
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> > mysql> select type from cloud.vm_template where
>> >> id='274';
>> >> >> >> > >> > >>> > +--------+
>> >> >> >> > >> > >>> >| type |
>> >> >> >> > >> > >>> > +--------+
>> >> >> >> > >> > >>> >| SYSTEM |
>> >> >> >> > >> > >>> > +--------+
>> >> >> >> > >> > >>> > 1 row in set (0.00 sec)
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> > mysql> select distinct(vm_template_id) from
>> >> >> cloud.vm_instance
>> >> >> >> > where
>> >> >> >> > >> > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
>> >> >> >> > >> > >>> > +----------------+
>> >> >> >> > >> > >>> >| vm_template_id |
>> >> >> >> > >> > >>> > +----------------+
>> >> >> >> > >> > >>> >| 274 |
>> >> >> >> > >> > >>> > +----------------+
>> >> >> >> > >> > >>> > 1 row in set (0.00 sec)
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> > mysql> select distinct(vm_template_id) from
>> >> >> cloud.vm_instance
>> >> >> >> > where
>> >> >> >> > >> > >>> > type='SecondaryStorageVm' and hypervisor_type =
>> 'KVM';
>> >> >> >> > >> > >>> > +----------------+
>> >> >> >> > >> > >>> >| vm_template_id |
>> >> >> >> > >> > >>> > +----------------+
>> >> >> >> > >> > >>> >| 274 |
>> >> >> >> > >> > >>> > +----------------+
>> >> >> >> > >> > >>> > 1 row in set (0.03 sec)
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> > mysql> select value from cloud.configuration where
>> name
>> >> =
>> >> >> >> > >> > >>> > 'router.template.kvm';
>> >> >> >> > >> > >>> > +------------------+
>> >> >> >> > >> > >>> >| value |
>> >> >> >> > >> > >>> > +------------------+
>> >> >> >> > >> > >>> >| systemvm-kvm-4.4 |
>> >> >> >> > >> > >>> > +------------------+
>> >> >> >> > >> > >>> > 1 row in set (0.00 sec)
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> > mysql>
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> > * PLEASE HELP ! *
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > >>> > Best Regards,
>> >> >> >> > >> > >>> >
>> >> >> >> > >> > > >> > Adam Scarcella
>> >> >> >> > >> >
>> >> >> >> >
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Thanks for your support,
>> >> >> >> Motty
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Adam <ad...@gmail.com>.
ok, so I've rolled back and now none of my hosts are connecting. My system
VMs are not running, and the only thing I can see in the logs is that my
zone is not ready for either the Console Proxy or the Secondary Storage VM
to start because none of my hosts are reachable, even though in the UI it
says they're all up and running.
I'm really losing all patience and confidence with CloudStack. I currently
have a 10 person team who cannot access any of their VMs because CloudStack
can't seem to find itself.
I stopped all services and erased all 4.4.1 packages on the management
server as well as all the KVM hosts.
# STOP ALL CLOUDSTACK SERVICES ON MANAGEMENT HOST
service cloudstack-agent stop
service cloudstack-usage stop
service cloudstack-management stop
# STOP CLOUDSTACK AGENT ON KVM HOST #N
service cloudstack-agent stop
# DROP ALL DATABASES
# THIS SCRIPT DROPS ALL DATABASES IN THE CLOUDSTACK MySQL DB (cloud,
cloud_usage, cloudbridge)
/root/cronScripts/mysql_DropAllDatabases.sh
# MODIFIED/etc/yum.repos.d/cloudstack.repo to use 4.3 repository
[root@cs-east-prod1 /]# cat /etc/yum.repos.d/cloudstack.repo
[cloudstack]
name=cloudstack
baseurl=http://cloudstack.apt-get.eu/rhel/4.3/
enabled=1
gpgcheck=0
[root@cs-east-prod1 /]#
# YUM ERASE/FS CLEAN/YUM INSTALL ON MANAGEMENT HOST
yum -y erase cloudstack\*
rm -Rf /var/log/cloudstack/management
rm -Rf /usr/share/cloudstack*
rm -Rf /usr/share/doc/cloudstack*
rm -Rf /var/log/cloudstack
rm -Rf /etc/cloudstack
rm -Rf /var/cache/cloudstack
rm -Rf /var/cloudstack
yum clean all
yum -y install cloudstack\*
# YUM ERASE/FS CLEAN/YUM INSTALL ON KVM HOST(S)
yum -y erase cloudstack\*
rm -Rf /usr/share/cloudstack*
rm -Rf /usr/share/doc/cloudstack*
rm -Rf /var/log/cloudstack
rm -Rf /etc/cloudstack
yum clean all
yum -y install cloudstack-agent
# RAN cloudstack-setup-databases
cloudstack-setup-databases cloud:<PASSWORD> --deploy-as=root:<PASSWORD> -i
10.97.38.10
# RESTORED MySQL BACKUPS
mysql -u root -p<PASSWORD> cloud <
/var/www/html/mysql-backups/cloud_11-01-2014.sql
mysql -u root -p<PASSWORD> cloud <
/var/www/html/mysql-backups/cloud_usage_11-01-2014.sql
mysql -u root -p<PASSWORD> cloud <
/var/www/html/mysql-backups/cloud_meta_11-01-2014.sql
mysql -u root -p<PASSWORD> cloud <
/var/www/html/mysql-backups/cloudbridge_11-01-2014.sql
# START ALL CLOUDSTACK SERVICES ON MANAGEMENT HOST
service cloudstack-management start
service cloudstack-usage start
service cloudstack-agent start
# START ALL CLOUDSTACK SERVICES ON KVM HOST(S)
service cloudstack-agent start
Where do I go from here?
Best Regards,
Adam Scarcella
On Nov 3, 2014 2:06 PM, "Nux!" <nu...@li.nux.ro> wrote:
> Adam,
>
> I take it replacing the VR's disk did not work? Well, I'm hoping 4.4.2
> will come with a fixed template.
>
> Re downgrading this is how I did it:
> 1 - stop all services (mgmt and agents) and "yum downgrade cloudstack-\*"
> 2 - drop the DBs and import backups
> 3 - restart the downgraded services
> 4 - you'll also need to rebuild your VR, possibly "virsh destroy" the
> 4.4.1 one if it's still around
>
> Good luck
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
> > From: "Adam" <ad...@gmail.com>
> > To: users@cloudstack.apache.org
> > Sent: Monday, 3 November, 2014 18:51:13
> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>
> > I've decided to revert back to 4.3.1 since I cannot wait any loner to get
> > 4.4.1 working.
> >
> > I have a complete backup of all three databases just before I attempted
> to
> > upgrade so I'm assuming all I would have to do is remove 4.4.1, reinstall
> > 4.3.1, restore the db and then start everything back up correct?
> >
> > Are there any official steps to roll back?
> >
> > Best Regards,
> >
> >
> >
> > Adam Scarcella
> >
> > On Mon, Nov 3, 2014 at 12:13 PM, Nux! <nu...@li.nux.ro> wrote:
> >
> >> What I am saying - and excuse my poor explanations - is that the HV may
> >> have already cached the old template, so the VR still uses the old
> broken
> >> one.
> >>
> >> What I was suggesting is find out which
> /var/lib/libvirt/images/blah.qcow2
> >> the VR is using and replace it with my template.
> >>
> >> --
> >> Sent from the Delta quadrant using Borg technology!
> >>
> >> Nux!
> >> www.nux.ro
> >>
> >> ----- Original Message -----
> >> > From: "Adam" <ad...@gmail.com>
> >> > To: users@cloudstack.apache.org
> >> > Sent: Monday, 3 November, 2014 17:03:01
> >> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >>
> >> >>
> >> >> Not sure if this operation that you performed replaces the current
> >> >> systemvm template and *especially if it replaces it also on the
> >> >> hypervisor where it is running.*
> >> >>
> >> >
> >> > That procedure is the one documented in the official CloudStack
> >> > documentation so I hope it actually replaces the systemvm template.
> The
> >> > final output from that command tells it was successful:
> >> >
> >> > Successfully installed system VM template to
> >> > /export/secondary/template/tmpl/1/274/
> >> >
> >> > Also, as stated before, the VR is not running on any hypervisor / host
> >> > because I cannot get it to run, so I'm unsure what you mean when you
> >> > say "*especially
> >> > if it replaces it also on the hypervisor where it is running.*"
> >> >
> >> > One more thing I would try is to make sure the VR on the HV is also
> using
> >> >> this image. You can try to compare md5sums of the image on the
> secondary
> >> >> storage and also the backend file for the VR's disk image (qemu-img
> info
> >> >> will show it to you) on the HV.
> >> >> If the sums differ then I would try to replace the VR disk image
> with my
> >> >> template directly on the HV and see if that does it.
> >> >
> >> >
> >> > I really don't understand what you're asking me to do.
> >> >
> >> > Best Regards,
> >> >
> >> >
> >> >
> >> > Adam Scarcella
> >> >
> >> > On Mon, Nov 3, 2014 at 11:50 AM, Nux! <nu...@li.nux.ro> wrote:
> >> >
> >> >> Adam,
> >> >>
> >> >> Not sure if this operation that you performed replaces the current
> >> >> systemvm template and especially if it replaces it also on the
> >> hypervisor
> >> >> where it is running.
> >> >>
> >> >> One more thing I would try is to make sure the VR on the HV is also
> >> using
> >> >> this image. You can try to compare md5sums of the image on the
> secondary
> >> >> storage and also the backend file for the VR's disk image (qemu-img
> info
> >> >> will show it to you) on the HV.
> >> >> If the sums differ then I would try to replace the VR disk image
> with my
> >> >> template directly on the HV and see if that does it.
> >> >>
> >> >> --
> >> >> Sent from the Delta quadrant using Borg technology!
> >> >>
> >> >> Nux!
> >> >> www.nux.ro
> >> >>
> >> >> ----- Original Message -----
> >> >> > From: "Adam" <ad...@gmail.com>
> >> >> > To: users@cloudstack.apache.org
> >> >> > Sent: Monday, 3 November, 2014 16:08:05
> >> >> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >> >>
> >> >> > Thanks HTH,
> >> >> >
> >> >> > I decided to try to use your already modified qcow2 file:
> >> >> >
> >> >> > [root@cs-east-prod1 /]#
> >> >> >
> >> >>
> >>
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> >> >> > -m /export/secondary -u
> >> >> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> >> >> > -h kvm -F
> >> >> > --2014-11-03 10:11:32--
> >> >> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> >> >> > Resolving tmp.nux.ro... 217.19.15.108
> >> >> > Connecting to tmp.nux.ro|217.19.15.108|:80... connected.
> >> >> > HTTP request sent, awaiting response... 200 OK
> >> >> > Length: 305498799 (291M) [application/x-bzip2]
> >> >> > Saving to:
> >> >> >
> >> >>
> >>
> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
> >> >> >
> >> >> >
> >> >>
> >>
> 100%[===================================================================================================================>]
> >> >> > 305,498,799 9.02M/s in 37s
> >> >> >
> >> >> > 2014-11-03 10:12:10 (7.82 MB/s) -
> >> >> >
> >> >>
> >>
> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
> >> >> > saved [305498799/305498799]
> >> >> >
> >> >> > Uncompressing to
> >> >> >
> >> >>
> >>
> /usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2.tmp
> >> >> > (type bz2)...could take a long time
> >> >> > Moving to
> >> >> >
> >> >>
> >>
> /export/secondary/template/tmpl/1/274///00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2...could
> >> >> > take a while
> >> >> > Successfully installed system VM template to
> >> >> > /export/secondary/template/tmpl/1/274/
> >> >> > [root@cs-east-prod1 /]#
> >> >> >
> >> >> > Then I went into the Admin GUI and tried to start the VR, but it
> >> failed
> >> >> > with the exact same message:
> >> >> >
> >> >> > [image: Inline image 1]
> >> >> >
> >> >> > So I destroyed the Virtual Router and stopped and started the
> >> following
> >> >> > services:
> >> >> >
> >> >> > [root@cs-east-prod1 /]# service iptables stop
> >> >> > iptables: Setting chains to policy ACCEPT: nat filter [ OK ]
> >> >> > iptables: Flushing firewall rules: [ OK ]
> >> >> > iptables: Unloading modules: [ OK ]
> >> >> > [root@cs-east-prod1 /]# service cloudstack-management stop
> >> >> > Stopping cloudstack-management: [ OK ]
> >> >> > [root@cs-east-prod1 /]# service cloudstack-agent stop
> >> >> > Stopping Cloud Agent:
> >> >> > [root@cs-east-prod1 /]# service cloudstack-usage stop
> >> >> > Stopping CloudStack Usage Monitor cloudstack-usage [ OK ]
> >> >> > [root@cs-east-prod1 /]# service cloudstack-management start
> >> >> > Starting cloudstack-management: [ OK ]
> >> >> > [root@cs-east-prod1 /]# service cloudstack-usage start
> >> >> > Starting CloudStack Usage Monitor cloudstack-usage [ OK ]
> >> >> > [root@cs-east-prod1 /]# service cloudstack-agent start
> >> >> > Starting Cloud Agent:
> >> >> > [root@cs-east-prod1 /]#
> >> >> >
> >> >> > I then tried starting any VM instance to have CS automatically
> >> re-create
> >> >> > the VR, but that failed as well.
> >> >> >
> >> >> > Motty, I also looked at trying to increase the RAM and CPU for the
> >> router
> >> >> > but only saw one option
> >> >> >
> >> >> > [image: Inline image 2]
> >> >> >
> >> >> > So I created a new Service Offering with twice as much resources:
> >> >> >
> >> >> > [image: Inline image 3]
> >> >> >
> >> >> > Then upgraded the service offering for the VR
> >> >> >
> >> >> > [image: Inline image 4]
> >> >> >
> >> >> > [image: Inline image 6]
> >> >> >
> >> >> > But it failed still with the exact same error:
> >> >> >
> >> >> > [image: Inline image 7]
> >> >> >
> >> >> > Best Regards,
> >> >> >
> >> >> >
> >> >> >
> >> >> > Adam Scarcella
> >> >> >
> >> >> > On Mon, Nov 3, 2014 at 10:23 AM, motty cruz <mo...@gmail.com>
> >> >> wrote:
> >> >> >
> >> >> >> Adam,
> >> >> >> what are the specifications of the VR? can you upgrade the RAM and
> >> CPU
> >> >> and
> >> >> >> try to restart that VR? I had similar issue and giving the VR more
> >> RAM
> >> >> and
> >> >> >> CPU will start the VR.
> >> >> >>
> >> >> >> Thanks,
> >> >> >> Motty
> >> >> >>
> >> >> >> On Mon, Nov 3, 2014 at 7:04 AM, Nux! <nu...@li.nux.ro> wrote:
> >> >> >>
> >> >> >> > Adam,
> >> >> >> >
> >> >> >> > You can modify the qcow2 file that is in your secondary storage,
> >> i.e.
> >> >> you
> >> >> >> > should mount that storage somewhere and go to
> >> >> >> > /mount/point/template/tmpl/274 and modify the qcow2 file.
> >> >> >> >
> >> >> >> > Alternatively you could use my already modified copy to
> overwrite
> >> your
> >> >> >> > uuid.qcow2 with (at your own risk of course):
> >> >> >> > http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> >> >> >> >
> >> >> >> > HTH
> >> >> >> >
> >> >> >> > --
> >> >> >> > Sent from the Delta quadrant using Borg technology!
> >> >> >> >
> >> >> >> > Nux!
> >> >> >> > www.nux.ro
> >> >> >> >
> >> >> >> > ----- Original Message -----
> >> >> >> > > From: "Adam" <ad...@gmail.com>
> >> >> >> > > To: users@cloudstack.apache.org
> >> >> >> > > Sent: Monday, 3 November, 2014 14:58:58
> >> >> >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >> >> >> >
> >> >> >> > > [root@cs-east-prod1 management]# netstat -a|grep 9090
> >> >> >> > > unix 3 [ ] STREAM CONNECTED 29090
> >> >> >> > > [root@cs-east-prod1 management]# nmap localhost
> >> >> >> > >
> >> >> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47
> EST
> >> >> >> > > Nmap scan report for localhost (127.0.0.1)
> >> >> >> > > Host is up (0.000015s latency).
> >> >> >> > > Other addresses for localhost (not scanned): 127.0.0.1
> >> >> >> > > Not shown: 990 closed ports
> >> >> >> > > PORT STATE SERVICE
> >> >> >> > > 22/tcp open ssh
> >> >> >> > > 25/tcp open smtp
> >> >> >> > > 80/tcp open http
> >> >> >> > > 111/tcp open rpcbind
> >> >> >> > > 199/tcp open smux
> >> >> >> > > 631/tcp open ipp
> >> >> >> > > 2049/tcp open nfs
> >> >> >> > > 3306/tcp open mysql
> >> >> >> > > 8080/tcp open http-proxy
> >> >> >> > > 9090/tcp open zeus-admin
> >> >> >> > >
> >> >> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
> >> >> >> > > [root@cs-east-prod1 management]# telnet localhost 9090
> >> >> >> > > Trying ::1...
> >> >> >> > > Connected to localhost.
> >> >> >> > > Escape character is '^]'.
> >> >> >> > > Connection closed by foreign host.
> >> >> >> > > [root@cs-east-prod1 management]# nmap 10.97.38.10
> >> >> >> > >
> >> >> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49
> EST
> >> >> >> > > Nmap scan report for cs-east-prod1.na.tibco.com (10.97.38.10)
> >> >> >> > > Host is up (0.000015s latency).
> >> >> >> > > Not shown: 993 closed ports
> >> >> >> > > PORT STATE SERVICE
> >> >> >> > > 22/tcp open ssh
> >> >> >> > > 80/tcp open http
> >> >> >> > > 111/tcp open rpcbind
> >> >> >> > > 2049/tcp open nfs
> >> >> >> > > 3306/tcp open mysql
> >> >> >> > > 8080/tcp open http-proxy
> >> >> >> > > 9090/tcp open zeus-admin
> >> >> >> > >
> >> >> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
> >> >> >> > > [root@cs-east-prod1 management]# telnet 10.97.38.10 9090
> >> >> >> > > Trying 10.97.38.10...
> >> >> >> > > Connected to 10.97.38.10.
> >> >> >> > > Escape character is '^]'.
> >> >> >> > > Connection closed by foreign host.
> >> >> >> > > [root@cs-east-prod1 management]#
> >> >> >> > >
> >> >> >> > > Also, I checked for the /opt/cloud/bin/ directory on my
> >> Management
> >> >> >> Host,
> >> >> >> > > but it's not there:
> >> >> >> > >
> >> >> >> > > [root@cs-east-prod1 management]# cd /opt/cloud/bin/
> >> >> >> > > -bash: cd: /opt/cloud/bin/: No such file or directory
> >> >> >> > > [root@cs-east-prod1 management]# cd /
> >> >> >> > > [root@cs-east-prod1 /]# find . -name "createipAlias.sh"
> >> >> >> > > [root@cs-east-prod1 /]# find . -name "createIpAlias.sh"
> >> >> >> > > [root@cs-east-prod1 /]#
> >> >> >> > >
> >> >> >> > > Are you saying that the /opt/cloud/bin/createipAlias.sh
> script is
> >> >> >> inside
> >> >> >> > > the Virtual Router VM? If so, how could I possibly change that
> >> >> since it
> >> >> >> > > won't start?
> >> >> >> > >
> >> >> >> > > Best Regards,
> >> >> >> > >
> >> >> >> > >
> >> >> >> > >
> >> >> >> > > Adam Scarcella
> >> >> >> > >
> >> >> >> > > On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion <
> >> >> pdion891@apache.org>
> >> >> >> > wrote:
> >> >> >> > >
> >> >> >> > >> Could it be possible the firewall is blocking port 9090 from
> the
> >> >> >> > >> management-server?
> >> >> >> > >>
> >> >> >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
> >> >> (main:null)
> >> >> >> > Unable
> >> >> >> > >> to ping management server at 10.97.38.10:9090 due to
> >> >> ConnectException
> >> >> >> > >> java.net.ConnectException: Connection refused
> >> >> >> > >>
> >> >> >> > >> On Mon, Nov 3, 2014 at 9:02 AM, Nux! <nu...@li.nux.ro> wrote:
> >> >> >> > >>
> >> >> >> > >> > BTW the agent logs should help clarify this.
> >> >> >> > >> > The aforementioned changes must be done either in the
> current
> >> >> >> > (broken) VR
> >> >> >> > >> > or in the systemvm (after which you should redeploy the VR
> to
> >> >> pick
> >> >> >> up
> >> >> >> > the
> >> >> >> > >> > change).
> >> >> >> > >> >
> >> >> >> > >> > HTH
> >> >> >> > >> > Lucian
> >> >> >> > >> >
> >> >> >> > >> > --
> >> >> >> > >> > Sent from the Delta quadrant using Borg technology!
> >> >> >> > >> >
> >> >> >> > >> > Nux!
> >> >> >> > >> > www.nux.ro
> >> >> >> > >> >
> >> >> >> > >> > ----- Original Message -----
> >> >> >> > >> > > From: "Nux!" <nu...@li.nux.ro>
> >> >> >> > >> > > To: users@cloudstack.apache.org
> >> >> >> > >> > > Sent: Monday, 3 November, 2014 13:53:10
> >> >> >> > >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to
> 4.4.1
> >> >> >> > >> >
> >> >> >> > >> > > Hello,
> >> >> >> > >> > >
> >> >> >> > >> > > I think you are hitting a problem I also had:
> >> >> >> > >> > > "2014-11-03 08:16:17,024 ERROR
> >> >> [c.c.v.VirtualMachineManagerImpl]
> >> >> >> > >> > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556
> >> >> >> ctx-fcb072b2)
> >> >> >> > >> > Failed to
> >> >> >> > >> > > start instance VM[DomainRouter|r-4758-VM]
> >> >> >> > >> > > com.cloud.utils.exception.ExecutionException: Unable to
> >> start
> >> >> >> > >> > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart,
> >> not
> >> >> >> > retrying"
> >> >> >> > >> > >
> >> >> >> > >> > > It's
> https://issues.apache.org/jira/browse/CLOUDSTACK-7781
> >> >> >> > >> > >
> >> >> >> > >> > > Basically the systemvm template has some problems, you
> need
> >> to
> >> >> >> > somehow
> >> >> >> > >> > copy
> >> >> >> > >> > > /opt/cloud/bin/createIpAlias.sh to
> >> >> /opt/cloud/bin/createipAlias.sh
> >> >> >> > to
> >> >> >> > >> > get it
> >> >> >> > >> > > working.
> >> >> >> > >> > >
> >> >> >> > >> > > --
> >> >> >> > >> > > Sent from the Delta quadrant using Borg technology!
> >> >> >> > >> > >
> >> >> >> > >> > > Nux!
> >> >> >> > >> > > www.nux.ro
> >> >> >> > >> > >
> >> >> >> > >> > > ----- Original Message -----
> >> >> >> > >> > >> From: "Adam" <ad...@gmail.com>
> >> >> >> > >> > >> To: users@cloudstack.apache.org
> >> >> >> > >> > >> Sent: Monday, 3 November, 2014 13:32:12
> >> >> >> > >> > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to
> >> 4.4.1
> >> >> >> > >> > >
> >> >> >> > >> > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release
> >> >> >> > >> > >> CentOS release 6.6 (Final)
> >> >> >> > >> > >> [root@cs-east-prod1 ~]# uname -a
> >> >> >> > >> > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed
> Oct 15
> >> >> >> > 04:27:16
> >> >> >> > >> UTC
> >> >> >> > >> > >> 2014 x86_64 x86_64 x86_64 GNU/Linux
> >> >> >> > >> > >> [root@cs-east-prod1 ~]# virsh --version
> >> >> >> > >> > >> 0.10.2
> >> >> >> > >> > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
> >> >> >> > >> > >> /etc/cloudstack/agent/log4j-cloud.xml
> >> >> >> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management
> stop
> >> >> >> > >> > >> Stopping cloudstack-management:
> >> [
> >> >> >> OK ]
> >> >> >> > >> > >> [root@cs-east-prod1 ~]# mv
> >> >> >> > >> > >> /var/log/cloudstack/management/management-server.log
> >> >> >> > >> > >> /var/log/cloudstack/management/management-server.bak
> >> >> >> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management
> >> start
> >> >> >> > >> > >> Starting cloudstack-management:
> >> [
> >> >> >> OK ]
> >> >> >> > >> > >> [root@cs-east-prod1 ~]#
> >> >> >> > >> > >>
> >> >> >> > >> > >> Please see attached ZIP of clean management-server.log
> in
> >> >> DEBUG
> >> >> >> > mode.
> >> >> >> > >> At
> >> >> >> > >> > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the
> >> attached
> >> >> >> > >> > >> management-server.log*) I tried starting the Virtual
> Router
> >> >> from
> >> >> >> > the
> >> >> >> > >> > >> Management UI and received this "Status" message in the
> UI.
> >> >> >> > >> > >>
> >> >> >> > >> > >> [image: Inline image 1]
> >> >> >> > >> > >>
> >> >> >> > >> > >> I can see in the management-server.log that while the
> >> >> >> > >> > cloudstack-management
> >> >> >> > >> > >> service is initializing, it throws the following ERROR
> at
> >> >> exactly
> >> >> >> > >> > (*2014-11-03
> >> >> >> > >> > >> 08:14:36,813 on line 342 of the attached
> >> >> management-server.log*):
> >> >> >> > >> > >>
> >> >> >> > >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
> >> >> >> > (main:null)
> >> >> >> > >> > Unable
> >> >> >> > >> > >> to ping management server at 10.97.38.10:9090 due to
> >> >> >> > ConnectException
> >> >> >> > >> > >> java.net.ConnectException: Connection refused
> >> >> >> > >> > >> at sun.nio.ch.Net.connect0(Native Method)
> >> >> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:465)
> >> >> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:457)
> >> >> >> > >> > >> at
> >> >> >> > sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> >
> >> >>
> >>
> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >>
> >> >> >>
> >> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
> >> >> >> > >> > >> at
> >> >> >> > >> >
> >> >> >>
> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> >
> >> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
> >> >> >> > >> > >> at
> >> >> >> > >>
> >> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> >> >> >> > >> > >> at
> >> >> >> > >> >
> >> >> >>
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
> >> >> >> > >> > >> at
> >> >> >> > >>
> >> org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
> >> >> >> > >> > >> at
> >> >> >> > >> >
> >> >> >>
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> >
> >> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >>
> >> >> >>
> >> org.apache.catalina.core.StandardService.start(StandardService.java:516)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> >
> >> org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
> >> >> >> > >> > >> at
> >> >> >> org.apache.catalina.startup.Catalina.start(Catalina.java:593)
> >> >> >> > >> > >> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> >> >> Method)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> >> >> > >> > >> at
> >> >> >> > >> > >>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> >> >> > >> > >> at java.lang.reflect.Method.invoke(Method.java:606)
> >> >> >> > >> > >> at
> >> >> >> > org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> >> >> >> > >> > >> at
> >> >> >> > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> >> >> >> > >> > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl]
> >> >> >> > (main:null)
> >> >> >> > >> > >> Detected that another management node with the same IP
> >> >> >> 10.97.38.10
> >> >> >> > is
> >> >> >> > >> > >> considered as running in DB, however it is not
> pingable, we
> >> >> will
> >> >> >> > >> > continue
> >> >> >> > >> > >> cluster initialization with this management server node
> >> >> >> > >> > >>
> >> >> >> > >> > >> I'm not sure what that error means, but more than likely
> >> >> that's
> >> >> >> the
> >> >> >> > >> > reason
> >> >> >> > >> > >> why the Virtual Router will not start.
> >> >> >> > >> > >>
> >> >> >> > >> > >> * PLEASE HELP ! *
> >> >> >> > >> > >>
> >> >> >> > >> > >> Best Regards,
> >> >> >> > >> > >>
> >> >> >> > >> > >>
> >> >> >> > >> > >>
> >> >> >> > >> > >> Adam Scarcella
> >> >> >> > >> > >>
> >> >> >> > >> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro>
> >> wrote:
> >> >> >> > >> > >>
> >> >> >> > >> > >>> Hi,
> >> >> >> > >> > >>>
> >> >> >> > >> > >>> More agent and management logs please, make sure they
> are
> >> in
> >> >> >> > DEBUG[1]
> >> >> >> > >> > mode.
> >> >> >> > >> > >>>
> >> >> >> > >> > >>> [1] -
> >> >> >> > >> > >>>
> >> >> >> > >>
> >> >> >>
> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
> >> >> >> > >> > >>>
> >> >> >> > >> > >>> --
> >> >> >> > >> > >>> Sent from the Delta quadrant using Borg technology!
> >> >> >> > >> > >>>
> >> >> >> > >> > >>> Nux!
> >> >> >> > >> > >>> www.nux.ro
> >> >> >> > >> > >>>
> >> >> >> > >> > >>> ----- Original Message -----
> >> >> >> > >> > >>> > From: "Adam" <ad...@gmail.com>
> >> >> >> > >> > >>> > To: users@cloudstack.apache.org
> >> >> >> > >> > >>> > Sent: Sunday, 2 November, 2014 23:33:22
> >> >> >> > >> > >>> > Subject: How to Upgrade Virtual Router from 4.3 to
> 4.4.1
> >> >> >> > >> > >>>
> >> >> >> > >> > >>> > I've performed a complete upgrade from 4.3 to 4.4.1
> and
> >> >> >> followed
> >> >> >> > >> the
> >> >> >> > >> > >>> > instructions perfectly, but my Virtual Router will
> not
> >> >> start,
> >> >> >> > nor
> >> >> >> > >> am
> >> >> >> > >> > I
> >> >> >> > >> > >>> able
> >> >> >> > >> > >>> > to "upgrade" it.
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> > I tried destroying the Virtual Router, starting any
> VM
> >> >> >> Instance,
> >> >> >> > >> and
> >> >> >> > >> > >>> > allowing CS to recreate the VR. However that didn't
> >> help.
> >> >> The
> >> >> >> VR
> >> >> >> > >> > still
> >> >> >> > >> > >>> did
> >> >> >> > >> > >>> > not start and it was still saying that it required
> to be
> >> >> >> > upgraded.
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> > [image: Inline image 1]
> >> >> >> > >> > >>> > [image: Inline image 2]
> >> >> >> > >> > >>> > [image: Inline image 3]
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> > I know there are manual steps to finalize the
> systemvm
> >> >> >> upgrade,
> >> >> >> > but
> >> >> >> > >> > I did
> >> >> >> > >> > >>> > those steps:
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>>
> >> >> >> > >> >
> >> >> >> > >>
> >> >> >> >
> >> >> >>
> >> >>
> >>
> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root
> -p
> >> >> >> > >> > >>> > Enter password:
> >> >> >> > >> > >>> > Welcome to the MySQL monitor. Commands end with ; or
> >> \g.
> >> >> >> > >> > >>> > Your MySQL connection id is 174
> >> >> >> > >> > >>> > Server version: 5.1.73-log Source distribution
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> > Copyright (c) 2000, 2013, Oracle and/or its
> affiliates.
> >> All
> >> >> >> > rights
> >> >> >> > >> > >>> reserved.
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> > Oracle is a registered trademark of Oracle
> Corporation
> >> >> and/or
> >> >> >> > its
> >> >> >> > >> > >>> > affiliates. Other names may be trademarks of their
> >> >> respective
> >> >> >> > >> > >>> > owners.
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the
> >> >> current
> >> >> >> > input
> >> >> >> > >> > >>> > statement.
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> > mysql> select id,name from cloud.vm_template where
> name
> >> =
> >> >> >> > >> > >>> > 'systemvm-kvm-4.4';
> >> >> >> > >> > >>> > +-----+------------------+
> >> >> >> > >> > >>> >| id | name |
> >> >> >> > >> > >>> > +-----+------------------+
> >> >> >> > >> > >>> >| 274 | systemvm-kvm-4.4 |
> >> >> >> > >> > >>> > +-----+------------------+
> >> >> >> > >> > >>> > 1 row in set (0.00 sec)
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> > mysql> select type from cloud.vm_template where
> >> id='274';
> >> >> >> > >> > >>> > +--------+
> >> >> >> > >> > >>> >| type |
> >> >> >> > >> > >>> > +--------+
> >> >> >> > >> > >>> >| SYSTEM |
> >> >> >> > >> > >>> > +--------+
> >> >> >> > >> > >>> > 1 row in set (0.00 sec)
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> > mysql> select distinct(vm_template_id) from
> >> >> cloud.vm_instance
> >> >> >> > where
> >> >> >> > >> > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
> >> >> >> > >> > >>> > +----------------+
> >> >> >> > >> > >>> >| vm_template_id |
> >> >> >> > >> > >>> > +----------------+
> >> >> >> > >> > >>> >| 274 |
> >> >> >> > >> > >>> > +----------------+
> >> >> >> > >> > >>> > 1 row in set (0.00 sec)
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> > mysql> select distinct(vm_template_id) from
> >> >> cloud.vm_instance
> >> >> >> > where
> >> >> >> > >> > >>> > type='SecondaryStorageVm' and hypervisor_type =
> 'KVM';
> >> >> >> > >> > >>> > +----------------+
> >> >> >> > >> > >>> >| vm_template_id |
> >> >> >> > >> > >>> > +----------------+
> >> >> >> > >> > >>> >| 274 |
> >> >> >> > >> > >>> > +----------------+
> >> >> >> > >> > >>> > 1 row in set (0.03 sec)
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> > mysql> select value from cloud.configuration where
> name
> >> =
> >> >> >> > >> > >>> > 'router.template.kvm';
> >> >> >> > >> > >>> > +------------------+
> >> >> >> > >> > >>> >| value |
> >> >> >> > >> > >>> > +------------------+
> >> >> >> > >> > >>> >| systemvm-kvm-4.4 |
> >> >> >> > >> > >>> > +------------------+
> >> >> >> > >> > >>> > 1 row in set (0.00 sec)
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> > mysql>
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> > * PLEASE HELP ! *
> >> >> >> > >> > >>> >
> >> >> >> > >> > >>> > Best Regards,
> >> >> >> > >> > >>> >
> >> >> >> > >> > > >> > Adam Scarcella
> >> >> >> > >> >
> >> >> >> >
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Thanks for your support,
> >> >> >> Motty
>
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Nux! <nu...@li.nux.ro>.
Adam,
I take it replacing the VR's disk did not work? Well, I'm hoping 4.4.2 will come with a fixed template.
Re downgrading this is how I did it:
1 - stop all services (mgmt and agents) and "yum downgrade cloudstack-\*"
2 - drop the DBs and import backups
3 - restart the downgraded services
4 - you'll also need to rebuild your VR, possibly "virsh destroy" the 4.4.1 one if it's still around
Good luck
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Adam" <ad...@gmail.com>
> To: users@cloudstack.apache.org
> Sent: Monday, 3 November, 2014 18:51:13
> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> I've decided to revert back to 4.3.1 since I cannot wait any loner to get
> 4.4.1 working.
>
> I have a complete backup of all three databases just before I attempted to
> upgrade so I'm assuming all I would have to do is remove 4.4.1, reinstall
> 4.3.1, restore the db and then start everything back up correct?
>
> Are there any official steps to roll back?
>
> Best Regards,
>
>
>
> Adam Scarcella
>
> On Mon, Nov 3, 2014 at 12:13 PM, Nux! <nu...@li.nux.ro> wrote:
>
>> What I am saying - and excuse my poor explanations - is that the HV may
>> have already cached the old template, so the VR still uses the old broken
>> one.
>>
>> What I was suggesting is find out which /var/lib/libvirt/images/blah.qcow2
>> the VR is using and replace it with my template.
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
>>
>> ----- Original Message -----
>> > From: "Adam" <ad...@gmail.com>
>> > To: users@cloudstack.apache.org
>> > Sent: Monday, 3 November, 2014 17:03:01
>> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>>
>> >>
>> >> Not sure if this operation that you performed replaces the current
>> >> systemvm template and *especially if it replaces it also on the
>> >> hypervisor where it is running.*
>> >>
>> >
>> > That procedure is the one documented in the official CloudStack
>> > documentation so I hope it actually replaces the systemvm template. The
>> > final output from that command tells it was successful:
>> >
>> > Successfully installed system VM template to
>> > /export/secondary/template/tmpl/1/274/
>> >
>> > Also, as stated before, the VR is not running on any hypervisor / host
>> > because I cannot get it to run, so I'm unsure what you mean when you
>> > say "*especially
>> > if it replaces it also on the hypervisor where it is running.*"
>> >
>> > One more thing I would try is to make sure the VR on the HV is also using
>> >> this image. You can try to compare md5sums of the image on the secondary
>> >> storage and also the backend file for the VR's disk image (qemu-img info
>> >> will show it to you) on the HV.
>> >> If the sums differ then I would try to replace the VR disk image with my
>> >> template directly on the HV and see if that does it.
>> >
>> >
>> > I really don't understand what you're asking me to do.
>> >
>> > Best Regards,
>> >
>> >
>> >
>> > Adam Scarcella
>> >
>> > On Mon, Nov 3, 2014 at 11:50 AM, Nux! <nu...@li.nux.ro> wrote:
>> >
>> >> Adam,
>> >>
>> >> Not sure if this operation that you performed replaces the current
>> >> systemvm template and especially if it replaces it also on the
>> hypervisor
>> >> where it is running.
>> >>
>> >> One more thing I would try is to make sure the VR on the HV is also
>> using
>> >> this image. You can try to compare md5sums of the image on the secondary
>> >> storage and also the backend file for the VR's disk image (qemu-img info
>> >> will show it to you) on the HV.
>> >> If the sums differ then I would try to replace the VR disk image with my
>> >> template directly on the HV and see if that does it.
>> >>
>> >> --
>> >> Sent from the Delta quadrant using Borg technology!
>> >>
>> >> Nux!
>> >> www.nux.ro
>> >>
>> >> ----- Original Message -----
>> >> > From: "Adam" <ad...@gmail.com>
>> >> > To: users@cloudstack.apache.org
>> >> > Sent: Monday, 3 November, 2014 16:08:05
>> >> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> >>
>> >> > Thanks HTH,
>> >> >
>> >> > I decided to try to use your already modified qcow2 file:
>> >> >
>> >> > [root@cs-east-prod1 /]#
>> >> >
>> >>
>> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
>> >> > -m /export/secondary -u
>> >> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
>> >> > -h kvm -F
>> >> > --2014-11-03 10:11:32--
>> >> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
>> >> > Resolving tmp.nux.ro... 217.19.15.108
>> >> > Connecting to tmp.nux.ro|217.19.15.108|:80... connected.
>> >> > HTTP request sent, awaiting response... 200 OK
>> >> > Length: 305498799 (291M) [application/x-bzip2]
>> >> > Saving to:
>> >> >
>> >>
>> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
>> >> >
>> >> >
>> >>
>> 100%[===================================================================================================================>]
>> >> > 305,498,799 9.02M/s in 37s
>> >> >
>> >> > 2014-11-03 10:12:10 (7.82 MB/s) -
>> >> >
>> >>
>> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
>> >> > saved [305498799/305498799]
>> >> >
>> >> > Uncompressing to
>> >> >
>> >>
>> /usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2.tmp
>> >> > (type bz2)...could take a long time
>> >> > Moving to
>> >> >
>> >>
>> /export/secondary/template/tmpl/1/274///00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2...could
>> >> > take a while
>> >> > Successfully installed system VM template to
>> >> > /export/secondary/template/tmpl/1/274/
>> >> > [root@cs-east-prod1 /]#
>> >> >
>> >> > Then I went into the Admin GUI and tried to start the VR, but it
>> failed
>> >> > with the exact same message:
>> >> >
>> >> > [image: Inline image 1]
>> >> >
>> >> > So I destroyed the Virtual Router and stopped and started the
>> following
>> >> > services:
>> >> >
>> >> > [root@cs-east-prod1 /]# service iptables stop
>> >> > iptables: Setting chains to policy ACCEPT: nat filter [ OK ]
>> >> > iptables: Flushing firewall rules: [ OK ]
>> >> > iptables: Unloading modules: [ OK ]
>> >> > [root@cs-east-prod1 /]# service cloudstack-management stop
>> >> > Stopping cloudstack-management: [ OK ]
>> >> > [root@cs-east-prod1 /]# service cloudstack-agent stop
>> >> > Stopping Cloud Agent:
>> >> > [root@cs-east-prod1 /]# service cloudstack-usage stop
>> >> > Stopping CloudStack Usage Monitor cloudstack-usage [ OK ]
>> >> > [root@cs-east-prod1 /]# service cloudstack-management start
>> >> > Starting cloudstack-management: [ OK ]
>> >> > [root@cs-east-prod1 /]# service cloudstack-usage start
>> >> > Starting CloudStack Usage Monitor cloudstack-usage [ OK ]
>> >> > [root@cs-east-prod1 /]# service cloudstack-agent start
>> >> > Starting Cloud Agent:
>> >> > [root@cs-east-prod1 /]#
>> >> >
>> >> > I then tried starting any VM instance to have CS automatically
>> re-create
>> >> > the VR, but that failed as well.
>> >> >
>> >> > Motty, I also looked at trying to increase the RAM and CPU for the
>> router
>> >> > but only saw one option
>> >> >
>> >> > [image: Inline image 2]
>> >> >
>> >> > So I created a new Service Offering with twice as much resources:
>> >> >
>> >> > [image: Inline image 3]
>> >> >
>> >> > Then upgraded the service offering for the VR
>> >> >
>> >> > [image: Inline image 4]
>> >> >
>> >> > [image: Inline image 6]
>> >> >
>> >> > But it failed still with the exact same error:
>> >> >
>> >> > [image: Inline image 7]
>> >> >
>> >> > Best Regards,
>> >> >
>> >> >
>> >> >
>> >> > Adam Scarcella
>> >> >
>> >> > On Mon, Nov 3, 2014 at 10:23 AM, motty cruz <mo...@gmail.com>
>> >> wrote:
>> >> >
>> >> >> Adam,
>> >> >> what are the specifications of the VR? can you upgrade the RAM and
>> CPU
>> >> and
>> >> >> try to restart that VR? I had similar issue and giving the VR more
>> RAM
>> >> and
>> >> >> CPU will start the VR.
>> >> >>
>> >> >> Thanks,
>> >> >> Motty
>> >> >>
>> >> >> On Mon, Nov 3, 2014 at 7:04 AM, Nux! <nu...@li.nux.ro> wrote:
>> >> >>
>> >> >> > Adam,
>> >> >> >
>> >> >> > You can modify the qcow2 file that is in your secondary storage,
>> i.e.
>> >> you
>> >> >> > should mount that storage somewhere and go to
>> >> >> > /mount/point/template/tmpl/274 and modify the qcow2 file.
>> >> >> >
>> >> >> > Alternatively you could use my already modified copy to overwrite
>> your
>> >> >> > uuid.qcow2 with (at your own risk of course):
>> >> >> > http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
>> >> >> >
>> >> >> > HTH
>> >> >> >
>> >> >> > --
>> >> >> > Sent from the Delta quadrant using Borg technology!
>> >> >> >
>> >> >> > Nux!
>> >> >> > www.nux.ro
>> >> >> >
>> >> >> > ----- Original Message -----
>> >> >> > > From: "Adam" <ad...@gmail.com>
>> >> >> > > To: users@cloudstack.apache.org
>> >> >> > > Sent: Monday, 3 November, 2014 14:58:58
>> >> >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> >> >> >
>> >> >> > > [root@cs-east-prod1 management]# netstat -a|grep 9090
>> >> >> > > unix 3 [ ] STREAM CONNECTED 29090
>> >> >> > > [root@cs-east-prod1 management]# nmap localhost
>> >> >> > >
>> >> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47 EST
>> >> >> > > Nmap scan report for localhost (127.0.0.1)
>> >> >> > > Host is up (0.000015s latency).
>> >> >> > > Other addresses for localhost (not scanned): 127.0.0.1
>> >> >> > > Not shown: 990 closed ports
>> >> >> > > PORT STATE SERVICE
>> >> >> > > 22/tcp open ssh
>> >> >> > > 25/tcp open smtp
>> >> >> > > 80/tcp open http
>> >> >> > > 111/tcp open rpcbind
>> >> >> > > 199/tcp open smux
>> >> >> > > 631/tcp open ipp
>> >> >> > > 2049/tcp open nfs
>> >> >> > > 3306/tcp open mysql
>> >> >> > > 8080/tcp open http-proxy
>> >> >> > > 9090/tcp open zeus-admin
>> >> >> > >
>> >> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
>> >> >> > > [root@cs-east-prod1 management]# telnet localhost 9090
>> >> >> > > Trying ::1...
>> >> >> > > Connected to localhost.
>> >> >> > > Escape character is '^]'.
>> >> >> > > Connection closed by foreign host.
>> >> >> > > [root@cs-east-prod1 management]# nmap 10.97.38.10
>> >> >> > >
>> >> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49 EST
>> >> >> > > Nmap scan report for cs-east-prod1.na.tibco.com (10.97.38.10)
>> >> >> > > Host is up (0.000015s latency).
>> >> >> > > Not shown: 993 closed ports
>> >> >> > > PORT STATE SERVICE
>> >> >> > > 22/tcp open ssh
>> >> >> > > 80/tcp open http
>> >> >> > > 111/tcp open rpcbind
>> >> >> > > 2049/tcp open nfs
>> >> >> > > 3306/tcp open mysql
>> >> >> > > 8080/tcp open http-proxy
>> >> >> > > 9090/tcp open zeus-admin
>> >> >> > >
>> >> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
>> >> >> > > [root@cs-east-prod1 management]# telnet 10.97.38.10 9090
>> >> >> > > Trying 10.97.38.10...
>> >> >> > > Connected to 10.97.38.10.
>> >> >> > > Escape character is '^]'.
>> >> >> > > Connection closed by foreign host.
>> >> >> > > [root@cs-east-prod1 management]#
>> >> >> > >
>> >> >> > > Also, I checked for the /opt/cloud/bin/ directory on my
>> Management
>> >> >> Host,
>> >> >> > > but it's not there:
>> >> >> > >
>> >> >> > > [root@cs-east-prod1 management]# cd /opt/cloud/bin/
>> >> >> > > -bash: cd: /opt/cloud/bin/: No such file or directory
>> >> >> > > [root@cs-east-prod1 management]# cd /
>> >> >> > > [root@cs-east-prod1 /]# find . -name "createipAlias.sh"
>> >> >> > > [root@cs-east-prod1 /]# find . -name "createIpAlias.sh"
>> >> >> > > [root@cs-east-prod1 /]#
>> >> >> > >
>> >> >> > > Are you saying that the /opt/cloud/bin/createipAlias.sh script is
>> >> >> inside
>> >> >> > > the Virtual Router VM? If so, how could I possibly change that
>> >> since it
>> >> >> > > won't start?
>> >> >> > >
>> >> >> > > Best Regards,
>> >> >> > >
>> >> >> > >
>> >> >> > >
>> >> >> > > Adam Scarcella
>> >> >> > >
>> >> >> > > On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion <
>> >> pdion891@apache.org>
>> >> >> > wrote:
>> >> >> > >
>> >> >> > >> Could it be possible the firewall is blocking port 9090 from the
>> >> >> > >> management-server?
>> >> >> > >>
>> >> >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
>> >> (main:null)
>> >> >> > Unable
>> >> >> > >> to ping management server at 10.97.38.10:9090 due to
>> >> ConnectException
>> >> >> > >> java.net.ConnectException: Connection refused
>> >> >> > >>
>> >> >> > >> On Mon, Nov 3, 2014 at 9:02 AM, Nux! <nu...@li.nux.ro> wrote:
>> >> >> > >>
>> >> >> > >> > BTW the agent logs should help clarify this.
>> >> >> > >> > The aforementioned changes must be done either in the current
>> >> >> > (broken) VR
>> >> >> > >> > or in the systemvm (after which you should redeploy the VR to
>> >> pick
>> >> >> up
>> >> >> > the
>> >> >> > >> > change).
>> >> >> > >> >
>> >> >> > >> > HTH
>> >> >> > >> > Lucian
>> >> >> > >> >
>> >> >> > >> > --
>> >> >> > >> > Sent from the Delta quadrant using Borg technology!
>> >> >> > >> >
>> >> >> > >> > Nux!
>> >> >> > >> > www.nux.ro
>> >> >> > >> >
>> >> >> > >> > ----- Original Message -----
>> >> >> > >> > > From: "Nux!" <nu...@li.nux.ro>
>> >> >> > >> > > To: users@cloudstack.apache.org
>> >> >> > >> > > Sent: Monday, 3 November, 2014 13:53:10
>> >> >> > >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> >> >> > >> >
>> >> >> > >> > > Hello,
>> >> >> > >> > >
>> >> >> > >> > > I think you are hitting a problem I also had:
>> >> >> > >> > > "2014-11-03 08:16:17,024 ERROR
>> >> [c.c.v.VirtualMachineManagerImpl]
>> >> >> > >> > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556
>> >> >> ctx-fcb072b2)
>> >> >> > >> > Failed to
>> >> >> > >> > > start instance VM[DomainRouter|r-4758-VM]
>> >> >> > >> > > com.cloud.utils.exception.ExecutionException: Unable to
>> start
>> >> >> > >> > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart,
>> not
>> >> >> > retrying"
>> >> >> > >> > >
>> >> >> > >> > > It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781
>> >> >> > >> > >
>> >> >> > >> > > Basically the systemvm template has some problems, you need
>> to
>> >> >> > somehow
>> >> >> > >> > copy
>> >> >> > >> > > /opt/cloud/bin/createIpAlias.sh to
>> >> /opt/cloud/bin/createipAlias.sh
>> >> >> > to
>> >> >> > >> > get it
>> >> >> > >> > > working.
>> >> >> > >> > >
>> >> >> > >> > > --
>> >> >> > >> > > Sent from the Delta quadrant using Borg technology!
>> >> >> > >> > >
>> >> >> > >> > > Nux!
>> >> >> > >> > > www.nux.ro
>> >> >> > >> > >
>> >> >> > >> > > ----- Original Message -----
>> >> >> > >> > >> From: "Adam" <ad...@gmail.com>
>> >> >> > >> > >> To: users@cloudstack.apache.org
>> >> >> > >> > >> Sent: Monday, 3 November, 2014 13:32:12
>> >> >> > >> > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to
>> 4.4.1
>> >> >> > >> > >
>> >> >> > >> > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release
>> >> >> > >> > >> CentOS release 6.6 (Final)
>> >> >> > >> > >> [root@cs-east-prod1 ~]# uname -a
>> >> >> > >> > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15
>> >> >> > 04:27:16
>> >> >> > >> UTC
>> >> >> > >> > >> 2014 x86_64 x86_64 x86_64 GNU/Linux
>> >> >> > >> > >> [root@cs-east-prod1 ~]# virsh --version
>> >> >> > >> > >> 0.10.2
>> >> >> > >> > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
>> >> >> > >> > >> /etc/cloudstack/agent/log4j-cloud.xml
>> >> >> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management stop
>> >> >> > >> > >> Stopping cloudstack-management:
>> [
>> >> >> OK ]
>> >> >> > >> > >> [root@cs-east-prod1 ~]# mv
>> >> >> > >> > >> /var/log/cloudstack/management/management-server.log
>> >> >> > >> > >> /var/log/cloudstack/management/management-server.bak
>> >> >> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management
>> start
>> >> >> > >> > >> Starting cloudstack-management:
>> [
>> >> >> OK ]
>> >> >> > >> > >> [root@cs-east-prod1 ~]#
>> >> >> > >> > >>
>> >> >> > >> > >> Please see attached ZIP of clean management-server.log in
>> >> DEBUG
>> >> >> > mode.
>> >> >> > >> At
>> >> >> > >> > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the
>> attached
>> >> >> > >> > >> management-server.log*) I tried starting the Virtual Router
>> >> from
>> >> >> > the
>> >> >> > >> > >> Management UI and received this "Status" message in the UI.
>> >> >> > >> > >>
>> >> >> > >> > >> [image: Inline image 1]
>> >> >> > >> > >>
>> >> >> > >> > >> I can see in the management-server.log that while the
>> >> >> > >> > cloudstack-management
>> >> >> > >> > >> service is initializing, it throws the following ERROR at
>> >> exactly
>> >> >> > >> > (*2014-11-03
>> >> >> > >> > >> 08:14:36,813 on line 342 of the attached
>> >> management-server.log*):
>> >> >> > >> > >>
>> >> >> > >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
>> >> >> > (main:null)
>> >> >> > >> > Unable
>> >> >> > >> > >> to ping management server at 10.97.38.10:9090 due to
>> >> >> > ConnectException
>> >> >> > >> > >> java.net.ConnectException: Connection refused
>> >> >> > >> > >> at sun.nio.ch.Net.connect0(Native Method)
>> >> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:465)
>> >> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:457)
>> >> >> > >> > >> at
>> >> >> > sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> >
>> >>
>> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >>
>> >> >>
>> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
>> >> >> > >> > >> at
>> >> >> > >> >
>> >> >> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> >
>> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
>> >> >> > >> > >> at
>> >> >> > >>
>> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
>> >> >> > >> > >> at
>> >> >> > >> >
>> >> >> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
>> >> >> > >> > >> at
>> >> >> > >>
>> org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
>> >> >> > >> > >> at
>> >> >> > >> >
>> >> >> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> >
>> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >>
>> >> >>
>> org.apache.catalina.core.StandardService.start(StandardService.java:516)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> >
>> org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
>> >> >> > >> > >> at
>> >> >> org.apache.catalina.startup.Catalina.start(Catalina.java:593)
>> >> >> > >> > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> >> Method)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> >> >> > >> > >> at
>> >> >> > >> > >>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >> >> > >> > >> at java.lang.reflect.Method.invoke(Method.java:606)
>> >> >> > >> > >> at
>> >> >> > org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
>> >> >> > >> > >> at
>> >> >> > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
>> >> >> > >> > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl]
>> >> >> > (main:null)
>> >> >> > >> > >> Detected that another management node with the same IP
>> >> >> 10.97.38.10
>> >> >> > is
>> >> >> > >> > >> considered as running in DB, however it is not pingable, we
>> >> will
>> >> >> > >> > continue
>> >> >> > >> > >> cluster initialization with this management server node
>> >> >> > >> > >>
>> >> >> > >> > >> I'm not sure what that error means, but more than likely
>> >> that's
>> >> >> the
>> >> >> > >> > reason
>> >> >> > >> > >> why the Virtual Router will not start.
>> >> >> > >> > >>
>> >> >> > >> > >> * PLEASE HELP ! *
>> >> >> > >> > >>
>> >> >> > >> > >> Best Regards,
>> >> >> > >> > >>
>> >> >> > >> > >>
>> >> >> > >> > >>
>> >> >> > >> > >> Adam Scarcella
>> >> >> > >> > >>
>> >> >> > >> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro>
>> wrote:
>> >> >> > >> > >>
>> >> >> > >> > >>> Hi,
>> >> >> > >> > >>>
>> >> >> > >> > >>> More agent and management logs please, make sure they are
>> in
>> >> >> > DEBUG[1]
>> >> >> > >> > mode.
>> >> >> > >> > >>>
>> >> >> > >> > >>> [1] -
>> >> >> > >> > >>>
>> >> >> > >>
>> >> >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
>> >> >> > >> > >>>
>> >> >> > >> > >>> --
>> >> >> > >> > >>> Sent from the Delta quadrant using Borg technology!
>> >> >> > >> > >>>
>> >> >> > >> > >>> Nux!
>> >> >> > >> > >>> www.nux.ro
>> >> >> > >> > >>>
>> >> >> > >> > >>> ----- Original Message -----
>> >> >> > >> > >>> > From: "Adam" <ad...@gmail.com>
>> >> >> > >> > >>> > To: users@cloudstack.apache.org
>> >> >> > >> > >>> > Sent: Sunday, 2 November, 2014 23:33:22
>> >> >> > >> > >>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> >> >> > >> > >>>
>> >> >> > >> > >>> > I've performed a complete upgrade from 4.3 to 4.4.1 and
>> >> >> followed
>> >> >> > >> the
>> >> >> > >> > >>> > instructions perfectly, but my Virtual Router will not
>> >> start,
>> >> >> > nor
>> >> >> > >> am
>> >> >> > >> > I
>> >> >> > >> > >>> able
>> >> >> > >> > >>> > to "upgrade" it.
>> >> >> > >> > >>> >
>> >> >> > >> > >>> > I tried destroying the Virtual Router, starting any VM
>> >> >> Instance,
>> >> >> > >> and
>> >> >> > >> > >>> > allowing CS to recreate the VR. However that didn't
>> help.
>> >> The
>> >> >> VR
>> >> >> > >> > still
>> >> >> > >> > >>> did
>> >> >> > >> > >>> > not start and it was still saying that it required to be
>> >> >> > upgraded.
>> >> >> > >> > >>> >
>> >> >> > >> > >>> > [image: Inline image 1]
>> >> >> > >> > >>> > [image: Inline image 2]
>> >> >> > >> > >>> > [image: Inline image 3]
>> >> >> > >> > >>> >
>> >> >> > >> > >>> > I know there are manual steps to finalize the systemvm
>> >> >> upgrade,
>> >> >> > but
>> >> >> > >> > I did
>> >> >> > >> > >>> > those steps:
>> >> >> > >> > >>> >
>> >> >> > >> > >>> >
>> >> >> > >> > >>>
>> >> >> > >> >
>> >> >> > >>
>> >> >> >
>> >> >>
>> >>
>> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
>> >> >> > >> > >>> >
>> >> >> > >> > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
>> >> >> > >> > >>> > Enter password:
>> >> >> > >> > >>> > Welcome to the MySQL monitor. Commands end with ; or
>> \g.
>> >> >> > >> > >>> > Your MySQL connection id is 174
>> >> >> > >> > >>> > Server version: 5.1.73-log Source distribution
>> >> >> > >> > >>> >
>> >> >> > >> > >>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates.
>> All
>> >> >> > rights
>> >> >> > >> > >>> reserved.
>> >> >> > >> > >>> >
>> >> >> > >> > >>> > Oracle is a registered trademark of Oracle Corporation
>> >> and/or
>> >> >> > its
>> >> >> > >> > >>> > affiliates. Other names may be trademarks of their
>> >> respective
>> >> >> > >> > >>> > owners.
>> >> >> > >> > >>> >
>> >> >> > >> > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the
>> >> current
>> >> >> > input
>> >> >> > >> > >>> > statement.
>> >> >> > >> > >>> >
>> >> >> > >> > >>> > mysql> select id,name from cloud.vm_template where name
>> =
>> >> >> > >> > >>> > 'systemvm-kvm-4.4';
>> >> >> > >> > >>> > +-----+------------------+
>> >> >> > >> > >>> >| id | name |
>> >> >> > >> > >>> > +-----+------------------+
>> >> >> > >> > >>> >| 274 | systemvm-kvm-4.4 |
>> >> >> > >> > >>> > +-----+------------------+
>> >> >> > >> > >>> > 1 row in set (0.00 sec)
>> >> >> > >> > >>> >
>> >> >> > >> > >>> > mysql> select type from cloud.vm_template where
>> id='274';
>> >> >> > >> > >>> > +--------+
>> >> >> > >> > >>> >| type |
>> >> >> > >> > >>> > +--------+
>> >> >> > >> > >>> >| SYSTEM |
>> >> >> > >> > >>> > +--------+
>> >> >> > >> > >>> > 1 row in set (0.00 sec)
>> >> >> > >> > >>> >
>> >> >> > >> > >>> > mysql> select distinct(vm_template_id) from
>> >> cloud.vm_instance
>> >> >> > where
>> >> >> > >> > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
>> >> >> > >> > >>> > +----------------+
>> >> >> > >> > >>> >| vm_template_id |
>> >> >> > >> > >>> > +----------------+
>> >> >> > >> > >>> >| 274 |
>> >> >> > >> > >>> > +----------------+
>> >> >> > >> > >>> > 1 row in set (0.00 sec)
>> >> >> > >> > >>> >
>> >> >> > >> > >>> > mysql> select distinct(vm_template_id) from
>> >> cloud.vm_instance
>> >> >> > where
>> >> >> > >> > >>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM';
>> >> >> > >> > >>> > +----------------+
>> >> >> > >> > >>> >| vm_template_id |
>> >> >> > >> > >>> > +----------------+
>> >> >> > >> > >>> >| 274 |
>> >> >> > >> > >>> > +----------------+
>> >> >> > >> > >>> > 1 row in set (0.03 sec)
>> >> >> > >> > >>> >
>> >> >> > >> > >>> > mysql> select value from cloud.configuration where name
>> =
>> >> >> > >> > >>> > 'router.template.kvm';
>> >> >> > >> > >>> > +------------------+
>> >> >> > >> > >>> >| value |
>> >> >> > >> > >>> > +------------------+
>> >> >> > >> > >>> >| systemvm-kvm-4.4 |
>> >> >> > >> > >>> > +------------------+
>> >> >> > >> > >>> > 1 row in set (0.00 sec)
>> >> >> > >> > >>> >
>> >> >> > >> > >>> > mysql>
>> >> >> > >> > >>> >
>> >> >> > >> > >>> > * PLEASE HELP ! *
>> >> >> > >> > >>> >
>> >> >> > >> > >>> > Best Regards,
>> >> >> > >> > >>> >
>> >> >> > >> > > >> > Adam Scarcella
>> >> >> > >> >
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Thanks for your support,
>> >> >> Motty
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Adam <ad...@gmail.com>.
I've decided to revert back to 4.3.1 since I cannot wait any loner to get
4.4.1 working.
I have a complete backup of all three databases just before I attempted to
upgrade so I'm assuming all I would have to do is remove 4.4.1, reinstall
4.3.1, restore the db and then start everything back up correct?
Are there any official steps to roll back?
Best Regards,
Adam Scarcella
On Mon, Nov 3, 2014 at 12:13 PM, Nux! <nu...@li.nux.ro> wrote:
> What I am saying - and excuse my poor explanations - is that the HV may
> have already cached the old template, so the VR still uses the old broken
> one.
>
> What I was suggesting is find out which /var/lib/libvirt/images/blah.qcow2
> the VR is using and replace it with my template.
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
> > From: "Adam" <ad...@gmail.com>
> > To: users@cloudstack.apache.org
> > Sent: Monday, 3 November, 2014 17:03:01
> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>
> >>
> >> Not sure if this operation that you performed replaces the current
> >> systemvm template and *especially if it replaces it also on the
> >> hypervisor where it is running.*
> >>
> >
> > That procedure is the one documented in the official CloudStack
> > documentation so I hope it actually replaces the systemvm template. The
> > final output from that command tells it was successful:
> >
> > Successfully installed system VM template to
> > /export/secondary/template/tmpl/1/274/
> >
> > Also, as stated before, the VR is not running on any hypervisor / host
> > because I cannot get it to run, so I'm unsure what you mean when you
> > say "*especially
> > if it replaces it also on the hypervisor where it is running.*"
> >
> > One more thing I would try is to make sure the VR on the HV is also using
> >> this image. You can try to compare md5sums of the image on the secondary
> >> storage and also the backend file for the VR's disk image (qemu-img info
> >> will show it to you) on the HV.
> >> If the sums differ then I would try to replace the VR disk image with my
> >> template directly on the HV and see if that does it.
> >
> >
> > I really don't understand what you're asking me to do.
> >
> > Best Regards,
> >
> >
> >
> > Adam Scarcella
> >
> > On Mon, Nov 3, 2014 at 11:50 AM, Nux! <nu...@li.nux.ro> wrote:
> >
> >> Adam,
> >>
> >> Not sure if this operation that you performed replaces the current
> >> systemvm template and especially if it replaces it also on the
> hypervisor
> >> where it is running.
> >>
> >> One more thing I would try is to make sure the VR on the HV is also
> using
> >> this image. You can try to compare md5sums of the image on the secondary
> >> storage and also the backend file for the VR's disk image (qemu-img info
> >> will show it to you) on the HV.
> >> If the sums differ then I would try to replace the VR disk image with my
> >> template directly on the HV and see if that does it.
> >>
> >> --
> >> Sent from the Delta quadrant using Borg technology!
> >>
> >> Nux!
> >> www.nux.ro
> >>
> >> ----- Original Message -----
> >> > From: "Adam" <ad...@gmail.com>
> >> > To: users@cloudstack.apache.org
> >> > Sent: Monday, 3 November, 2014 16:08:05
> >> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >>
> >> > Thanks HTH,
> >> >
> >> > I decided to try to use your already modified qcow2 file:
> >> >
> >> > [root@cs-east-prod1 /]#
> >> >
> >>
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> >> > -m /export/secondary -u
> >> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> >> > -h kvm -F
> >> > --2014-11-03 10:11:32--
> >> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> >> > Resolving tmp.nux.ro... 217.19.15.108
> >> > Connecting to tmp.nux.ro|217.19.15.108|:80... connected.
> >> > HTTP request sent, awaiting response... 200 OK
> >> > Length: 305498799 (291M) [application/x-bzip2]
> >> > Saving to:
> >> >
> >>
> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
> >> >
> >> >
> >>
> 100%[===================================================================================================================>]
> >> > 305,498,799 9.02M/s in 37s
> >> >
> >> > 2014-11-03 10:12:10 (7.82 MB/s) -
> >> >
> >>
> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
> >> > saved [305498799/305498799]
> >> >
> >> > Uncompressing to
> >> >
> >>
> /usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2.tmp
> >> > (type bz2)...could take a long time
> >> > Moving to
> >> >
> >>
> /export/secondary/template/tmpl/1/274///00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2...could
> >> > take a while
> >> > Successfully installed system VM template to
> >> > /export/secondary/template/tmpl/1/274/
> >> > [root@cs-east-prod1 /]#
> >> >
> >> > Then I went into the Admin GUI and tried to start the VR, but it
> failed
> >> > with the exact same message:
> >> >
> >> > [image: Inline image 1]
> >> >
> >> > So I destroyed the Virtual Router and stopped and started the
> following
> >> > services:
> >> >
> >> > [root@cs-east-prod1 /]# service iptables stop
> >> > iptables: Setting chains to policy ACCEPT: nat filter [ OK ]
> >> > iptables: Flushing firewall rules: [ OK ]
> >> > iptables: Unloading modules: [ OK ]
> >> > [root@cs-east-prod1 /]# service cloudstack-management stop
> >> > Stopping cloudstack-management: [ OK ]
> >> > [root@cs-east-prod1 /]# service cloudstack-agent stop
> >> > Stopping Cloud Agent:
> >> > [root@cs-east-prod1 /]# service cloudstack-usage stop
> >> > Stopping CloudStack Usage Monitor cloudstack-usage [ OK ]
> >> > [root@cs-east-prod1 /]# service cloudstack-management start
> >> > Starting cloudstack-management: [ OK ]
> >> > [root@cs-east-prod1 /]# service cloudstack-usage start
> >> > Starting CloudStack Usage Monitor cloudstack-usage [ OK ]
> >> > [root@cs-east-prod1 /]# service cloudstack-agent start
> >> > Starting Cloud Agent:
> >> > [root@cs-east-prod1 /]#
> >> >
> >> > I then tried starting any VM instance to have CS automatically
> re-create
> >> > the VR, but that failed as well.
> >> >
> >> > Motty, I also looked at trying to increase the RAM and CPU for the
> router
> >> > but only saw one option
> >> >
> >> > [image: Inline image 2]
> >> >
> >> > So I created a new Service Offering with twice as much resources:
> >> >
> >> > [image: Inline image 3]
> >> >
> >> > Then upgraded the service offering for the VR
> >> >
> >> > [image: Inline image 4]
> >> >
> >> > [image: Inline image 6]
> >> >
> >> > But it failed still with the exact same error:
> >> >
> >> > [image: Inline image 7]
> >> >
> >> > Best Regards,
> >> >
> >> >
> >> >
> >> > Adam Scarcella
> >> >
> >> > On Mon, Nov 3, 2014 at 10:23 AM, motty cruz <mo...@gmail.com>
> >> wrote:
> >> >
> >> >> Adam,
> >> >> what are the specifications of the VR? can you upgrade the RAM and
> CPU
> >> and
> >> >> try to restart that VR? I had similar issue and giving the VR more
> RAM
> >> and
> >> >> CPU will start the VR.
> >> >>
> >> >> Thanks,
> >> >> Motty
> >> >>
> >> >> On Mon, Nov 3, 2014 at 7:04 AM, Nux! <nu...@li.nux.ro> wrote:
> >> >>
> >> >> > Adam,
> >> >> >
> >> >> > You can modify the qcow2 file that is in your secondary storage,
> i.e.
> >> you
> >> >> > should mount that storage somewhere and go to
> >> >> > /mount/point/template/tmpl/274 and modify the qcow2 file.
> >> >> >
> >> >> > Alternatively you could use my already modified copy to overwrite
> your
> >> >> > uuid.qcow2 with (at your own risk of course):
> >> >> > http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> >> >> >
> >> >> > HTH
> >> >> >
> >> >> > --
> >> >> > Sent from the Delta quadrant using Borg technology!
> >> >> >
> >> >> > Nux!
> >> >> > www.nux.ro
> >> >> >
> >> >> > ----- Original Message -----
> >> >> > > From: "Adam" <ad...@gmail.com>
> >> >> > > To: users@cloudstack.apache.org
> >> >> > > Sent: Monday, 3 November, 2014 14:58:58
> >> >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >> >> >
> >> >> > > [root@cs-east-prod1 management]# netstat -a|grep 9090
> >> >> > > unix 3 [ ] STREAM CONNECTED 29090
> >> >> > > [root@cs-east-prod1 management]# nmap localhost
> >> >> > >
> >> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47 EST
> >> >> > > Nmap scan report for localhost (127.0.0.1)
> >> >> > > Host is up (0.000015s latency).
> >> >> > > Other addresses for localhost (not scanned): 127.0.0.1
> >> >> > > Not shown: 990 closed ports
> >> >> > > PORT STATE SERVICE
> >> >> > > 22/tcp open ssh
> >> >> > > 25/tcp open smtp
> >> >> > > 80/tcp open http
> >> >> > > 111/tcp open rpcbind
> >> >> > > 199/tcp open smux
> >> >> > > 631/tcp open ipp
> >> >> > > 2049/tcp open nfs
> >> >> > > 3306/tcp open mysql
> >> >> > > 8080/tcp open http-proxy
> >> >> > > 9090/tcp open zeus-admin
> >> >> > >
> >> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
> >> >> > > [root@cs-east-prod1 management]# telnet localhost 9090
> >> >> > > Trying ::1...
> >> >> > > Connected to localhost.
> >> >> > > Escape character is '^]'.
> >> >> > > Connection closed by foreign host.
> >> >> > > [root@cs-east-prod1 management]# nmap 10.97.38.10
> >> >> > >
> >> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49 EST
> >> >> > > Nmap scan report for cs-east-prod1.na.tibco.com (10.97.38.10)
> >> >> > > Host is up (0.000015s latency).
> >> >> > > Not shown: 993 closed ports
> >> >> > > PORT STATE SERVICE
> >> >> > > 22/tcp open ssh
> >> >> > > 80/tcp open http
> >> >> > > 111/tcp open rpcbind
> >> >> > > 2049/tcp open nfs
> >> >> > > 3306/tcp open mysql
> >> >> > > 8080/tcp open http-proxy
> >> >> > > 9090/tcp open zeus-admin
> >> >> > >
> >> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
> >> >> > > [root@cs-east-prod1 management]# telnet 10.97.38.10 9090
> >> >> > > Trying 10.97.38.10...
> >> >> > > Connected to 10.97.38.10.
> >> >> > > Escape character is '^]'.
> >> >> > > Connection closed by foreign host.
> >> >> > > [root@cs-east-prod1 management]#
> >> >> > >
> >> >> > > Also, I checked for the /opt/cloud/bin/ directory on my
> Management
> >> >> Host,
> >> >> > > but it's not there:
> >> >> > >
> >> >> > > [root@cs-east-prod1 management]# cd /opt/cloud/bin/
> >> >> > > -bash: cd: /opt/cloud/bin/: No such file or directory
> >> >> > > [root@cs-east-prod1 management]# cd /
> >> >> > > [root@cs-east-prod1 /]# find . -name "createipAlias.sh"
> >> >> > > [root@cs-east-prod1 /]# find . -name "createIpAlias.sh"
> >> >> > > [root@cs-east-prod1 /]#
> >> >> > >
> >> >> > > Are you saying that the /opt/cloud/bin/createipAlias.sh script is
> >> >> inside
> >> >> > > the Virtual Router VM? If so, how could I possibly change that
> >> since it
> >> >> > > won't start?
> >> >> > >
> >> >> > > Best Regards,
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > > Adam Scarcella
> >> >> > >
> >> >> > > On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion <
> >> pdion891@apache.org>
> >> >> > wrote:
> >> >> > >
> >> >> > >> Could it be possible the firewall is blocking port 9090 from the
> >> >> > >> management-server?
> >> >> > >>
> >> >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
> >> (main:null)
> >> >> > Unable
> >> >> > >> to ping management server at 10.97.38.10:9090 due to
> >> ConnectException
> >> >> > >> java.net.ConnectException: Connection refused
> >> >> > >>
> >> >> > >> On Mon, Nov 3, 2014 at 9:02 AM, Nux! <nu...@li.nux.ro> wrote:
> >> >> > >>
> >> >> > >> > BTW the agent logs should help clarify this.
> >> >> > >> > The aforementioned changes must be done either in the current
> >> >> > (broken) VR
> >> >> > >> > or in the systemvm (after which you should redeploy the VR to
> >> pick
> >> >> up
> >> >> > the
> >> >> > >> > change).
> >> >> > >> >
> >> >> > >> > HTH
> >> >> > >> > Lucian
> >> >> > >> >
> >> >> > >> > --
> >> >> > >> > Sent from the Delta quadrant using Borg technology!
> >> >> > >> >
> >> >> > >> > Nux!
> >> >> > >> > www.nux.ro
> >> >> > >> >
> >> >> > >> > ----- Original Message -----
> >> >> > >> > > From: "Nux!" <nu...@li.nux.ro>
> >> >> > >> > > To: users@cloudstack.apache.org
> >> >> > >> > > Sent: Monday, 3 November, 2014 13:53:10
> >> >> > >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >> >> > >> >
> >> >> > >> > > Hello,
> >> >> > >> > >
> >> >> > >> > > I think you are hitting a problem I also had:
> >> >> > >> > > "2014-11-03 08:16:17,024 ERROR
> >> [c.c.v.VirtualMachineManagerImpl]
> >> >> > >> > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556
> >> >> ctx-fcb072b2)
> >> >> > >> > Failed to
> >> >> > >> > > start instance VM[DomainRouter|r-4758-VM]
> >> >> > >> > > com.cloud.utils.exception.ExecutionException: Unable to
> start
> >> >> > >> > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart,
> not
> >> >> > retrying"
> >> >> > >> > >
> >> >> > >> > > It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781
> >> >> > >> > >
> >> >> > >> > > Basically the systemvm template has some problems, you need
> to
> >> >> > somehow
> >> >> > >> > copy
> >> >> > >> > > /opt/cloud/bin/createIpAlias.sh to
> >> /opt/cloud/bin/createipAlias.sh
> >> >> > to
> >> >> > >> > get it
> >> >> > >> > > working.
> >> >> > >> > >
> >> >> > >> > > --
> >> >> > >> > > Sent from the Delta quadrant using Borg technology!
> >> >> > >> > >
> >> >> > >> > > Nux!
> >> >> > >> > > www.nux.ro
> >> >> > >> > >
> >> >> > >> > > ----- Original Message -----
> >> >> > >> > >> From: "Adam" <ad...@gmail.com>
> >> >> > >> > >> To: users@cloudstack.apache.org
> >> >> > >> > >> Sent: Monday, 3 November, 2014 13:32:12
> >> >> > >> > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to
> 4.4.1
> >> >> > >> > >
> >> >> > >> > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release
> >> >> > >> > >> CentOS release 6.6 (Final)
> >> >> > >> > >> [root@cs-east-prod1 ~]# uname -a
> >> >> > >> > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15
> >> >> > 04:27:16
> >> >> > >> UTC
> >> >> > >> > >> 2014 x86_64 x86_64 x86_64 GNU/Linux
> >> >> > >> > >> [root@cs-east-prod1 ~]# virsh --version
> >> >> > >> > >> 0.10.2
> >> >> > >> > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
> >> >> > >> > >> /etc/cloudstack/agent/log4j-cloud.xml
> >> >> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management stop
> >> >> > >> > >> Stopping cloudstack-management:
> [
> >> >> OK ]
> >> >> > >> > >> [root@cs-east-prod1 ~]# mv
> >> >> > >> > >> /var/log/cloudstack/management/management-server.log
> >> >> > >> > >> /var/log/cloudstack/management/management-server.bak
> >> >> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management
> start
> >> >> > >> > >> Starting cloudstack-management:
> [
> >> >> OK ]
> >> >> > >> > >> [root@cs-east-prod1 ~]#
> >> >> > >> > >>
> >> >> > >> > >> Please see attached ZIP of clean management-server.log in
> >> DEBUG
> >> >> > mode.
> >> >> > >> At
> >> >> > >> > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the
> attached
> >> >> > >> > >> management-server.log*) I tried starting the Virtual Router
> >> from
> >> >> > the
> >> >> > >> > >> Management UI and received this "Status" message in the UI.
> >> >> > >> > >>
> >> >> > >> > >> [image: Inline image 1]
> >> >> > >> > >>
> >> >> > >> > >> I can see in the management-server.log that while the
> >> >> > >> > cloudstack-management
> >> >> > >> > >> service is initializing, it throws the following ERROR at
> >> exactly
> >> >> > >> > (*2014-11-03
> >> >> > >> > >> 08:14:36,813 on line 342 of the attached
> >> management-server.log*):
> >> >> > >> > >>
> >> >> > >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
> >> >> > (main:null)
> >> >> > >> > Unable
> >> >> > >> > >> to ping management server at 10.97.38.10:9090 due to
> >> >> > ConnectException
> >> >> > >> > >> java.net.ConnectException: Connection refused
> >> >> > >> > >> at sun.nio.ch.Net.connect0(Native Method)
> >> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:465)
> >> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:457)
> >> >> > >> > >> at
> >> >> > sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> >
> >>
> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >>
> >> >>
> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
> >> >> > >> > >> at
> >> >> > >> >
> >> >> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> >
> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
> >> >> > >> > >> at
> >> >> > >>
> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> >> >> > >> > >> at
> >> >> > >> >
> >> >> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
> >> >> > >> > >> at
> >> >> > >>
> org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
> >> >> > >> > >> at
> >> >> > >> >
> >> >> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> >
> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >>
> >> >>
> org.apache.catalina.core.StandardService.start(StandardService.java:516)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> >
> org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
> >> >> > >> > >> at
> >> >> org.apache.catalina.startup.Catalina.start(Catalina.java:593)
> >> >> > >> > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> >> Method)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> >> > >> > >> at
> >> >> > >> > >>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> >> > >> > >> at java.lang.reflect.Method.invoke(Method.java:606)
> >> >> > >> > >> at
> >> >> > org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> >> >> > >> > >> at
> >> >> > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> >> >> > >> > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl]
> >> >> > (main:null)
> >> >> > >> > >> Detected that another management node with the same IP
> >> >> 10.97.38.10
> >> >> > is
> >> >> > >> > >> considered as running in DB, however it is not pingable, we
> >> will
> >> >> > >> > continue
> >> >> > >> > >> cluster initialization with this management server node
> >> >> > >> > >>
> >> >> > >> > >> I'm not sure what that error means, but more than likely
> >> that's
> >> >> the
> >> >> > >> > reason
> >> >> > >> > >> why the Virtual Router will not start.
> >> >> > >> > >>
> >> >> > >> > >> * PLEASE HELP ! *
> >> >> > >> > >>
> >> >> > >> > >> Best Regards,
> >> >> > >> > >>
> >> >> > >> > >>
> >> >> > >> > >>
> >> >> > >> > >> Adam Scarcella
> >> >> > >> > >>
> >> >> > >> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro>
> wrote:
> >> >> > >> > >>
> >> >> > >> > >>> Hi,
> >> >> > >> > >>>
> >> >> > >> > >>> More agent and management logs please, make sure they are
> in
> >> >> > DEBUG[1]
> >> >> > >> > mode.
> >> >> > >> > >>>
> >> >> > >> > >>> [1] -
> >> >> > >> > >>>
> >> >> > >>
> >> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
> >> >> > >> > >>>
> >> >> > >> > >>> --
> >> >> > >> > >>> Sent from the Delta quadrant using Borg technology!
> >> >> > >> > >>>
> >> >> > >> > >>> Nux!
> >> >> > >> > >>> www.nux.ro
> >> >> > >> > >>>
> >> >> > >> > >>> ----- Original Message -----
> >> >> > >> > >>> > From: "Adam" <ad...@gmail.com>
> >> >> > >> > >>> > To: users@cloudstack.apache.org
> >> >> > >> > >>> > Sent: Sunday, 2 November, 2014 23:33:22
> >> >> > >> > >>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >> >> > >> > >>>
> >> >> > >> > >>> > I've performed a complete upgrade from 4.3 to 4.4.1 and
> >> >> followed
> >> >> > >> the
> >> >> > >> > >>> > instructions perfectly, but my Virtual Router will not
> >> start,
> >> >> > nor
> >> >> > >> am
> >> >> > >> > I
> >> >> > >> > >>> able
> >> >> > >> > >>> > to "upgrade" it.
> >> >> > >> > >>> >
> >> >> > >> > >>> > I tried destroying the Virtual Router, starting any VM
> >> >> Instance,
> >> >> > >> and
> >> >> > >> > >>> > allowing CS to recreate the VR. However that didn't
> help.
> >> The
> >> >> VR
> >> >> > >> > still
> >> >> > >> > >>> did
> >> >> > >> > >>> > not start and it was still saying that it required to be
> >> >> > upgraded.
> >> >> > >> > >>> >
> >> >> > >> > >>> > [image: Inline image 1]
> >> >> > >> > >>> > [image: Inline image 2]
> >> >> > >> > >>> > [image: Inline image 3]
> >> >> > >> > >>> >
> >> >> > >> > >>> > I know there are manual steps to finalize the systemvm
> >> >> upgrade,
> >> >> > but
> >> >> > >> > I did
> >> >> > >> > >>> > those steps:
> >> >> > >> > >>> >
> >> >> > >> > >>> >
> >> >> > >> > >>>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
> >> >> > >> > >>> >
> >> >> > >> > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
> >> >> > >> > >>> > Enter password:
> >> >> > >> > >>> > Welcome to the MySQL monitor. Commands end with ; or
> \g.
> >> >> > >> > >>> > Your MySQL connection id is 174
> >> >> > >> > >>> > Server version: 5.1.73-log Source distribution
> >> >> > >> > >>> >
> >> >> > >> > >>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates.
> All
> >> >> > rights
> >> >> > >> > >>> reserved.
> >> >> > >> > >>> >
> >> >> > >> > >>> > Oracle is a registered trademark of Oracle Corporation
> >> and/or
> >> >> > its
> >> >> > >> > >>> > affiliates. Other names may be trademarks of their
> >> respective
> >> >> > >> > >>> > owners.
> >> >> > >> > >>> >
> >> >> > >> > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the
> >> current
> >> >> > input
> >> >> > >> > >>> > statement.
> >> >> > >> > >>> >
> >> >> > >> > >>> > mysql> select id,name from cloud.vm_template where name
> =
> >> >> > >> > >>> > 'systemvm-kvm-4.4';
> >> >> > >> > >>> > +-----+------------------+
> >> >> > >> > >>> >| id | name |
> >> >> > >> > >>> > +-----+------------------+
> >> >> > >> > >>> >| 274 | systemvm-kvm-4.4 |
> >> >> > >> > >>> > +-----+------------------+
> >> >> > >> > >>> > 1 row in set (0.00 sec)
> >> >> > >> > >>> >
> >> >> > >> > >>> > mysql> select type from cloud.vm_template where
> id='274';
> >> >> > >> > >>> > +--------+
> >> >> > >> > >>> >| type |
> >> >> > >> > >>> > +--------+
> >> >> > >> > >>> >| SYSTEM |
> >> >> > >> > >>> > +--------+
> >> >> > >> > >>> > 1 row in set (0.00 sec)
> >> >> > >> > >>> >
> >> >> > >> > >>> > mysql> select distinct(vm_template_id) from
> >> cloud.vm_instance
> >> >> > where
> >> >> > >> > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
> >> >> > >> > >>> > +----------------+
> >> >> > >> > >>> >| vm_template_id |
> >> >> > >> > >>> > +----------------+
> >> >> > >> > >>> >| 274 |
> >> >> > >> > >>> > +----------------+
> >> >> > >> > >>> > 1 row in set (0.00 sec)
> >> >> > >> > >>> >
> >> >> > >> > >>> > mysql> select distinct(vm_template_id) from
> >> cloud.vm_instance
> >> >> > where
> >> >> > >> > >>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM';
> >> >> > >> > >>> > +----------------+
> >> >> > >> > >>> >| vm_template_id |
> >> >> > >> > >>> > +----------------+
> >> >> > >> > >>> >| 274 |
> >> >> > >> > >>> > +----------------+
> >> >> > >> > >>> > 1 row in set (0.03 sec)
> >> >> > >> > >>> >
> >> >> > >> > >>> > mysql> select value from cloud.configuration where name
> =
> >> >> > >> > >>> > 'router.template.kvm';
> >> >> > >> > >>> > +------------------+
> >> >> > >> > >>> >| value |
> >> >> > >> > >>> > +------------------+
> >> >> > >> > >>> >| systemvm-kvm-4.4 |
> >> >> > >> > >>> > +------------------+
> >> >> > >> > >>> > 1 row in set (0.00 sec)
> >> >> > >> > >>> >
> >> >> > >> > >>> > mysql>
> >> >> > >> > >>> >
> >> >> > >> > >>> > * PLEASE HELP ! *
> >> >> > >> > >>> >
> >> >> > >> > >>> > Best Regards,
> >> >> > >> > >>> >
> >> >> > >> > > >> > Adam Scarcella
> >> >> > >> >
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Thanks for your support,
> >> >> Motty
>
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Nux! <nu...@li.nux.ro>.
What I am saying - and excuse my poor explanations - is that the HV may have already cached the old template, so the VR still uses the old broken one.
What I was suggesting is find out which /var/lib/libvirt/images/blah.qcow2 the VR is using and replace it with my template.
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Adam" <ad...@gmail.com>
> To: users@cloudstack.apache.org
> Sent: Monday, 3 November, 2014 17:03:01
> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>>
>> Not sure if this operation that you performed replaces the current
>> systemvm template and *especially if it replaces it also on the
>> hypervisor where it is running.*
>>
>
> That procedure is the one documented in the official CloudStack
> documentation so I hope it actually replaces the systemvm template. The
> final output from that command tells it was successful:
>
> Successfully installed system VM template to
> /export/secondary/template/tmpl/1/274/
>
> Also, as stated before, the VR is not running on any hypervisor / host
> because I cannot get it to run, so I'm unsure what you mean when you
> say "*especially
> if it replaces it also on the hypervisor where it is running.*"
>
> One more thing I would try is to make sure the VR on the HV is also using
>> this image. You can try to compare md5sums of the image on the secondary
>> storage and also the backend file for the VR's disk image (qemu-img info
>> will show it to you) on the HV.
>> If the sums differ then I would try to replace the VR disk image with my
>> template directly on the HV and see if that does it.
>
>
> I really don't understand what you're asking me to do.
>
> Best Regards,
>
>
>
> Adam Scarcella
>
> On Mon, Nov 3, 2014 at 11:50 AM, Nux! <nu...@li.nux.ro> wrote:
>
>> Adam,
>>
>> Not sure if this operation that you performed replaces the current
>> systemvm template and especially if it replaces it also on the hypervisor
>> where it is running.
>>
>> One more thing I would try is to make sure the VR on the HV is also using
>> this image. You can try to compare md5sums of the image on the secondary
>> storage and also the backend file for the VR's disk image (qemu-img info
>> will show it to you) on the HV.
>> If the sums differ then I would try to replace the VR disk image with my
>> template directly on the HV and see if that does it.
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
>>
>> ----- Original Message -----
>> > From: "Adam" <ad...@gmail.com>
>> > To: users@cloudstack.apache.org
>> > Sent: Monday, 3 November, 2014 16:08:05
>> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>>
>> > Thanks HTH,
>> >
>> > I decided to try to use your already modified qcow2 file:
>> >
>> > [root@cs-east-prod1 /]#
>> >
>> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
>> > -m /export/secondary -u
>> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
>> > -h kvm -F
>> > --2014-11-03 10:11:32--
>> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
>> > Resolving tmp.nux.ro... 217.19.15.108
>> > Connecting to tmp.nux.ro|217.19.15.108|:80... connected.
>> > HTTP request sent, awaiting response... 200 OK
>> > Length: 305498799 (291M) [application/x-bzip2]
>> > Saving to:
>> >
>> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
>> >
>> >
>> 100%[===================================================================================================================>]
>> > 305,498,799 9.02M/s in 37s
>> >
>> > 2014-11-03 10:12:10 (7.82 MB/s) -
>> >
>> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
>> > saved [305498799/305498799]
>> >
>> > Uncompressing to
>> >
>> /usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2.tmp
>> > (type bz2)...could take a long time
>> > Moving to
>> >
>> /export/secondary/template/tmpl/1/274///00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2...could
>> > take a while
>> > Successfully installed system VM template to
>> > /export/secondary/template/tmpl/1/274/
>> > [root@cs-east-prod1 /]#
>> >
>> > Then I went into the Admin GUI and tried to start the VR, but it failed
>> > with the exact same message:
>> >
>> > [image: Inline image 1]
>> >
>> > So I destroyed the Virtual Router and stopped and started the following
>> > services:
>> >
>> > [root@cs-east-prod1 /]# service iptables stop
>> > iptables: Setting chains to policy ACCEPT: nat filter [ OK ]
>> > iptables: Flushing firewall rules: [ OK ]
>> > iptables: Unloading modules: [ OK ]
>> > [root@cs-east-prod1 /]# service cloudstack-management stop
>> > Stopping cloudstack-management: [ OK ]
>> > [root@cs-east-prod1 /]# service cloudstack-agent stop
>> > Stopping Cloud Agent:
>> > [root@cs-east-prod1 /]# service cloudstack-usage stop
>> > Stopping CloudStack Usage Monitor cloudstack-usage [ OK ]
>> > [root@cs-east-prod1 /]# service cloudstack-management start
>> > Starting cloudstack-management: [ OK ]
>> > [root@cs-east-prod1 /]# service cloudstack-usage start
>> > Starting CloudStack Usage Monitor cloudstack-usage [ OK ]
>> > [root@cs-east-prod1 /]# service cloudstack-agent start
>> > Starting Cloud Agent:
>> > [root@cs-east-prod1 /]#
>> >
>> > I then tried starting any VM instance to have CS automatically re-create
>> > the VR, but that failed as well.
>> >
>> > Motty, I also looked at trying to increase the RAM and CPU for the router
>> > but only saw one option
>> >
>> > [image: Inline image 2]
>> >
>> > So I created a new Service Offering with twice as much resources:
>> >
>> > [image: Inline image 3]
>> >
>> > Then upgraded the service offering for the VR
>> >
>> > [image: Inline image 4]
>> >
>> > [image: Inline image 6]
>> >
>> > But it failed still with the exact same error:
>> >
>> > [image: Inline image 7]
>> >
>> > Best Regards,
>> >
>> >
>> >
>> > Adam Scarcella
>> >
>> > On Mon, Nov 3, 2014 at 10:23 AM, motty cruz <mo...@gmail.com>
>> wrote:
>> >
>> >> Adam,
>> >> what are the specifications of the VR? can you upgrade the RAM and CPU
>> and
>> >> try to restart that VR? I had similar issue and giving the VR more RAM
>> and
>> >> CPU will start the VR.
>> >>
>> >> Thanks,
>> >> Motty
>> >>
>> >> On Mon, Nov 3, 2014 at 7:04 AM, Nux! <nu...@li.nux.ro> wrote:
>> >>
>> >> > Adam,
>> >> >
>> >> > You can modify the qcow2 file that is in your secondary storage, i.e.
>> you
>> >> > should mount that storage somewhere and go to
>> >> > /mount/point/template/tmpl/274 and modify the qcow2 file.
>> >> >
>> >> > Alternatively you could use my already modified copy to overwrite your
>> >> > uuid.qcow2 with (at your own risk of course):
>> >> > http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
>> >> >
>> >> > HTH
>> >> >
>> >> > --
>> >> > Sent from the Delta quadrant using Borg technology!
>> >> >
>> >> > Nux!
>> >> > www.nux.ro
>> >> >
>> >> > ----- Original Message -----
>> >> > > From: "Adam" <ad...@gmail.com>
>> >> > > To: users@cloudstack.apache.org
>> >> > > Sent: Monday, 3 November, 2014 14:58:58
>> >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> >> >
>> >> > > [root@cs-east-prod1 management]# netstat -a|grep 9090
>> >> > > unix 3 [ ] STREAM CONNECTED 29090
>> >> > > [root@cs-east-prod1 management]# nmap localhost
>> >> > >
>> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47 EST
>> >> > > Nmap scan report for localhost (127.0.0.1)
>> >> > > Host is up (0.000015s latency).
>> >> > > Other addresses for localhost (not scanned): 127.0.0.1
>> >> > > Not shown: 990 closed ports
>> >> > > PORT STATE SERVICE
>> >> > > 22/tcp open ssh
>> >> > > 25/tcp open smtp
>> >> > > 80/tcp open http
>> >> > > 111/tcp open rpcbind
>> >> > > 199/tcp open smux
>> >> > > 631/tcp open ipp
>> >> > > 2049/tcp open nfs
>> >> > > 3306/tcp open mysql
>> >> > > 8080/tcp open http-proxy
>> >> > > 9090/tcp open zeus-admin
>> >> > >
>> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
>> >> > > [root@cs-east-prod1 management]# telnet localhost 9090
>> >> > > Trying ::1...
>> >> > > Connected to localhost.
>> >> > > Escape character is '^]'.
>> >> > > Connection closed by foreign host.
>> >> > > [root@cs-east-prod1 management]# nmap 10.97.38.10
>> >> > >
>> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49 EST
>> >> > > Nmap scan report for cs-east-prod1.na.tibco.com (10.97.38.10)
>> >> > > Host is up (0.000015s latency).
>> >> > > Not shown: 993 closed ports
>> >> > > PORT STATE SERVICE
>> >> > > 22/tcp open ssh
>> >> > > 80/tcp open http
>> >> > > 111/tcp open rpcbind
>> >> > > 2049/tcp open nfs
>> >> > > 3306/tcp open mysql
>> >> > > 8080/tcp open http-proxy
>> >> > > 9090/tcp open zeus-admin
>> >> > >
>> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
>> >> > > [root@cs-east-prod1 management]# telnet 10.97.38.10 9090
>> >> > > Trying 10.97.38.10...
>> >> > > Connected to 10.97.38.10.
>> >> > > Escape character is '^]'.
>> >> > > Connection closed by foreign host.
>> >> > > [root@cs-east-prod1 management]#
>> >> > >
>> >> > > Also, I checked for the /opt/cloud/bin/ directory on my Management
>> >> Host,
>> >> > > but it's not there:
>> >> > >
>> >> > > [root@cs-east-prod1 management]# cd /opt/cloud/bin/
>> >> > > -bash: cd: /opt/cloud/bin/: No such file or directory
>> >> > > [root@cs-east-prod1 management]# cd /
>> >> > > [root@cs-east-prod1 /]# find . -name "createipAlias.sh"
>> >> > > [root@cs-east-prod1 /]# find . -name "createIpAlias.sh"
>> >> > > [root@cs-east-prod1 /]#
>> >> > >
>> >> > > Are you saying that the /opt/cloud/bin/createipAlias.sh script is
>> >> inside
>> >> > > the Virtual Router VM? If so, how could I possibly change that
>> since it
>> >> > > won't start?
>> >> > >
>> >> > > Best Regards,
>> >> > >
>> >> > >
>> >> > >
>> >> > > Adam Scarcella
>> >> > >
>> >> > > On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion <
>> pdion891@apache.org>
>> >> > wrote:
>> >> > >
>> >> > >> Could it be possible the firewall is blocking port 9090 from the
>> >> > >> management-server?
>> >> > >>
>> >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
>> (main:null)
>> >> > Unable
>> >> > >> to ping management server at 10.97.38.10:9090 due to
>> ConnectException
>> >> > >> java.net.ConnectException: Connection refused
>> >> > >>
>> >> > >> On Mon, Nov 3, 2014 at 9:02 AM, Nux! <nu...@li.nux.ro> wrote:
>> >> > >>
>> >> > >> > BTW the agent logs should help clarify this.
>> >> > >> > The aforementioned changes must be done either in the current
>> >> > (broken) VR
>> >> > >> > or in the systemvm (after which you should redeploy the VR to
>> pick
>> >> up
>> >> > the
>> >> > >> > change).
>> >> > >> >
>> >> > >> > HTH
>> >> > >> > Lucian
>> >> > >> >
>> >> > >> > --
>> >> > >> > Sent from the Delta quadrant using Borg technology!
>> >> > >> >
>> >> > >> > Nux!
>> >> > >> > www.nux.ro
>> >> > >> >
>> >> > >> > ----- Original Message -----
>> >> > >> > > From: "Nux!" <nu...@li.nux.ro>
>> >> > >> > > To: users@cloudstack.apache.org
>> >> > >> > > Sent: Monday, 3 November, 2014 13:53:10
>> >> > >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> >> > >> >
>> >> > >> > > Hello,
>> >> > >> > >
>> >> > >> > > I think you are hitting a problem I also had:
>> >> > >> > > "2014-11-03 08:16:17,024 ERROR
>> [c.c.v.VirtualMachineManagerImpl]
>> >> > >> > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556
>> >> ctx-fcb072b2)
>> >> > >> > Failed to
>> >> > >> > > start instance VM[DomainRouter|r-4758-VM]
>> >> > >> > > com.cloud.utils.exception.ExecutionException: Unable to start
>> >> > >> > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart, not
>> >> > retrying"
>> >> > >> > >
>> >> > >> > > It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781
>> >> > >> > >
>> >> > >> > > Basically the systemvm template has some problems, you need to
>> >> > somehow
>> >> > >> > copy
>> >> > >> > > /opt/cloud/bin/createIpAlias.sh to
>> /opt/cloud/bin/createipAlias.sh
>> >> > to
>> >> > >> > get it
>> >> > >> > > working.
>> >> > >> > >
>> >> > >> > > --
>> >> > >> > > Sent from the Delta quadrant using Borg technology!
>> >> > >> > >
>> >> > >> > > Nux!
>> >> > >> > > www.nux.ro
>> >> > >> > >
>> >> > >> > > ----- Original Message -----
>> >> > >> > >> From: "Adam" <ad...@gmail.com>
>> >> > >> > >> To: users@cloudstack.apache.org
>> >> > >> > >> Sent: Monday, 3 November, 2014 13:32:12
>> >> > >> > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> >> > >> > >
>> >> > >> > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release
>> >> > >> > >> CentOS release 6.6 (Final)
>> >> > >> > >> [root@cs-east-prod1 ~]# uname -a
>> >> > >> > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15
>> >> > 04:27:16
>> >> > >> UTC
>> >> > >> > >> 2014 x86_64 x86_64 x86_64 GNU/Linux
>> >> > >> > >> [root@cs-east-prod1 ~]# virsh --version
>> >> > >> > >> 0.10.2
>> >> > >> > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
>> >> > >> > >> /etc/cloudstack/agent/log4j-cloud.xml
>> >> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management stop
>> >> > >> > >> Stopping cloudstack-management: [
>> >> OK ]
>> >> > >> > >> [root@cs-east-prod1 ~]# mv
>> >> > >> > >> /var/log/cloudstack/management/management-server.log
>> >> > >> > >> /var/log/cloudstack/management/management-server.bak
>> >> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management start
>> >> > >> > >> Starting cloudstack-management: [
>> >> OK ]
>> >> > >> > >> [root@cs-east-prod1 ~]#
>> >> > >> > >>
>> >> > >> > >> Please see attached ZIP of clean management-server.log in
>> DEBUG
>> >> > mode.
>> >> > >> At
>> >> > >> > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the attached
>> >> > >> > >> management-server.log*) I tried starting the Virtual Router
>> from
>> >> > the
>> >> > >> > >> Management UI and received this "Status" message in the UI.
>> >> > >> > >>
>> >> > >> > >> [image: Inline image 1]
>> >> > >> > >>
>> >> > >> > >> I can see in the management-server.log that while the
>> >> > >> > cloudstack-management
>> >> > >> > >> service is initializing, it throws the following ERROR at
>> exactly
>> >> > >> > (*2014-11-03
>> >> > >> > >> 08:14:36,813 on line 342 of the attached
>> management-server.log*):
>> >> > >> > >>
>> >> > >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
>> >> > (main:null)
>> >> > >> > Unable
>> >> > >> > >> to ping management server at 10.97.38.10:9090 due to
>> >> > ConnectException
>> >> > >> > >> java.net.ConnectException: Connection refused
>> >> > >> > >> at sun.nio.ch.Net.connect0(Native Method)
>> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:465)
>> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:457)
>> >> > >> > >> at
>> >> > sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> >
>> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >>
>> >> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
>> >> > >> > >> at
>> >> > >> >
>> >> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
>> >> > >> > >> at
>> >> > >> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
>> >> > >> > >> at
>> >> > >> >
>> >> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
>> >> > >> > >> at
>> >> > >> org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
>> >> > >> > >> at
>> >> > >> >
>> >> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >>
>> >> org.apache.catalina.core.StandardService.start(StandardService.java:516)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
>> >> > >> > >> at
>> >> org.apache.catalina.startup.Catalina.start(Catalina.java:593)
>> >> > >> > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> >> > >> > >> at
>> >> > >> > >>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >> > >> > >> at java.lang.reflect.Method.invoke(Method.java:606)
>> >> > >> > >> at
>> >> > org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
>> >> > >> > >> at
>> >> > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
>> >> > >> > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl]
>> >> > (main:null)
>> >> > >> > >> Detected that another management node with the same IP
>> >> 10.97.38.10
>> >> > is
>> >> > >> > >> considered as running in DB, however it is not pingable, we
>> will
>> >> > >> > continue
>> >> > >> > >> cluster initialization with this management server node
>> >> > >> > >>
>> >> > >> > >> I'm not sure what that error means, but more than likely
>> that's
>> >> the
>> >> > >> > reason
>> >> > >> > >> why the Virtual Router will not start.
>> >> > >> > >>
>> >> > >> > >> * PLEASE HELP ! *
>> >> > >> > >>
>> >> > >> > >> Best Regards,
>> >> > >> > >>
>> >> > >> > >>
>> >> > >> > >>
>> >> > >> > >> Adam Scarcella
>> >> > >> > >>
>> >> > >> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro> wrote:
>> >> > >> > >>
>> >> > >> > >>> Hi,
>> >> > >> > >>>
>> >> > >> > >>> More agent and management logs please, make sure they are in
>> >> > DEBUG[1]
>> >> > >> > mode.
>> >> > >> > >>>
>> >> > >> > >>> [1] -
>> >> > >> > >>>
>> >> > >>
>> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
>> >> > >> > >>>
>> >> > >> > >>> --
>> >> > >> > >>> Sent from the Delta quadrant using Borg technology!
>> >> > >> > >>>
>> >> > >> > >>> Nux!
>> >> > >> > >>> www.nux.ro
>> >> > >> > >>>
>> >> > >> > >>> ----- Original Message -----
>> >> > >> > >>> > From: "Adam" <ad...@gmail.com>
>> >> > >> > >>> > To: users@cloudstack.apache.org
>> >> > >> > >>> > Sent: Sunday, 2 November, 2014 23:33:22
>> >> > >> > >>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> >> > >> > >>>
>> >> > >> > >>> > I've performed a complete upgrade from 4.3 to 4.4.1 and
>> >> followed
>> >> > >> the
>> >> > >> > >>> > instructions perfectly, but my Virtual Router will not
>> start,
>> >> > nor
>> >> > >> am
>> >> > >> > I
>> >> > >> > >>> able
>> >> > >> > >>> > to "upgrade" it.
>> >> > >> > >>> >
>> >> > >> > >>> > I tried destroying the Virtual Router, starting any VM
>> >> Instance,
>> >> > >> and
>> >> > >> > >>> > allowing CS to recreate the VR. However that didn't help.
>> The
>> >> VR
>> >> > >> > still
>> >> > >> > >>> did
>> >> > >> > >>> > not start and it was still saying that it required to be
>> >> > upgraded.
>> >> > >> > >>> >
>> >> > >> > >>> > [image: Inline image 1]
>> >> > >> > >>> > [image: Inline image 2]
>> >> > >> > >>> > [image: Inline image 3]
>> >> > >> > >>> >
>> >> > >> > >>> > I know there are manual steps to finalize the systemvm
>> >> upgrade,
>> >> > but
>> >> > >> > I did
>> >> > >> > >>> > those steps:
>> >> > >> > >>> >
>> >> > >> > >>> >
>> >> > >> > >>>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
>> >> > >> > >>> >
>> >> > >> > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
>> >> > >> > >>> > Enter password:
>> >> > >> > >>> > Welcome to the MySQL monitor. Commands end with ; or \g.
>> >> > >> > >>> > Your MySQL connection id is 174
>> >> > >> > >>> > Server version: 5.1.73-log Source distribution
>> >> > >> > >>> >
>> >> > >> > >>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. All
>> >> > rights
>> >> > >> > >>> reserved.
>> >> > >> > >>> >
>> >> > >> > >>> > Oracle is a registered trademark of Oracle Corporation
>> and/or
>> >> > its
>> >> > >> > >>> > affiliates. Other names may be trademarks of their
>> respective
>> >> > >> > >>> > owners.
>> >> > >> > >>> >
>> >> > >> > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the
>> current
>> >> > input
>> >> > >> > >>> > statement.
>> >> > >> > >>> >
>> >> > >> > >>> > mysql> select id,name from cloud.vm_template where name =
>> >> > >> > >>> > 'systemvm-kvm-4.4';
>> >> > >> > >>> > +-----+------------------+
>> >> > >> > >>> >| id | name |
>> >> > >> > >>> > +-----+------------------+
>> >> > >> > >>> >| 274 | systemvm-kvm-4.4 |
>> >> > >> > >>> > +-----+------------------+
>> >> > >> > >>> > 1 row in set (0.00 sec)
>> >> > >> > >>> >
>> >> > >> > >>> > mysql> select type from cloud.vm_template where id='274';
>> >> > >> > >>> > +--------+
>> >> > >> > >>> >| type |
>> >> > >> > >>> > +--------+
>> >> > >> > >>> >| SYSTEM |
>> >> > >> > >>> > +--------+
>> >> > >> > >>> > 1 row in set (0.00 sec)
>> >> > >> > >>> >
>> >> > >> > >>> > mysql> select distinct(vm_template_id) from
>> cloud.vm_instance
>> >> > where
>> >> > >> > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
>> >> > >> > >>> > +----------------+
>> >> > >> > >>> >| vm_template_id |
>> >> > >> > >>> > +----------------+
>> >> > >> > >>> >| 274 |
>> >> > >> > >>> > +----------------+
>> >> > >> > >>> > 1 row in set (0.00 sec)
>> >> > >> > >>> >
>> >> > >> > >>> > mysql> select distinct(vm_template_id) from
>> cloud.vm_instance
>> >> > where
>> >> > >> > >>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM';
>> >> > >> > >>> > +----------------+
>> >> > >> > >>> >| vm_template_id |
>> >> > >> > >>> > +----------------+
>> >> > >> > >>> >| 274 |
>> >> > >> > >>> > +----------------+
>> >> > >> > >>> > 1 row in set (0.03 sec)
>> >> > >> > >>> >
>> >> > >> > >>> > mysql> select value from cloud.configuration where name =
>> >> > >> > >>> > 'router.template.kvm';
>> >> > >> > >>> > +------------------+
>> >> > >> > >>> >| value |
>> >> > >> > >>> > +------------------+
>> >> > >> > >>> >| systemvm-kvm-4.4 |
>> >> > >> > >>> > +------------------+
>> >> > >> > >>> > 1 row in set (0.00 sec)
>> >> > >> > >>> >
>> >> > >> > >>> > mysql>
>> >> > >> > >>> >
>> >> > >> > >>> > * PLEASE HELP ! *
>> >> > >> > >>> >
>> >> > >> > >>> > Best Regards,
>> >> > >> > >>> >
>> >> > >> > > >> > Adam Scarcella
>> >> > >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Thanks for your support,
>> >> Motty
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Adam <ad...@gmail.com>.
>
> Not sure if this operation that you performed replaces the current
> systemvm template and *especially if it replaces it also on the
> hypervisor where it is running.*
>
That procedure is the one documented in the official CloudStack
documentation so I hope it actually replaces the systemvm template. The
final output from that command tells it was successful:
Successfully installed system VM template to
/export/secondary/template/tmpl/1/274/
Also, as stated before, the VR is not running on any hypervisor / host
because I cannot get it to run, so I'm unsure what you mean when you
say "*especially
if it replaces it also on the hypervisor where it is running.*"
One more thing I would try is to make sure the VR on the HV is also using
> this image. You can try to compare md5sums of the image on the secondary
> storage and also the backend file for the VR's disk image (qemu-img info
> will show it to you) on the HV.
> If the sums differ then I would try to replace the VR disk image with my
> template directly on the HV and see if that does it.
I really don't understand what you're asking me to do.
Best Regards,
Adam Scarcella
On Mon, Nov 3, 2014 at 11:50 AM, Nux! <nu...@li.nux.ro> wrote:
> Adam,
>
> Not sure if this operation that you performed replaces the current
> systemvm template and especially if it replaces it also on the hypervisor
> where it is running.
>
> One more thing I would try is to make sure the VR on the HV is also using
> this image. You can try to compare md5sums of the image on the secondary
> storage and also the backend file for the VR's disk image (qemu-img info
> will show it to you) on the HV.
> If the sums differ then I would try to replace the VR disk image with my
> template directly on the HV and see if that does it.
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
> > From: "Adam" <ad...@gmail.com>
> > To: users@cloudstack.apache.org
> > Sent: Monday, 3 November, 2014 16:08:05
> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>
> > Thanks HTH,
> >
> > I decided to try to use your already modified qcow2 file:
> >
> > [root@cs-east-prod1 /]#
> >
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> > -m /export/secondary -u
> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> > -h kvm -F
> > --2014-11-03 10:11:32--
> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> > Resolving tmp.nux.ro... 217.19.15.108
> > Connecting to tmp.nux.ro|217.19.15.108|:80... connected.
> > HTTP request sent, awaiting response... 200 OK
> > Length: 305498799 (291M) [application/x-bzip2]
> > Saving to:
> >
> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
> >
> >
> 100%[===================================================================================================================>]
> > 305,498,799 9.02M/s in 37s
> >
> > 2014-11-03 10:12:10 (7.82 MB/s) -
> >
> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
> > saved [305498799/305498799]
> >
> > Uncompressing to
> >
> /usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2.tmp
> > (type bz2)...could take a long time
> > Moving to
> >
> /export/secondary/template/tmpl/1/274///00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2...could
> > take a while
> > Successfully installed system VM template to
> > /export/secondary/template/tmpl/1/274/
> > [root@cs-east-prod1 /]#
> >
> > Then I went into the Admin GUI and tried to start the VR, but it failed
> > with the exact same message:
> >
> > [image: Inline image 1]
> >
> > So I destroyed the Virtual Router and stopped and started the following
> > services:
> >
> > [root@cs-east-prod1 /]# service iptables stop
> > iptables: Setting chains to policy ACCEPT: nat filter [ OK ]
> > iptables: Flushing firewall rules: [ OK ]
> > iptables: Unloading modules: [ OK ]
> > [root@cs-east-prod1 /]# service cloudstack-management stop
> > Stopping cloudstack-management: [ OK ]
> > [root@cs-east-prod1 /]# service cloudstack-agent stop
> > Stopping Cloud Agent:
> > [root@cs-east-prod1 /]# service cloudstack-usage stop
> > Stopping CloudStack Usage Monitor cloudstack-usage [ OK ]
> > [root@cs-east-prod1 /]# service cloudstack-management start
> > Starting cloudstack-management: [ OK ]
> > [root@cs-east-prod1 /]# service cloudstack-usage start
> > Starting CloudStack Usage Monitor cloudstack-usage [ OK ]
> > [root@cs-east-prod1 /]# service cloudstack-agent start
> > Starting Cloud Agent:
> > [root@cs-east-prod1 /]#
> >
> > I then tried starting any VM instance to have CS automatically re-create
> > the VR, but that failed as well.
> >
> > Motty, I also looked at trying to increase the RAM and CPU for the router
> > but only saw one option
> >
> > [image: Inline image 2]
> >
> > So I created a new Service Offering with twice as much resources:
> >
> > [image: Inline image 3]
> >
> > Then upgraded the service offering for the VR
> >
> > [image: Inline image 4]
> >
> > [image: Inline image 6]
> >
> > But it failed still with the exact same error:
> >
> > [image: Inline image 7]
> >
> > Best Regards,
> >
> >
> >
> > Adam Scarcella
> >
> > On Mon, Nov 3, 2014 at 10:23 AM, motty cruz <mo...@gmail.com>
> wrote:
> >
> >> Adam,
> >> what are the specifications of the VR? can you upgrade the RAM and CPU
> and
> >> try to restart that VR? I had similar issue and giving the VR more RAM
> and
> >> CPU will start the VR.
> >>
> >> Thanks,
> >> Motty
> >>
> >> On Mon, Nov 3, 2014 at 7:04 AM, Nux! <nu...@li.nux.ro> wrote:
> >>
> >> > Adam,
> >> >
> >> > You can modify the qcow2 file that is in your secondary storage, i.e.
> you
> >> > should mount that storage somewhere and go to
> >> > /mount/point/template/tmpl/274 and modify the qcow2 file.
> >> >
> >> > Alternatively you could use my already modified copy to overwrite your
> >> > uuid.qcow2 with (at your own risk of course):
> >> > http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> >> >
> >> > HTH
> >> >
> >> > --
> >> > Sent from the Delta quadrant using Borg technology!
> >> >
> >> > Nux!
> >> > www.nux.ro
> >> >
> >> > ----- Original Message -----
> >> > > From: "Adam" <ad...@gmail.com>
> >> > > To: users@cloudstack.apache.org
> >> > > Sent: Monday, 3 November, 2014 14:58:58
> >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >> >
> >> > > [root@cs-east-prod1 management]# netstat -a|grep 9090
> >> > > unix 3 [ ] STREAM CONNECTED 29090
> >> > > [root@cs-east-prod1 management]# nmap localhost
> >> > >
> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47 EST
> >> > > Nmap scan report for localhost (127.0.0.1)
> >> > > Host is up (0.000015s latency).
> >> > > Other addresses for localhost (not scanned): 127.0.0.1
> >> > > Not shown: 990 closed ports
> >> > > PORT STATE SERVICE
> >> > > 22/tcp open ssh
> >> > > 25/tcp open smtp
> >> > > 80/tcp open http
> >> > > 111/tcp open rpcbind
> >> > > 199/tcp open smux
> >> > > 631/tcp open ipp
> >> > > 2049/tcp open nfs
> >> > > 3306/tcp open mysql
> >> > > 8080/tcp open http-proxy
> >> > > 9090/tcp open zeus-admin
> >> > >
> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
> >> > > [root@cs-east-prod1 management]# telnet localhost 9090
> >> > > Trying ::1...
> >> > > Connected to localhost.
> >> > > Escape character is '^]'.
> >> > > Connection closed by foreign host.
> >> > > [root@cs-east-prod1 management]# nmap 10.97.38.10
> >> > >
> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49 EST
> >> > > Nmap scan report for cs-east-prod1.na.tibco.com (10.97.38.10)
> >> > > Host is up (0.000015s latency).
> >> > > Not shown: 993 closed ports
> >> > > PORT STATE SERVICE
> >> > > 22/tcp open ssh
> >> > > 80/tcp open http
> >> > > 111/tcp open rpcbind
> >> > > 2049/tcp open nfs
> >> > > 3306/tcp open mysql
> >> > > 8080/tcp open http-proxy
> >> > > 9090/tcp open zeus-admin
> >> > >
> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
> >> > > [root@cs-east-prod1 management]# telnet 10.97.38.10 9090
> >> > > Trying 10.97.38.10...
> >> > > Connected to 10.97.38.10.
> >> > > Escape character is '^]'.
> >> > > Connection closed by foreign host.
> >> > > [root@cs-east-prod1 management]#
> >> > >
> >> > > Also, I checked for the /opt/cloud/bin/ directory on my Management
> >> Host,
> >> > > but it's not there:
> >> > >
> >> > > [root@cs-east-prod1 management]# cd /opt/cloud/bin/
> >> > > -bash: cd: /opt/cloud/bin/: No such file or directory
> >> > > [root@cs-east-prod1 management]# cd /
> >> > > [root@cs-east-prod1 /]# find . -name "createipAlias.sh"
> >> > > [root@cs-east-prod1 /]# find . -name "createIpAlias.sh"
> >> > > [root@cs-east-prod1 /]#
> >> > >
> >> > > Are you saying that the /opt/cloud/bin/createipAlias.sh script is
> >> inside
> >> > > the Virtual Router VM? If so, how could I possibly change that
> since it
> >> > > won't start?
> >> > >
> >> > > Best Regards,
> >> > >
> >> > >
> >> > >
> >> > > Adam Scarcella
> >> > >
> >> > > On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion <
> pdion891@apache.org>
> >> > wrote:
> >> > >
> >> > >> Could it be possible the firewall is blocking port 9090 from the
> >> > >> management-server?
> >> > >>
> >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
> (main:null)
> >> > Unable
> >> > >> to ping management server at 10.97.38.10:9090 due to
> ConnectException
> >> > >> java.net.ConnectException: Connection refused
> >> > >>
> >> > >> On Mon, Nov 3, 2014 at 9:02 AM, Nux! <nu...@li.nux.ro> wrote:
> >> > >>
> >> > >> > BTW the agent logs should help clarify this.
> >> > >> > The aforementioned changes must be done either in the current
> >> > (broken) VR
> >> > >> > or in the systemvm (after which you should redeploy the VR to
> pick
> >> up
> >> > the
> >> > >> > change).
> >> > >> >
> >> > >> > HTH
> >> > >> > Lucian
> >> > >> >
> >> > >> > --
> >> > >> > Sent from the Delta quadrant using Borg technology!
> >> > >> >
> >> > >> > Nux!
> >> > >> > www.nux.ro
> >> > >> >
> >> > >> > ----- Original Message -----
> >> > >> > > From: "Nux!" <nu...@li.nux.ro>
> >> > >> > > To: users@cloudstack.apache.org
> >> > >> > > Sent: Monday, 3 November, 2014 13:53:10
> >> > >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >> > >> >
> >> > >> > > Hello,
> >> > >> > >
> >> > >> > > I think you are hitting a problem I also had:
> >> > >> > > "2014-11-03 08:16:17,024 ERROR
> [c.c.v.VirtualMachineManagerImpl]
> >> > >> > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556
> >> ctx-fcb072b2)
> >> > >> > Failed to
> >> > >> > > start instance VM[DomainRouter|r-4758-VM]
> >> > >> > > com.cloud.utils.exception.ExecutionException: Unable to start
> >> > >> > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart, not
> >> > retrying"
> >> > >> > >
> >> > >> > > It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781
> >> > >> > >
> >> > >> > > Basically the systemvm template has some problems, you need to
> >> > somehow
> >> > >> > copy
> >> > >> > > /opt/cloud/bin/createIpAlias.sh to
> /opt/cloud/bin/createipAlias.sh
> >> > to
> >> > >> > get it
> >> > >> > > working.
> >> > >> > >
> >> > >> > > --
> >> > >> > > Sent from the Delta quadrant using Borg technology!
> >> > >> > >
> >> > >> > > Nux!
> >> > >> > > www.nux.ro
> >> > >> > >
> >> > >> > > ----- Original Message -----
> >> > >> > >> From: "Adam" <ad...@gmail.com>
> >> > >> > >> To: users@cloudstack.apache.org
> >> > >> > >> Sent: Monday, 3 November, 2014 13:32:12
> >> > >> > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >> > >> > >
> >> > >> > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release
> >> > >> > >> CentOS release 6.6 (Final)
> >> > >> > >> [root@cs-east-prod1 ~]# uname -a
> >> > >> > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15
> >> > 04:27:16
> >> > >> UTC
> >> > >> > >> 2014 x86_64 x86_64 x86_64 GNU/Linux
> >> > >> > >> [root@cs-east-prod1 ~]# virsh --version
> >> > >> > >> 0.10.2
> >> > >> > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
> >> > >> > >> /etc/cloudstack/agent/log4j-cloud.xml
> >> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management stop
> >> > >> > >> Stopping cloudstack-management: [
> >> OK ]
> >> > >> > >> [root@cs-east-prod1 ~]# mv
> >> > >> > >> /var/log/cloudstack/management/management-server.log
> >> > >> > >> /var/log/cloudstack/management/management-server.bak
> >> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management start
> >> > >> > >> Starting cloudstack-management: [
> >> OK ]
> >> > >> > >> [root@cs-east-prod1 ~]#
> >> > >> > >>
> >> > >> > >> Please see attached ZIP of clean management-server.log in
> DEBUG
> >> > mode.
> >> > >> At
> >> > >> > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the attached
> >> > >> > >> management-server.log*) I tried starting the Virtual Router
> from
> >> > the
> >> > >> > >> Management UI and received this "Status" message in the UI.
> >> > >> > >>
> >> > >> > >> [image: Inline image 1]
> >> > >> > >>
> >> > >> > >> I can see in the management-server.log that while the
> >> > >> > cloudstack-management
> >> > >> > >> service is initializing, it throws the following ERROR at
> exactly
> >> > >> > (*2014-11-03
> >> > >> > >> 08:14:36,813 on line 342 of the attached
> management-server.log*):
> >> > >> > >>
> >> > >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
> >> > (main:null)
> >> > >> > Unable
> >> > >> > >> to ping management server at 10.97.38.10:9090 due to
> >> > ConnectException
> >> > >> > >> java.net.ConnectException: Connection refused
> >> > >> > >> at sun.nio.ch.Net.connect0(Native Method)
> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:465)
> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:457)
> >> > >> > >> at
> >> > sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> >
> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
> >> > >> > >> at
> >> > >> > >>
> >> > >>
> >> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
> >> > >> > >> at
> >> > >> >
> >> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
> >> > >> > >> at
> >> > >> > >>
> >> > org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
> >> > >> > >> at
> >> > >> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> >> > >> > >> at
> >> > >> >
> >> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
> >> > >> > >> at
> >> > >> org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
> >> > >> > >> at
> >> > >> >
> >> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
> >> > >> > >> at
> >> > >> > >>
> >> > org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
> >> > >> > >> at
> >> > >> > >>
> >> > >>
> >> org.apache.catalina.core.StandardService.start(StandardService.java:516)
> >> > >> > >> at
> >> > >> > >>
> >> > org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
> >> > >> > >> at
> >> org.apache.catalina.startup.Catalina.start(Catalina.java:593)
> >> > >> > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> > >> > >> at
> >> > >> > >>
> >> > >> >
> >> > >>
> >> >
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> > >> > >> at java.lang.reflect.Method.invoke(Method.java:606)
> >> > >> > >> at
> >> > org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> >> > >> > >> at
> >> > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> >> > >> > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl]
> >> > (main:null)
> >> > >> > >> Detected that another management node with the same IP
> >> 10.97.38.10
> >> > is
> >> > >> > >> considered as running in DB, however it is not pingable, we
> will
> >> > >> > continue
> >> > >> > >> cluster initialization with this management server node
> >> > >> > >>
> >> > >> > >> I'm not sure what that error means, but more than likely
> that's
> >> the
> >> > >> > reason
> >> > >> > >> why the Virtual Router will not start.
> >> > >> > >>
> >> > >> > >> * PLEASE HELP ! *
> >> > >> > >>
> >> > >> > >> Best Regards,
> >> > >> > >>
> >> > >> > >>
> >> > >> > >>
> >> > >> > >> Adam Scarcella
> >> > >> > >>
> >> > >> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro> wrote:
> >> > >> > >>
> >> > >> > >>> Hi,
> >> > >> > >>>
> >> > >> > >>> More agent and management logs please, make sure they are in
> >> > DEBUG[1]
> >> > >> > mode.
> >> > >> > >>>
> >> > >> > >>> [1] -
> >> > >> > >>>
> >> > >>
> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
> >> > >> > >>>
> >> > >> > >>> --
> >> > >> > >>> Sent from the Delta quadrant using Borg technology!
> >> > >> > >>>
> >> > >> > >>> Nux!
> >> > >> > >>> www.nux.ro
> >> > >> > >>>
> >> > >> > >>> ----- Original Message -----
> >> > >> > >>> > From: "Adam" <ad...@gmail.com>
> >> > >> > >>> > To: users@cloudstack.apache.org
> >> > >> > >>> > Sent: Sunday, 2 November, 2014 23:33:22
> >> > >> > >>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >> > >> > >>>
> >> > >> > >>> > I've performed a complete upgrade from 4.3 to 4.4.1 and
> >> followed
> >> > >> the
> >> > >> > >>> > instructions perfectly, but my Virtual Router will not
> start,
> >> > nor
> >> > >> am
> >> > >> > I
> >> > >> > >>> able
> >> > >> > >>> > to "upgrade" it.
> >> > >> > >>> >
> >> > >> > >>> > I tried destroying the Virtual Router, starting any VM
> >> Instance,
> >> > >> and
> >> > >> > >>> > allowing CS to recreate the VR. However that didn't help.
> The
> >> VR
> >> > >> > still
> >> > >> > >>> did
> >> > >> > >>> > not start and it was still saying that it required to be
> >> > upgraded.
> >> > >> > >>> >
> >> > >> > >>> > [image: Inline image 1]
> >> > >> > >>> > [image: Inline image 2]
> >> > >> > >>> > [image: Inline image 3]
> >> > >> > >>> >
> >> > >> > >>> > I know there are manual steps to finalize the systemvm
> >> upgrade,
> >> > but
> >> > >> > I did
> >> > >> > >>> > those steps:
> >> > >> > >>> >
> >> > >> > >>> >
> >> > >> > >>>
> >> > >> >
> >> > >>
> >> >
> >>
> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
> >> > >> > >>> >
> >> > >> > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
> >> > >> > >>> > Enter password:
> >> > >> > >>> > Welcome to the MySQL monitor. Commands end with ; or \g.
> >> > >> > >>> > Your MySQL connection id is 174
> >> > >> > >>> > Server version: 5.1.73-log Source distribution
> >> > >> > >>> >
> >> > >> > >>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. All
> >> > rights
> >> > >> > >>> reserved.
> >> > >> > >>> >
> >> > >> > >>> > Oracle is a registered trademark of Oracle Corporation
> and/or
> >> > its
> >> > >> > >>> > affiliates. Other names may be trademarks of their
> respective
> >> > >> > >>> > owners.
> >> > >> > >>> >
> >> > >> > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the
> current
> >> > input
> >> > >> > >>> > statement.
> >> > >> > >>> >
> >> > >> > >>> > mysql> select id,name from cloud.vm_template where name =
> >> > >> > >>> > 'systemvm-kvm-4.4';
> >> > >> > >>> > +-----+------------------+
> >> > >> > >>> >| id | name |
> >> > >> > >>> > +-----+------------------+
> >> > >> > >>> >| 274 | systemvm-kvm-4.4 |
> >> > >> > >>> > +-----+------------------+
> >> > >> > >>> > 1 row in set (0.00 sec)
> >> > >> > >>> >
> >> > >> > >>> > mysql> select type from cloud.vm_template where id='274';
> >> > >> > >>> > +--------+
> >> > >> > >>> >| type |
> >> > >> > >>> > +--------+
> >> > >> > >>> >| SYSTEM |
> >> > >> > >>> > +--------+
> >> > >> > >>> > 1 row in set (0.00 sec)
> >> > >> > >>> >
> >> > >> > >>> > mysql> select distinct(vm_template_id) from
> cloud.vm_instance
> >> > where
> >> > >> > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
> >> > >> > >>> > +----------------+
> >> > >> > >>> >| vm_template_id |
> >> > >> > >>> > +----------------+
> >> > >> > >>> >| 274 |
> >> > >> > >>> > +----------------+
> >> > >> > >>> > 1 row in set (0.00 sec)
> >> > >> > >>> >
> >> > >> > >>> > mysql> select distinct(vm_template_id) from
> cloud.vm_instance
> >> > where
> >> > >> > >>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM';
> >> > >> > >>> > +----------------+
> >> > >> > >>> >| vm_template_id |
> >> > >> > >>> > +----------------+
> >> > >> > >>> >| 274 |
> >> > >> > >>> > +----------------+
> >> > >> > >>> > 1 row in set (0.03 sec)
> >> > >> > >>> >
> >> > >> > >>> > mysql> select value from cloud.configuration where name =
> >> > >> > >>> > 'router.template.kvm';
> >> > >> > >>> > +------------------+
> >> > >> > >>> >| value |
> >> > >> > >>> > +------------------+
> >> > >> > >>> >| systemvm-kvm-4.4 |
> >> > >> > >>> > +------------------+
> >> > >> > >>> > 1 row in set (0.00 sec)
> >> > >> > >>> >
> >> > >> > >>> > mysql>
> >> > >> > >>> >
> >> > >> > >>> > * PLEASE HELP ! *
> >> > >> > >>> >
> >> > >> > >>> > Best Regards,
> >> > >> > >>> >
> >> > >> > > >> > Adam Scarcella
> >> > >> >
> >> >
> >>
> >>
> >>
> >> --
> >> Thanks for your support,
> >> Motty
>
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Nux! <nu...@li.nux.ro>.
Adam,
Not sure if this operation that you performed replaces the current systemvm template and especially if it replaces it also on the hypervisor where it is running.
One more thing I would try is to make sure the VR on the HV is also using this image. You can try to compare md5sums of the image on the secondary storage and also the backend file for the VR's disk image (qemu-img info will show it to you) on the HV.
If the sums differ then I would try to replace the VR disk image with my template directly on the HV and see if that does it.
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Adam" <ad...@gmail.com>
> To: users@cloudstack.apache.org
> Sent: Monday, 3 November, 2014 16:08:05
> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> Thanks HTH,
>
> I decided to try to use your already modified qcow2 file:
>
> [root@cs-east-prod1 /]#
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> -m /export/secondary -u http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> -h kvm -F
> --2014-11-03 10:11:32-- http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> Resolving tmp.nux.ro... 217.19.15.108
> Connecting to tmp.nux.ro|217.19.15.108|:80... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 305498799 (291M) [application/x-bzip2]
> Saving to:
> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
>
> 100%[===================================================================================================================>]
> 305,498,799 9.02M/s in 37s
>
> 2014-11-03 10:12:10 (7.82 MB/s) -
> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
> saved [305498799/305498799]
>
> Uncompressing to
> /usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2.tmp
> (type bz2)...could take a long time
> Moving to
> /export/secondary/template/tmpl/1/274///00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2...could
> take a while
> Successfully installed system VM template to
> /export/secondary/template/tmpl/1/274/
> [root@cs-east-prod1 /]#
>
> Then I went into the Admin GUI and tried to start the VR, but it failed
> with the exact same message:
>
> [image: Inline image 1]
>
> So I destroyed the Virtual Router and stopped and started the following
> services:
>
> [root@cs-east-prod1 /]# service iptables stop
> iptables: Setting chains to policy ACCEPT: nat filter [ OK ]
> iptables: Flushing firewall rules: [ OK ]
> iptables: Unloading modules: [ OK ]
> [root@cs-east-prod1 /]# service cloudstack-management stop
> Stopping cloudstack-management: [ OK ]
> [root@cs-east-prod1 /]# service cloudstack-agent stop
> Stopping Cloud Agent:
> [root@cs-east-prod1 /]# service cloudstack-usage stop
> Stopping CloudStack Usage Monitor cloudstack-usage [ OK ]
> [root@cs-east-prod1 /]# service cloudstack-management start
> Starting cloudstack-management: [ OK ]
> [root@cs-east-prod1 /]# service cloudstack-usage start
> Starting CloudStack Usage Monitor cloudstack-usage [ OK ]
> [root@cs-east-prod1 /]# service cloudstack-agent start
> Starting Cloud Agent:
> [root@cs-east-prod1 /]#
>
> I then tried starting any VM instance to have CS automatically re-create
> the VR, but that failed as well.
>
> Motty, I also looked at trying to increase the RAM and CPU for the router
> but only saw one option
>
> [image: Inline image 2]
>
> So I created a new Service Offering with twice as much resources:
>
> [image: Inline image 3]
>
> Then upgraded the service offering for the VR
>
> [image: Inline image 4]
>
> [image: Inline image 6]
>
> But it failed still with the exact same error:
>
> [image: Inline image 7]
>
> Best Regards,
>
>
>
> Adam Scarcella
>
> On Mon, Nov 3, 2014 at 10:23 AM, motty cruz <mo...@gmail.com> wrote:
>
>> Adam,
>> what are the specifications of the VR? can you upgrade the RAM and CPU and
>> try to restart that VR? I had similar issue and giving the VR more RAM and
>> CPU will start the VR.
>>
>> Thanks,
>> Motty
>>
>> On Mon, Nov 3, 2014 at 7:04 AM, Nux! <nu...@li.nux.ro> wrote:
>>
>> > Adam,
>> >
>> > You can modify the qcow2 file that is in your secondary storage, i.e. you
>> > should mount that storage somewhere and go to
>> > /mount/point/template/tmpl/274 and modify the qcow2 file.
>> >
>> > Alternatively you could use my already modified copy to overwrite your
>> > uuid.qcow2 with (at your own risk of course):
>> > http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
>> >
>> > HTH
>> >
>> > --
>> > Sent from the Delta quadrant using Borg technology!
>> >
>> > Nux!
>> > www.nux.ro
>> >
>> > ----- Original Message -----
>> > > From: "Adam" <ad...@gmail.com>
>> > > To: users@cloudstack.apache.org
>> > > Sent: Monday, 3 November, 2014 14:58:58
>> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> >
>> > > [root@cs-east-prod1 management]# netstat -a|grep 9090
>> > > unix 3 [ ] STREAM CONNECTED 29090
>> > > [root@cs-east-prod1 management]# nmap localhost
>> > >
>> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47 EST
>> > > Nmap scan report for localhost (127.0.0.1)
>> > > Host is up (0.000015s latency).
>> > > Other addresses for localhost (not scanned): 127.0.0.1
>> > > Not shown: 990 closed ports
>> > > PORT STATE SERVICE
>> > > 22/tcp open ssh
>> > > 25/tcp open smtp
>> > > 80/tcp open http
>> > > 111/tcp open rpcbind
>> > > 199/tcp open smux
>> > > 631/tcp open ipp
>> > > 2049/tcp open nfs
>> > > 3306/tcp open mysql
>> > > 8080/tcp open http-proxy
>> > > 9090/tcp open zeus-admin
>> > >
>> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
>> > > [root@cs-east-prod1 management]# telnet localhost 9090
>> > > Trying ::1...
>> > > Connected to localhost.
>> > > Escape character is '^]'.
>> > > Connection closed by foreign host.
>> > > [root@cs-east-prod1 management]# nmap 10.97.38.10
>> > >
>> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49 EST
>> > > Nmap scan report for cs-east-prod1.na.tibco.com (10.97.38.10)
>> > > Host is up (0.000015s latency).
>> > > Not shown: 993 closed ports
>> > > PORT STATE SERVICE
>> > > 22/tcp open ssh
>> > > 80/tcp open http
>> > > 111/tcp open rpcbind
>> > > 2049/tcp open nfs
>> > > 3306/tcp open mysql
>> > > 8080/tcp open http-proxy
>> > > 9090/tcp open zeus-admin
>> > >
>> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
>> > > [root@cs-east-prod1 management]# telnet 10.97.38.10 9090
>> > > Trying 10.97.38.10...
>> > > Connected to 10.97.38.10.
>> > > Escape character is '^]'.
>> > > Connection closed by foreign host.
>> > > [root@cs-east-prod1 management]#
>> > >
>> > > Also, I checked for the /opt/cloud/bin/ directory on my Management
>> Host,
>> > > but it's not there:
>> > >
>> > > [root@cs-east-prod1 management]# cd /opt/cloud/bin/
>> > > -bash: cd: /opt/cloud/bin/: No such file or directory
>> > > [root@cs-east-prod1 management]# cd /
>> > > [root@cs-east-prod1 /]# find . -name "createipAlias.sh"
>> > > [root@cs-east-prod1 /]# find . -name "createIpAlias.sh"
>> > > [root@cs-east-prod1 /]#
>> > >
>> > > Are you saying that the /opt/cloud/bin/createipAlias.sh script is
>> inside
>> > > the Virtual Router VM? If so, how could I possibly change that since it
>> > > won't start?
>> > >
>> > > Best Regards,
>> > >
>> > >
>> > >
>> > > Adam Scarcella
>> > >
>> > > On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion <pd...@apache.org>
>> > wrote:
>> > >
>> > >> Could it be possible the firewall is blocking port 9090 from the
>> > >> management-server?
>> > >>
>> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null)
>> > Unable
>> > >> to ping management server at 10.97.38.10:9090 due to ConnectException
>> > >> java.net.ConnectException: Connection refused
>> > >>
>> > >> On Mon, Nov 3, 2014 at 9:02 AM, Nux! <nu...@li.nux.ro> wrote:
>> > >>
>> > >> > BTW the agent logs should help clarify this.
>> > >> > The aforementioned changes must be done either in the current
>> > (broken) VR
>> > >> > or in the systemvm (after which you should redeploy the VR to pick
>> up
>> > the
>> > >> > change).
>> > >> >
>> > >> > HTH
>> > >> > Lucian
>> > >> >
>> > >> > --
>> > >> > Sent from the Delta quadrant using Borg technology!
>> > >> >
>> > >> > Nux!
>> > >> > www.nux.ro
>> > >> >
>> > >> > ----- Original Message -----
>> > >> > > From: "Nux!" <nu...@li.nux.ro>
>> > >> > > To: users@cloudstack.apache.org
>> > >> > > Sent: Monday, 3 November, 2014 13:53:10
>> > >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> > >> >
>> > >> > > Hello,
>> > >> > >
>> > >> > > I think you are hitting a problem I also had:
>> > >> > > "2014-11-03 08:16:17,024 ERROR [c.c.v.VirtualMachineManagerImpl]
>> > >> > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556
>> ctx-fcb072b2)
>> > >> > Failed to
>> > >> > > start instance VM[DomainRouter|r-4758-VM]
>> > >> > > com.cloud.utils.exception.ExecutionException: Unable to start
>> > >> > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart, not
>> > retrying"
>> > >> > >
>> > >> > > It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781
>> > >> > >
>> > >> > > Basically the systemvm template has some problems, you need to
>> > somehow
>> > >> > copy
>> > >> > > /opt/cloud/bin/createIpAlias.sh to /opt/cloud/bin/createipAlias.sh
>> > to
>> > >> > get it
>> > >> > > working.
>> > >> > >
>> > >> > > --
>> > >> > > Sent from the Delta quadrant using Borg technology!
>> > >> > >
>> > >> > > Nux!
>> > >> > > www.nux.ro
>> > >> > >
>> > >> > > ----- Original Message -----
>> > >> > >> From: "Adam" <ad...@gmail.com>
>> > >> > >> To: users@cloudstack.apache.org
>> > >> > >> Sent: Monday, 3 November, 2014 13:32:12
>> > >> > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> > >> > >
>> > >> > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release
>> > >> > >> CentOS release 6.6 (Final)
>> > >> > >> [root@cs-east-prod1 ~]# uname -a
>> > >> > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15
>> > 04:27:16
>> > >> UTC
>> > >> > >> 2014 x86_64 x86_64 x86_64 GNU/Linux
>> > >> > >> [root@cs-east-prod1 ~]# virsh --version
>> > >> > >> 0.10.2
>> > >> > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
>> > >> > >> /etc/cloudstack/agent/log4j-cloud.xml
>> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management stop
>> > >> > >> Stopping cloudstack-management: [
>> OK ]
>> > >> > >> [root@cs-east-prod1 ~]# mv
>> > >> > >> /var/log/cloudstack/management/management-server.log
>> > >> > >> /var/log/cloudstack/management/management-server.bak
>> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management start
>> > >> > >> Starting cloudstack-management: [
>> OK ]
>> > >> > >> [root@cs-east-prod1 ~]#
>> > >> > >>
>> > >> > >> Please see attached ZIP of clean management-server.log in DEBUG
>> > mode.
>> > >> At
>> > >> > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the attached
>> > >> > >> management-server.log*) I tried starting the Virtual Router from
>> > the
>> > >> > >> Management UI and received this "Status" message in the UI.
>> > >> > >>
>> > >> > >> [image: Inline image 1]
>> > >> > >>
>> > >> > >> I can see in the management-server.log that while the
>> > >> > cloudstack-management
>> > >> > >> service is initializing, it throws the following ERROR at exactly
>> > >> > (*2014-11-03
>> > >> > >> 08:14:36,813 on line 342 of the attached management-server.log*):
>> > >> > >>
>> > >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
>> > (main:null)
>> > >> > Unable
>> > >> > >> to ping management server at 10.97.38.10:9090 due to
>> > ConnectException
>> > >> > >> java.net.ConnectException: Connection refused
>> > >> > >> at sun.nio.ch.Net.connect0(Native Method)
>> > >> > >> at sun.nio.ch.Net.connect(Net.java:465)
>> > >> > >> at sun.nio.ch.Net.connect(Net.java:457)
>> > >> > >> at
>> > sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
>> > >> > >> at
>> > >> > >>
>> > >>
>> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
>> > >> > >> at
>> > >> >
>> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
>> > >> > >> at
>> > >> > >>
>> > org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
>> > >> > >> at
>> > >> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
>> > >> > >> at
>> > >> >
>> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
>> > >> > >> at
>> > >> org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
>> > >> > >> at
>> > >> >
>> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
>> > >> > >> at
>> > >> > >>
>> > org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
>> > >> > >> at
>> > >> > >>
>> > >>
>> org.apache.catalina.core.StandardService.start(StandardService.java:516)
>> > >> > >> at
>> > >> > >>
>> > org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
>> > >> > >> at
>> org.apache.catalina.startup.Catalina.start(Catalina.java:593)
>> > >> > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > >> > >> at java.lang.reflect.Method.invoke(Method.java:606)
>> > >> > >> at
>> > org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
>> > >> > >> at
>> > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
>> > >> > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl]
>> > (main:null)
>> > >> > >> Detected that another management node with the same IP
>> 10.97.38.10
>> > is
>> > >> > >> considered as running in DB, however it is not pingable, we will
>> > >> > continue
>> > >> > >> cluster initialization with this management server node
>> > >> > >>
>> > >> > >> I'm not sure what that error means, but more than likely that's
>> the
>> > >> > reason
>> > >> > >> why the Virtual Router will not start.
>> > >> > >>
>> > >> > >> * PLEASE HELP ! *
>> > >> > >>
>> > >> > >> Best Regards,
>> > >> > >>
>> > >> > >>
>> > >> > >>
>> > >> > >> Adam Scarcella
>> > >> > >>
>> > >> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro> wrote:
>> > >> > >>
>> > >> > >>> Hi,
>> > >> > >>>
>> > >> > >>> More agent and management logs please, make sure they are in
>> > DEBUG[1]
>> > >> > mode.
>> > >> > >>>
>> > >> > >>> [1] -
>> > >> > >>>
>> > >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
>> > >> > >>>
>> > >> > >>> --
>> > >> > >>> Sent from the Delta quadrant using Borg technology!
>> > >> > >>>
>> > >> > >>> Nux!
>> > >> > >>> www.nux.ro
>> > >> > >>>
>> > >> > >>> ----- Original Message -----
>> > >> > >>> > From: "Adam" <ad...@gmail.com>
>> > >> > >>> > To: users@cloudstack.apache.org
>> > >> > >>> > Sent: Sunday, 2 November, 2014 23:33:22
>> > >> > >>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> > >> > >>>
>> > >> > >>> > I've performed a complete upgrade from 4.3 to 4.4.1 and
>> followed
>> > >> the
>> > >> > >>> > instructions perfectly, but my Virtual Router will not start,
>> > nor
>> > >> am
>> > >> > I
>> > >> > >>> able
>> > >> > >>> > to "upgrade" it.
>> > >> > >>> >
>> > >> > >>> > I tried destroying the Virtual Router, starting any VM
>> Instance,
>> > >> and
>> > >> > >>> > allowing CS to recreate the VR. However that didn't help. The
>> VR
>> > >> > still
>> > >> > >>> did
>> > >> > >>> > not start and it was still saying that it required to be
>> > upgraded.
>> > >> > >>> >
>> > >> > >>> > [image: Inline image 1]
>> > >> > >>> > [image: Inline image 2]
>> > >> > >>> > [image: Inline image 3]
>> > >> > >>> >
>> > >> > >>> > I know there are manual steps to finalize the systemvm
>> upgrade,
>> > but
>> > >> > I did
>> > >> > >>> > those steps:
>> > >> > >>> >
>> > >> > >>> >
>> > >> > >>>
>> > >> >
>> > >>
>> >
>> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
>> > >> > >>> >
>> > >> > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
>> > >> > >>> > Enter password:
>> > >> > >>> > Welcome to the MySQL monitor. Commands end with ; or \g.
>> > >> > >>> > Your MySQL connection id is 174
>> > >> > >>> > Server version: 5.1.73-log Source distribution
>> > >> > >>> >
>> > >> > >>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. All
>> > rights
>> > >> > >>> reserved.
>> > >> > >>> >
>> > >> > >>> > Oracle is a registered trademark of Oracle Corporation and/or
>> > its
>> > >> > >>> > affiliates. Other names may be trademarks of their respective
>> > >> > >>> > owners.
>> > >> > >>> >
>> > >> > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the current
>> > input
>> > >> > >>> > statement.
>> > >> > >>> >
>> > >> > >>> > mysql> select id,name from cloud.vm_template where name =
>> > >> > >>> > 'systemvm-kvm-4.4';
>> > >> > >>> > +-----+------------------+
>> > >> > >>> >| id | name |
>> > >> > >>> > +-----+------------------+
>> > >> > >>> >| 274 | systemvm-kvm-4.4 |
>> > >> > >>> > +-----+------------------+
>> > >> > >>> > 1 row in set (0.00 sec)
>> > >> > >>> >
>> > >> > >>> > mysql> select type from cloud.vm_template where id='274';
>> > >> > >>> > +--------+
>> > >> > >>> >| type |
>> > >> > >>> > +--------+
>> > >> > >>> >| SYSTEM |
>> > >> > >>> > +--------+
>> > >> > >>> > 1 row in set (0.00 sec)
>> > >> > >>> >
>> > >> > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance
>> > where
>> > >> > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
>> > >> > >>> > +----------------+
>> > >> > >>> >| vm_template_id |
>> > >> > >>> > +----------------+
>> > >> > >>> >| 274 |
>> > >> > >>> > +----------------+
>> > >> > >>> > 1 row in set (0.00 sec)
>> > >> > >>> >
>> > >> > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance
>> > where
>> > >> > >>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM';
>> > >> > >>> > +----------------+
>> > >> > >>> >| vm_template_id |
>> > >> > >>> > +----------------+
>> > >> > >>> >| 274 |
>> > >> > >>> > +----------------+
>> > >> > >>> > 1 row in set (0.03 sec)
>> > >> > >>> >
>> > >> > >>> > mysql> select value from cloud.configuration where name =
>> > >> > >>> > 'router.template.kvm';
>> > >> > >>> > +------------------+
>> > >> > >>> >| value |
>> > >> > >>> > +------------------+
>> > >> > >>> >| systemvm-kvm-4.4 |
>> > >> > >>> > +------------------+
>> > >> > >>> > 1 row in set (0.00 sec)
>> > >> > >>> >
>> > >> > >>> > mysql>
>> > >> > >>> >
>> > >> > >>> > * PLEASE HELP ! *
>> > >> > >>> >
>> > >> > >>> > Best Regards,
>> > >> > >>> >
>> > >> > > >> > Adam Scarcella
>> > >> >
>> >
>>
>>
>>
>> --
>> Thanks for your support,
>> Motty
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Adam <ad...@gmail.com>.
I'm stuck at this point and have no idea what to try next. All of the VMs
that my team uses are offline because I cannot get a virtual router
running. At this point it looks like my only option is to roll back to 4.3.
Thoughts?
Best Regards,
Adam Scarcella
On Mon, Nov 3, 2014 at 11:08 AM, Adam <ad...@gmail.com> wrote:
> Thanks HTH,
>
> I decided to try to use your already modified qcow2 file:
>
> [root@cs-east-prod1 /]#
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> -m /export/secondary -u
> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2 -h kvm -F
> --2014-11-03 10:11:32--
> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> Resolving tmp.nux.ro... 217.19.15.108
> Connecting to tmp.nux.ro|217.19.15.108|:80... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 305498799 (291M) [application/x-bzip2]
> Saving to:
> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
>
> 100%[===================================================================================================================>]
> 305,498,799 9.02M/s in 37s
>
> 2014-11-03 10:12:10 (7.82 MB/s) -
> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
> saved [305498799/305498799]
>
> Uncompressing to
> /usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2.tmp
> (type bz2)...could take a long time
> Moving to
> /export/secondary/template/tmpl/1/274///00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2...could
> take a while
> Successfully installed system VM template to
> /export/secondary/template/tmpl/1/274/
> [root@cs-east-prod1 /]#
>
> Then I went into the Admin GUI and tried to start the VR, but it failed
> with the exact same message:
>
> [image: Inline image 1]
>
> So I destroyed the Virtual Router and stopped and started the following
> services:
>
> [root@cs-east-prod1 /]# service iptables stop
> iptables: Setting chains to policy ACCEPT: nat filter [ OK ]
> iptables: Flushing firewall rules: [ OK ]
> iptables: Unloading modules: [ OK ]
> [root@cs-east-prod1 /]# service cloudstack-management stop
> Stopping cloudstack-management: [ OK ]
> [root@cs-east-prod1 /]# service cloudstack-agent stop
> Stopping Cloud Agent:
> [root@cs-east-prod1 /]# service cloudstack-usage stop
> Stopping CloudStack Usage Monitor cloudstack-usage [ OK ]
> [root@cs-east-prod1 /]# service cloudstack-management start
> Starting cloudstack-management: [ OK ]
> [root@cs-east-prod1 /]# service cloudstack-usage start
> Starting CloudStack Usage Monitor cloudstack-usage [ OK ]
> [root@cs-east-prod1 /]# service cloudstack-agent start
> Starting Cloud Agent:
> [root@cs-east-prod1 /]#
>
> I then tried starting any VM instance to have CS automatically re-create
> the VR, but that failed as well.
>
> Motty, I also looked at trying to increase the RAM and CPU for the router
> but only saw one option
>
> [image: Inline image 2]
>
> So I created a new Service Offering with twice as much resources:
>
> [image: Inline image 3]
>
> Then upgraded the service offering for the VR
>
> [image: Inline image 4]
>
> [image: Inline image 6]
>
> But it failed still with the exact same error:
>
> [image: Inline image 7]
>
> Best Regards,
>
>
>
> Adam Scarcella
>
> On Mon, Nov 3, 2014 at 10:23 AM, motty cruz <mo...@gmail.com> wrote:
>
>> Adam,
>> what are the specifications of the VR? can you upgrade the RAM and CPU and
>> try to restart that VR? I had similar issue and giving the VR more RAM and
>> CPU will start the VR.
>>
>> Thanks,
>> Motty
>>
>> On Mon, Nov 3, 2014 at 7:04 AM, Nux! <nu...@li.nux.ro> wrote:
>>
>> > Adam,
>> >
>> > You can modify the qcow2 file that is in your secondary storage, i.e.
>> you
>> > should mount that storage somewhere and go to
>> > /mount/point/template/tmpl/274 and modify the qcow2 file.
>> >
>> > Alternatively you could use my already modified copy to overwrite your
>> > uuid.qcow2 with (at your own risk of course):
>> > http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
>> >
>> > HTH
>> >
>> > --
>> > Sent from the Delta quadrant using Borg technology!
>> >
>> > Nux!
>> > www.nux.ro
>> >
>> > ----- Original Message -----
>> > > From: "Adam" <ad...@gmail.com>
>> > > To: users@cloudstack.apache.org
>> > > Sent: Monday, 3 November, 2014 14:58:58
>> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> >
>> > > [root@cs-east-prod1 management]# netstat -a|grep 9090
>> > > unix 3 [ ] STREAM CONNECTED 29090
>> > > [root@cs-east-prod1 management]# nmap localhost
>> > >
>> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47 EST
>> > > Nmap scan report for localhost (127.0.0.1)
>> > > Host is up (0.000015s latency).
>> > > Other addresses for localhost (not scanned): 127.0.0.1
>> > > Not shown: 990 closed ports
>> > > PORT STATE SERVICE
>> > > 22/tcp open ssh
>> > > 25/tcp open smtp
>> > > 80/tcp open http
>> > > 111/tcp open rpcbind
>> > > 199/tcp open smux
>> > > 631/tcp open ipp
>> > > 2049/tcp open nfs
>> > > 3306/tcp open mysql
>> > > 8080/tcp open http-proxy
>> > > 9090/tcp open zeus-admin
>> > >
>> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
>> > > [root@cs-east-prod1 management]# telnet localhost 9090
>> > > Trying ::1...
>> > > Connected to localhost.
>> > > Escape character is '^]'.
>> > > Connection closed by foreign host.
>> > > [root@cs-east-prod1 management]# nmap 10.97.38.10
>> > >
>> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49 EST
>> > > Nmap scan report for cs-east-prod1.na.tibco.com (10.97.38.10)
>> > > Host is up (0.000015s latency).
>> > > Not shown: 993 closed ports
>> > > PORT STATE SERVICE
>> > > 22/tcp open ssh
>> > > 80/tcp open http
>> > > 111/tcp open rpcbind
>> > > 2049/tcp open nfs
>> > > 3306/tcp open mysql
>> > > 8080/tcp open http-proxy
>> > > 9090/tcp open zeus-admin
>> > >
>> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
>> > > [root@cs-east-prod1 management]# telnet 10.97.38.10 9090
>> > > Trying 10.97.38.10...
>> > > Connected to 10.97.38.10.
>> > > Escape character is '^]'.
>> > > Connection closed by foreign host.
>> > > [root@cs-east-prod1 management]#
>> > >
>> > > Also, I checked for the /opt/cloud/bin/ directory on my Management
>> Host,
>> > > but it's not there:
>> > >
>> > > [root@cs-east-prod1 management]# cd /opt/cloud/bin/
>> > > -bash: cd: /opt/cloud/bin/: No such file or directory
>> > > [root@cs-east-prod1 management]# cd /
>> > > [root@cs-east-prod1 /]# find . -name "createipAlias.sh"
>> > > [root@cs-east-prod1 /]# find . -name "createIpAlias.sh"
>> > > [root@cs-east-prod1 /]#
>> > >
>> > > Are you saying that the /opt/cloud/bin/createipAlias.sh script is
>> inside
>> > > the Virtual Router VM? If so, how could I possibly change that since
>> it
>> > > won't start?
>> > >
>> > > Best Regards,
>> > >
>> > >
>> > >
>> > > Adam Scarcella
>> > >
>> > > On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion <pd...@apache.org>
>> > wrote:
>> > >
>> > >> Could it be possible the firewall is blocking port 9090 from the
>> > >> management-server?
>> > >>
>> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null)
>> > Unable
>> > >> to ping management server at 10.97.38.10:9090 due to
>> ConnectException
>> > >> java.net.ConnectException: Connection refused
>> > >>
>> > >> On Mon, Nov 3, 2014 at 9:02 AM, Nux! <nu...@li.nux.ro> wrote:
>> > >>
>> > >> > BTW the agent logs should help clarify this.
>> > >> > The aforementioned changes must be done either in the current
>> > (broken) VR
>> > >> > or in the systemvm (after which you should redeploy the VR to pick
>> up
>> > the
>> > >> > change).
>> > >> >
>> > >> > HTH
>> > >> > Lucian
>> > >> >
>> > >> > --
>> > >> > Sent from the Delta quadrant using Borg technology!
>> > >> >
>> > >> > Nux!
>> > >> > www.nux.ro
>> > >> >
>> > >> > ----- Original Message -----
>> > >> > > From: "Nux!" <nu...@li.nux.ro>
>> > >> > > To: users@cloudstack.apache.org
>> > >> > > Sent: Monday, 3 November, 2014 13:53:10
>> > >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> > >> >
>> > >> > > Hello,
>> > >> > >
>> > >> > > I think you are hitting a problem I also had:
>> > >> > > "2014-11-03 08:16:17,024 ERROR [c.c.v.VirtualMachineManagerImpl]
>> > >> > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556
>> ctx-fcb072b2)
>> > >> > Failed to
>> > >> > > start instance VM[DomainRouter|r-4758-VM]
>> > >> > > com.cloud.utils.exception.ExecutionException: Unable to start
>> > >> > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart, not
>> > retrying"
>> > >> > >
>> > >> > > It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781
>> > >> > >
>> > >> > > Basically the systemvm template has some problems, you need to
>> > somehow
>> > >> > copy
>> > >> > > /opt/cloud/bin/createIpAlias.sh to
>> /opt/cloud/bin/createipAlias.sh
>> > to
>> > >> > get it
>> > >> > > working.
>> > >> > >
>> > >> > > --
>> > >> > > Sent from the Delta quadrant using Borg technology!
>> > >> > >
>> > >> > > Nux!
>> > >> > > www.nux.ro
>> > >> > >
>> > >> > > ----- Original Message -----
>> > >> > >> From: "Adam" <ad...@gmail.com>
>> > >> > >> To: users@cloudstack.apache.org
>> > >> > >> Sent: Monday, 3 November, 2014 13:32:12
>> > >> > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> > >> > >
>> > >> > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release
>> > >> > >> CentOS release 6.6 (Final)
>> > >> > >> [root@cs-east-prod1 ~]# uname -a
>> > >> > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15
>> > 04:27:16
>> > >> UTC
>> > >> > >> 2014 x86_64 x86_64 x86_64 GNU/Linux
>> > >> > >> [root@cs-east-prod1 ~]# virsh --version
>> > >> > >> 0.10.2
>> > >> > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
>> > >> > >> /etc/cloudstack/agent/log4j-cloud.xml
>> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management stop
>> > >> > >> Stopping cloudstack-management: [
>> OK ]
>> > >> > >> [root@cs-east-prod1 ~]# mv
>> > >> > >> /var/log/cloudstack/management/management-server.log
>> > >> > >> /var/log/cloudstack/management/management-server.bak
>> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management start
>> > >> > >> Starting cloudstack-management: [
>> OK ]
>> > >> > >> [root@cs-east-prod1 ~]#
>> > >> > >>
>> > >> > >> Please see attached ZIP of clean management-server.log in DEBUG
>> > mode.
>> > >> At
>> > >> > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the attached
>> > >> > >> management-server.log*) I tried starting the Virtual Router from
>> > the
>> > >> > >> Management UI and received this "Status" message in the UI.
>> > >> > >>
>> > >> > >> [image: Inline image 1]
>> > >> > >>
>> > >> > >> I can see in the management-server.log that while the
>> > >> > cloudstack-management
>> > >> > >> service is initializing, it throws the following ERROR at
>> exactly
>> > >> > (*2014-11-03
>> > >> > >> 08:14:36,813 on line 342 of the attached
>> management-server.log*):
>> > >> > >>
>> > >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
>> > (main:null)
>> > >> > Unable
>> > >> > >> to ping management server at 10.97.38.10:9090 due to
>> > ConnectException
>> > >> > >> java.net.ConnectException: Connection refused
>> > >> > >> at sun.nio.ch.Net.connect0(Native Method)
>> > >> > >> at sun.nio.ch.Net.connect(Net.java:465)
>> > >> > >> at sun.nio.ch.Net.connect(Net.java:457)
>> > >> > >> at
>> > sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> >
>> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
>> > >> > >> at
>> > >> > >>
>> > >>
>> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
>> > >> > >> at
>> > >> >
>> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
>> > >> > >> at
>> > >> > >>
>> > org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
>> > >> > >> at
>> > >> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
>> > >> > >> at
>> > >> >
>> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
>> > >> > >> at
>> > >> org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
>> > >> > >> at
>> > >> >
>> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
>> > >> > >> at
>> > >> > >>
>> > org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
>> > >> > >> at
>> > >> > >>
>> > >>
>> org.apache.catalina.core.StandardService.start(StandardService.java:516)
>> > >> > >> at
>> > >> > >>
>> > org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
>> > >> > >> at
>> org.apache.catalina.startup.Catalina.start(Catalina.java:593)
>> > >> > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> > >> > >> at
>> > >> > >>
>> > >> >
>> > >>
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > >> > >> at java.lang.reflect.Method.invoke(Method.java:606)
>> > >> > >> at
>> > org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
>> > >> > >> at
>> > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
>> > >> > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl]
>> > (main:null)
>> > >> > >> Detected that another management node with the same IP
>> 10.97.38.10
>> > is
>> > >> > >> considered as running in DB, however it is not pingable, we will
>> > >> > continue
>> > >> > >> cluster initialization with this management server node
>> > >> > >>
>> > >> > >> I'm not sure what that error means, but more than likely that's
>> the
>> > >> > reason
>> > >> > >> why the Virtual Router will not start.
>> > >> > >>
>> > >> > >> * PLEASE HELP ! *
>> > >> > >>
>> > >> > >> Best Regards,
>> > >> > >>
>> > >> > >>
>> > >> > >>
>> > >> > >> Adam Scarcella
>> > >> > >>
>> > >> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro> wrote:
>> > >> > >>
>> > >> > >>> Hi,
>> > >> > >>>
>> > >> > >>> More agent and management logs please, make sure they are in
>> > DEBUG[1]
>> > >> > mode.
>> > >> > >>>
>> > >> > >>> [1] -
>> > >> > >>>
>> > >>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
>> > >> > >>>
>> > >> > >>> --
>> > >> > >>> Sent from the Delta quadrant using Borg technology!
>> > >> > >>>
>> > >> > >>> Nux!
>> > >> > >>> www.nux.ro
>> > >> > >>>
>> > >> > >>> ----- Original Message -----
>> > >> > >>> > From: "Adam" <ad...@gmail.com>
>> > >> > >>> > To: users@cloudstack.apache.org
>> > >> > >>> > Sent: Sunday, 2 November, 2014 23:33:22
>> > >> > >>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> > >> > >>>
>> > >> > >>> > I've performed a complete upgrade from 4.3 to 4.4.1 and
>> followed
>> > >> the
>> > >> > >>> > instructions perfectly, but my Virtual Router will not start,
>> > nor
>> > >> am
>> > >> > I
>> > >> > >>> able
>> > >> > >>> > to "upgrade" it.
>> > >> > >>> >
>> > >> > >>> > I tried destroying the Virtual Router, starting any VM
>> Instance,
>> > >> and
>> > >> > >>> > allowing CS to recreate the VR. However that didn't help.
>> The VR
>> > >> > still
>> > >> > >>> did
>> > >> > >>> > not start and it was still saying that it required to be
>> > upgraded.
>> > >> > >>> >
>> > >> > >>> > [image: Inline image 1]
>> > >> > >>> > [image: Inline image 2]
>> > >> > >>> > [image: Inline image 3]
>> > >> > >>> >
>> > >> > >>> > I know there are manual steps to finalize the systemvm
>> upgrade,
>> > but
>> > >> > I did
>> > >> > >>> > those steps:
>> > >> > >>> >
>> > >> > >>> >
>> > >> > >>>
>> > >> >
>> > >>
>> >
>> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
>> > >> > >>> >
>> > >> > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
>> > >> > >>> > Enter password:
>> > >> > >>> > Welcome to the MySQL monitor. Commands end with ; or \g.
>> > >> > >>> > Your MySQL connection id is 174
>> > >> > >>> > Server version: 5.1.73-log Source distribution
>> > >> > >>> >
>> > >> > >>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. All
>> > rights
>> > >> > >>> reserved.
>> > >> > >>> >
>> > >> > >>> > Oracle is a registered trademark of Oracle Corporation and/or
>> > its
>> > >> > >>> > affiliates. Other names may be trademarks of their respective
>> > >> > >>> > owners.
>> > >> > >>> >
>> > >> > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the current
>> > input
>> > >> > >>> > statement.
>> > >> > >>> >
>> > >> > >>> > mysql> select id,name from cloud.vm_template where name =
>> > >> > >>> > 'systemvm-kvm-4.4';
>> > >> > >>> > +-----+------------------+
>> > >> > >>> >| id | name |
>> > >> > >>> > +-----+------------------+
>> > >> > >>> >| 274 | systemvm-kvm-4.4 |
>> > >> > >>> > +-----+------------------+
>> > >> > >>> > 1 row in set (0.00 sec)
>> > >> > >>> >
>> > >> > >>> > mysql> select type from cloud.vm_template where id='274';
>> > >> > >>> > +--------+
>> > >> > >>> >| type |
>> > >> > >>> > +--------+
>> > >> > >>> >| SYSTEM |
>> > >> > >>> > +--------+
>> > >> > >>> > 1 row in set (0.00 sec)
>> > >> > >>> >
>> > >> > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance
>> > where
>> > >> > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
>> > >> > >>> > +----------------+
>> > >> > >>> >| vm_template_id |
>> > >> > >>> > +----------------+
>> > >> > >>> >| 274 |
>> > >> > >>> > +----------------+
>> > >> > >>> > 1 row in set (0.00 sec)
>> > >> > >>> >
>> > >> > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance
>> > where
>> > >> > >>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM';
>> > >> > >>> > +----------------+
>> > >> > >>> >| vm_template_id |
>> > >> > >>> > +----------------+
>> > >> > >>> >| 274 |
>> > >> > >>> > +----------------+
>> > >> > >>> > 1 row in set (0.03 sec)
>> > >> > >>> >
>> > >> > >>> > mysql> select value from cloud.configuration where name =
>> > >> > >>> > 'router.template.kvm';
>> > >> > >>> > +------------------+
>> > >> > >>> >| value |
>> > >> > >>> > +------------------+
>> > >> > >>> >| systemvm-kvm-4.4 |
>> > >> > >>> > +------------------+
>> > >> > >>> > 1 row in set (0.00 sec)
>> > >> > >>> >
>> > >> > >>> > mysql>
>> > >> > >>> >
>> > >> > >>> > * PLEASE HELP ! *
>> > >> > >>> >
>> > >> > >>> > Best Regards,
>> > >> > >>> >
>> > >> > > >> > Adam Scarcella
>> > >> >
>> >
>>
>>
>>
>> --
>> Thanks for your support,
>> Motty
>>
>
>
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Adam <ad...@gmail.com>.
Thanks HTH,
I decided to try to use your already modified qcow2 file:
[root@cs-east-prod1 /]#
/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
-m /export/secondary -u http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
-h kvm -F
--2014-11-03 10:11:32-- http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
Resolving tmp.nux.ro... 217.19.15.108
Connecting to tmp.nux.ro|217.19.15.108|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 305498799 (291M) [application/x-bzip2]
Saving to:
“/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
100%[===================================================================================================================>]
305,498,799 9.02M/s in 37s
2014-11-03 10:12:10 (7.82 MB/s) -
“/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2”
saved [305498799/305498799]
Uncompressing to
/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2.tmp
(type bz2)...could take a long time
Moving to
/export/secondary/template/tmpl/1/274///00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2...could
take a while
Successfully installed system VM template to
/export/secondary/template/tmpl/1/274/
[root@cs-east-prod1 /]#
Then I went into the Admin GUI and tried to start the VR, but it failed
with the exact same message:
[image: Inline image 1]
So I destroyed the Virtual Router and stopped and started the following
services:
[root@cs-east-prod1 /]# service iptables stop
iptables: Setting chains to policy ACCEPT: nat filter [ OK ]
iptables: Flushing firewall rules: [ OK ]
iptables: Unloading modules: [ OK ]
[root@cs-east-prod1 /]# service cloudstack-management stop
Stopping cloudstack-management: [ OK ]
[root@cs-east-prod1 /]# service cloudstack-agent stop
Stopping Cloud Agent:
[root@cs-east-prod1 /]# service cloudstack-usage stop
Stopping CloudStack Usage Monitor cloudstack-usage [ OK ]
[root@cs-east-prod1 /]# service cloudstack-management start
Starting cloudstack-management: [ OK ]
[root@cs-east-prod1 /]# service cloudstack-usage start
Starting CloudStack Usage Monitor cloudstack-usage [ OK ]
[root@cs-east-prod1 /]# service cloudstack-agent start
Starting Cloud Agent:
[root@cs-east-prod1 /]#
I then tried starting any VM instance to have CS automatically re-create
the VR, but that failed as well.
Motty, I also looked at trying to increase the RAM and CPU for the router
but only saw one option
[image: Inline image 2]
So I created a new Service Offering with twice as much resources:
[image: Inline image 3]
Then upgraded the service offering for the VR
[image: Inline image 4]
[image: Inline image 6]
But it failed still with the exact same error:
[image: Inline image 7]
Best Regards,
Adam Scarcella
On Mon, Nov 3, 2014 at 10:23 AM, motty cruz <mo...@gmail.com> wrote:
> Adam,
> what are the specifications of the VR? can you upgrade the RAM and CPU and
> try to restart that VR? I had similar issue and giving the VR more RAM and
> CPU will start the VR.
>
> Thanks,
> Motty
>
> On Mon, Nov 3, 2014 at 7:04 AM, Nux! <nu...@li.nux.ro> wrote:
>
> > Adam,
> >
> > You can modify the qcow2 file that is in your secondary storage, i.e. you
> > should mount that storage somewhere and go to
> > /mount/point/template/tmpl/274 and modify the qcow2 file.
> >
> > Alternatively you could use my already modified copy to overwrite your
> > uuid.qcow2 with (at your own risk of course):
> > http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
> >
> > HTH
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> > ----- Original Message -----
> > > From: "Adam" <ad...@gmail.com>
> > > To: users@cloudstack.apache.org
> > > Sent: Monday, 3 November, 2014 14:58:58
> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >
> > > [root@cs-east-prod1 management]# netstat -a|grep 9090
> > > unix 3 [ ] STREAM CONNECTED 29090
> > > [root@cs-east-prod1 management]# nmap localhost
> > >
> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47 EST
> > > Nmap scan report for localhost (127.0.0.1)
> > > Host is up (0.000015s latency).
> > > Other addresses for localhost (not scanned): 127.0.0.1
> > > Not shown: 990 closed ports
> > > PORT STATE SERVICE
> > > 22/tcp open ssh
> > > 25/tcp open smtp
> > > 80/tcp open http
> > > 111/tcp open rpcbind
> > > 199/tcp open smux
> > > 631/tcp open ipp
> > > 2049/tcp open nfs
> > > 3306/tcp open mysql
> > > 8080/tcp open http-proxy
> > > 9090/tcp open zeus-admin
> > >
> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
> > > [root@cs-east-prod1 management]# telnet localhost 9090
> > > Trying ::1...
> > > Connected to localhost.
> > > Escape character is '^]'.
> > > Connection closed by foreign host.
> > > [root@cs-east-prod1 management]# nmap 10.97.38.10
> > >
> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49 EST
> > > Nmap scan report for cs-east-prod1.na.tibco.com (10.97.38.10)
> > > Host is up (0.000015s latency).
> > > Not shown: 993 closed ports
> > > PORT STATE SERVICE
> > > 22/tcp open ssh
> > > 80/tcp open http
> > > 111/tcp open rpcbind
> > > 2049/tcp open nfs
> > > 3306/tcp open mysql
> > > 8080/tcp open http-proxy
> > > 9090/tcp open zeus-admin
> > >
> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
> > > [root@cs-east-prod1 management]# telnet 10.97.38.10 9090
> > > Trying 10.97.38.10...
> > > Connected to 10.97.38.10.
> > > Escape character is '^]'.
> > > Connection closed by foreign host.
> > > [root@cs-east-prod1 management]#
> > >
> > > Also, I checked for the /opt/cloud/bin/ directory on my Management
> Host,
> > > but it's not there:
> > >
> > > [root@cs-east-prod1 management]# cd /opt/cloud/bin/
> > > -bash: cd: /opt/cloud/bin/: No such file or directory
> > > [root@cs-east-prod1 management]# cd /
> > > [root@cs-east-prod1 /]# find . -name "createipAlias.sh"
> > > [root@cs-east-prod1 /]# find . -name "createIpAlias.sh"
> > > [root@cs-east-prod1 /]#
> > >
> > > Are you saying that the /opt/cloud/bin/createipAlias.sh script is
> inside
> > > the Virtual Router VM? If so, how could I possibly change that since it
> > > won't start?
> > >
> > > Best Regards,
> > >
> > >
> > >
> > > Adam Scarcella
> > >
> > > On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion <pd...@apache.org>
> > wrote:
> > >
> > >> Could it be possible the firewall is blocking port 9090 from the
> > >> management-server?
> > >>
> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null)
> > Unable
> > >> to ping management server at 10.97.38.10:9090 due to ConnectException
> > >> java.net.ConnectException: Connection refused
> > >>
> > >> On Mon, Nov 3, 2014 at 9:02 AM, Nux! <nu...@li.nux.ro> wrote:
> > >>
> > >> > BTW the agent logs should help clarify this.
> > >> > The aforementioned changes must be done either in the current
> > (broken) VR
> > >> > or in the systemvm (after which you should redeploy the VR to pick
> up
> > the
> > >> > change).
> > >> >
> > >> > HTH
> > >> > Lucian
> > >> >
> > >> > --
> > >> > Sent from the Delta quadrant using Borg technology!
> > >> >
> > >> > Nux!
> > >> > www.nux.ro
> > >> >
> > >> > ----- Original Message -----
> > >> > > From: "Nux!" <nu...@li.nux.ro>
> > >> > > To: users@cloudstack.apache.org
> > >> > > Sent: Monday, 3 November, 2014 13:53:10
> > >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> > >> >
> > >> > > Hello,
> > >> > >
> > >> > > I think you are hitting a problem I also had:
> > >> > > "2014-11-03 08:16:17,024 ERROR [c.c.v.VirtualMachineManagerImpl]
> > >> > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556
> ctx-fcb072b2)
> > >> > Failed to
> > >> > > start instance VM[DomainRouter|r-4758-VM]
> > >> > > com.cloud.utils.exception.ExecutionException: Unable to start
> > >> > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart, not
> > retrying"
> > >> > >
> > >> > > It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781
> > >> > >
> > >> > > Basically the systemvm template has some problems, you need to
> > somehow
> > >> > copy
> > >> > > /opt/cloud/bin/createIpAlias.sh to /opt/cloud/bin/createipAlias.sh
> > to
> > >> > get it
> > >> > > working.
> > >> > >
> > >> > > --
> > >> > > Sent from the Delta quadrant using Borg technology!
> > >> > >
> > >> > > Nux!
> > >> > > www.nux.ro
> > >> > >
> > >> > > ----- Original Message -----
> > >> > >> From: "Adam" <ad...@gmail.com>
> > >> > >> To: users@cloudstack.apache.org
> > >> > >> Sent: Monday, 3 November, 2014 13:32:12
> > >> > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> > >> > >
> > >> > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release
> > >> > >> CentOS release 6.6 (Final)
> > >> > >> [root@cs-east-prod1 ~]# uname -a
> > >> > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15
> > 04:27:16
> > >> UTC
> > >> > >> 2014 x86_64 x86_64 x86_64 GNU/Linux
> > >> > >> [root@cs-east-prod1 ~]# virsh --version
> > >> > >> 0.10.2
> > >> > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
> > >> > >> /etc/cloudstack/agent/log4j-cloud.xml
> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management stop
> > >> > >> Stopping cloudstack-management: [
> OK ]
> > >> > >> [root@cs-east-prod1 ~]# mv
> > >> > >> /var/log/cloudstack/management/management-server.log
> > >> > >> /var/log/cloudstack/management/management-server.bak
> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management start
> > >> > >> Starting cloudstack-management: [
> OK ]
> > >> > >> [root@cs-east-prod1 ~]#
> > >> > >>
> > >> > >> Please see attached ZIP of clean management-server.log in DEBUG
> > mode.
> > >> At
> > >> > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the attached
> > >> > >> management-server.log*) I tried starting the Virtual Router from
> > the
> > >> > >> Management UI and received this "Status" message in the UI.
> > >> > >>
> > >> > >> [image: Inline image 1]
> > >> > >>
> > >> > >> I can see in the management-server.log that while the
> > >> > cloudstack-management
> > >> > >> service is initializing, it throws the following ERROR at exactly
> > >> > (*2014-11-03
> > >> > >> 08:14:36,813 on line 342 of the attached management-server.log*):
> > >> > >>
> > >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
> > (main:null)
> > >> > Unable
> > >> > >> to ping management server at 10.97.38.10:9090 due to
> > ConnectException
> > >> > >> java.net.ConnectException: Connection refused
> > >> > >> at sun.nio.ch.Net.connect0(Native Method)
> > >> > >> at sun.nio.ch.Net.connect(Net.java:465)
> > >> > >> at sun.nio.ch.Net.connect(Net.java:457)
> > >> > >> at
> > sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
> > >> > >> at
> > >> > >>
> > >> >
> > org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
> > >> > >> at
> > >> > >>
> > >>
> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
> > >> > >> at
> > >> >
> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
> > >> > >> at
> > >> > >>
> > org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
> > >> > >> at
> > >> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> > >> > >> at
> > >> >
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
> > >> > >> at
> > >> org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
> > >> > >> at
> > >> >
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
> > >> > >> at
> > >> > >>
> > org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
> > >> > >> at
> > >> > >>
> > >>
> org.apache.catalina.core.StandardService.start(StandardService.java:516)
> > >> > >> at
> > >> > >>
> > org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
> > >> > >> at
> org.apache.catalina.startup.Catalina.start(Catalina.java:593)
> > >> > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > >> > >> at
> > >> > >>
> > >> >
> > >>
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > >> > >> at java.lang.reflect.Method.invoke(Method.java:606)
> > >> > >> at
> > org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> > >> > >> at
> > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> > >> > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl]
> > (main:null)
> > >> > >> Detected that another management node with the same IP
> 10.97.38.10
> > is
> > >> > >> considered as running in DB, however it is not pingable, we will
> > >> > continue
> > >> > >> cluster initialization with this management server node
> > >> > >>
> > >> > >> I'm not sure what that error means, but more than likely that's
> the
> > >> > reason
> > >> > >> why the Virtual Router will not start.
> > >> > >>
> > >> > >> * PLEASE HELP ! *
> > >> > >>
> > >> > >> Best Regards,
> > >> > >>
> > >> > >>
> > >> > >>
> > >> > >> Adam Scarcella
> > >> > >>
> > >> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro> wrote:
> > >> > >>
> > >> > >>> Hi,
> > >> > >>>
> > >> > >>> More agent and management logs please, make sure they are in
> > DEBUG[1]
> > >> > mode.
> > >> > >>>
> > >> > >>> [1] -
> > >> > >>>
> > >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
> > >> > >>>
> > >> > >>> --
> > >> > >>> Sent from the Delta quadrant using Borg technology!
> > >> > >>>
> > >> > >>> Nux!
> > >> > >>> www.nux.ro
> > >> > >>>
> > >> > >>> ----- Original Message -----
> > >> > >>> > From: "Adam" <ad...@gmail.com>
> > >> > >>> > To: users@cloudstack.apache.org
> > >> > >>> > Sent: Sunday, 2 November, 2014 23:33:22
> > >> > >>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1
> > >> > >>>
> > >> > >>> > I've performed a complete upgrade from 4.3 to 4.4.1 and
> followed
> > >> the
> > >> > >>> > instructions perfectly, but my Virtual Router will not start,
> > nor
> > >> am
> > >> > I
> > >> > >>> able
> > >> > >>> > to "upgrade" it.
> > >> > >>> >
> > >> > >>> > I tried destroying the Virtual Router, starting any VM
> Instance,
> > >> and
> > >> > >>> > allowing CS to recreate the VR. However that didn't help. The
> VR
> > >> > still
> > >> > >>> did
> > >> > >>> > not start and it was still saying that it required to be
> > upgraded.
> > >> > >>> >
> > >> > >>> > [image: Inline image 1]
> > >> > >>> > [image: Inline image 2]
> > >> > >>> > [image: Inline image 3]
> > >> > >>> >
> > >> > >>> > I know there are manual steps to finalize the systemvm
> upgrade,
> > but
> > >> > I did
> > >> > >>> > those steps:
> > >> > >>> >
> > >> > >>> >
> > >> > >>>
> > >> >
> > >>
> >
> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
> > >> > >>> >
> > >> > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
> > >> > >>> > Enter password:
> > >> > >>> > Welcome to the MySQL monitor. Commands end with ; or \g.
> > >> > >>> > Your MySQL connection id is 174
> > >> > >>> > Server version: 5.1.73-log Source distribution
> > >> > >>> >
> > >> > >>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. All
> > rights
> > >> > >>> reserved.
> > >> > >>> >
> > >> > >>> > Oracle is a registered trademark of Oracle Corporation and/or
> > its
> > >> > >>> > affiliates. Other names may be trademarks of their respective
> > >> > >>> > owners.
> > >> > >>> >
> > >> > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the current
> > input
> > >> > >>> > statement.
> > >> > >>> >
> > >> > >>> > mysql> select id,name from cloud.vm_template where name =
> > >> > >>> > 'systemvm-kvm-4.4';
> > >> > >>> > +-----+------------------+
> > >> > >>> >| id | name |
> > >> > >>> > +-----+------------------+
> > >> > >>> >| 274 | systemvm-kvm-4.4 |
> > >> > >>> > +-----+------------------+
> > >> > >>> > 1 row in set (0.00 sec)
> > >> > >>> >
> > >> > >>> > mysql> select type from cloud.vm_template where id='274';
> > >> > >>> > +--------+
> > >> > >>> >| type |
> > >> > >>> > +--------+
> > >> > >>> >| SYSTEM |
> > >> > >>> > +--------+
> > >> > >>> > 1 row in set (0.00 sec)
> > >> > >>> >
> > >> > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance
> > where
> > >> > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
> > >> > >>> > +----------------+
> > >> > >>> >| vm_template_id |
> > >> > >>> > +----------------+
> > >> > >>> >| 274 |
> > >> > >>> > +----------------+
> > >> > >>> > 1 row in set (0.00 sec)
> > >> > >>> >
> > >> > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance
> > where
> > >> > >>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM';
> > >> > >>> > +----------------+
> > >> > >>> >| vm_template_id |
> > >> > >>> > +----------------+
> > >> > >>> >| 274 |
> > >> > >>> > +----------------+
> > >> > >>> > 1 row in set (0.03 sec)
> > >> > >>> >
> > >> > >>> > mysql> select value from cloud.configuration where name =
> > >> > >>> > 'router.template.kvm';
> > >> > >>> > +------------------+
> > >> > >>> >| value |
> > >> > >>> > +------------------+
> > >> > >>> >| systemvm-kvm-4.4 |
> > >> > >>> > +------------------+
> > >> > >>> > 1 row in set (0.00 sec)
> > >> > >>> >
> > >> > >>> > mysql>
> > >> > >>> >
> > >> > >>> > * PLEASE HELP ! *
> > >> > >>> >
> > >> > >>> > Best Regards,
> > >> > >>> >
> > >> > > >> > Adam Scarcella
> > >> >
> >
>
>
>
> --
> Thanks for your support,
> Motty
>
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by motty cruz <mo...@gmail.com>.
Adam,
what are the specifications of the VR? can you upgrade the RAM and CPU and
try to restart that VR? I had similar issue and giving the VR more RAM and
CPU will start the VR.
Thanks,
Motty
On Mon, Nov 3, 2014 at 7:04 AM, Nux! <nu...@li.nux.ro> wrote:
> Adam,
>
> You can modify the qcow2 file that is in your secondary storage, i.e. you
> should mount that storage somewhere and go to
> /mount/point/template/tmpl/274 and modify the qcow2 file.
>
> Alternatively you could use my already modified copy to overwrite your
> uuid.qcow2 with (at your own risk of course):
> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
>
> HTH
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
> > From: "Adam" <ad...@gmail.com>
> > To: users@cloudstack.apache.org
> > Sent: Monday, 3 November, 2014 14:58:58
> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>
> > [root@cs-east-prod1 management]# netstat -a|grep 9090
> > unix 3 [ ] STREAM CONNECTED 29090
> > [root@cs-east-prod1 management]# nmap localhost
> >
> > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47 EST
> > Nmap scan report for localhost (127.0.0.1)
> > Host is up (0.000015s latency).
> > Other addresses for localhost (not scanned): 127.0.0.1
> > Not shown: 990 closed ports
> > PORT STATE SERVICE
> > 22/tcp open ssh
> > 25/tcp open smtp
> > 80/tcp open http
> > 111/tcp open rpcbind
> > 199/tcp open smux
> > 631/tcp open ipp
> > 2049/tcp open nfs
> > 3306/tcp open mysql
> > 8080/tcp open http-proxy
> > 9090/tcp open zeus-admin
> >
> > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
> > [root@cs-east-prod1 management]# telnet localhost 9090
> > Trying ::1...
> > Connected to localhost.
> > Escape character is '^]'.
> > Connection closed by foreign host.
> > [root@cs-east-prod1 management]# nmap 10.97.38.10
> >
> > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49 EST
> > Nmap scan report for cs-east-prod1.na.tibco.com (10.97.38.10)
> > Host is up (0.000015s latency).
> > Not shown: 993 closed ports
> > PORT STATE SERVICE
> > 22/tcp open ssh
> > 80/tcp open http
> > 111/tcp open rpcbind
> > 2049/tcp open nfs
> > 3306/tcp open mysql
> > 8080/tcp open http-proxy
> > 9090/tcp open zeus-admin
> >
> > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
> > [root@cs-east-prod1 management]# telnet 10.97.38.10 9090
> > Trying 10.97.38.10...
> > Connected to 10.97.38.10.
> > Escape character is '^]'.
> > Connection closed by foreign host.
> > [root@cs-east-prod1 management]#
> >
> > Also, I checked for the /opt/cloud/bin/ directory on my Management Host,
> > but it's not there:
> >
> > [root@cs-east-prod1 management]# cd /opt/cloud/bin/
> > -bash: cd: /opt/cloud/bin/: No such file or directory
> > [root@cs-east-prod1 management]# cd /
> > [root@cs-east-prod1 /]# find . -name "createipAlias.sh"
> > [root@cs-east-prod1 /]# find . -name "createIpAlias.sh"
> > [root@cs-east-prod1 /]#
> >
> > Are you saying that the /opt/cloud/bin/createipAlias.sh script is inside
> > the Virtual Router VM? If so, how could I possibly change that since it
> > won't start?
> >
> > Best Regards,
> >
> >
> >
> > Adam Scarcella
> >
> > On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion <pd...@apache.org>
> wrote:
> >
> >> Could it be possible the firewall is blocking port 9090 from the
> >> management-server?
> >>
> >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null)
> Unable
> >> to ping management server at 10.97.38.10:9090 due to ConnectException
> >> java.net.ConnectException: Connection refused
> >>
> >> On Mon, Nov 3, 2014 at 9:02 AM, Nux! <nu...@li.nux.ro> wrote:
> >>
> >> > BTW the agent logs should help clarify this.
> >> > The aforementioned changes must be done either in the current
> (broken) VR
> >> > or in the systemvm (after which you should redeploy the VR to pick up
> the
> >> > change).
> >> >
> >> > HTH
> >> > Lucian
> >> >
> >> > --
> >> > Sent from the Delta quadrant using Borg technology!
> >> >
> >> > Nux!
> >> > www.nux.ro
> >> >
> >> > ----- Original Message -----
> >> > > From: "Nux!" <nu...@li.nux.ro>
> >> > > To: users@cloudstack.apache.org
> >> > > Sent: Monday, 3 November, 2014 13:53:10
> >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >> >
> >> > > Hello,
> >> > >
> >> > > I think you are hitting a problem I also had:
> >> > > "2014-11-03 08:16:17,024 ERROR [c.c.v.VirtualMachineManagerImpl]
> >> > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556 ctx-fcb072b2)
> >> > Failed to
> >> > > start instance VM[DomainRouter|r-4758-VM]
> >> > > com.cloud.utils.exception.ExecutionException: Unable to start
> >> > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart, not
> retrying"
> >> > >
> >> > > It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781
> >> > >
> >> > > Basically the systemvm template has some problems, you need to
> somehow
> >> > copy
> >> > > /opt/cloud/bin/createIpAlias.sh to /opt/cloud/bin/createipAlias.sh
> to
> >> > get it
> >> > > working.
> >> > >
> >> > > --
> >> > > Sent from the Delta quadrant using Borg technology!
> >> > >
> >> > > Nux!
> >> > > www.nux.ro
> >> > >
> >> > > ----- Original Message -----
> >> > >> From: "Adam" <ad...@gmail.com>
> >> > >> To: users@cloudstack.apache.org
> >> > >> Sent: Monday, 3 November, 2014 13:32:12
> >> > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >> > >
> >> > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release
> >> > >> CentOS release 6.6 (Final)
> >> > >> [root@cs-east-prod1 ~]# uname -a
> >> > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15
> 04:27:16
> >> UTC
> >> > >> 2014 x86_64 x86_64 x86_64 GNU/Linux
> >> > >> [root@cs-east-prod1 ~]# virsh --version
> >> > >> 0.10.2
> >> > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
> >> > >> /etc/cloudstack/agent/log4j-cloud.xml
> >> > >> [root@cs-east-prod1 ~]# service cloudstack-management stop
> >> > >> Stopping cloudstack-management: [ OK ]
> >> > >> [root@cs-east-prod1 ~]# mv
> >> > >> /var/log/cloudstack/management/management-server.log
> >> > >> /var/log/cloudstack/management/management-server.bak
> >> > >> [root@cs-east-prod1 ~]# service cloudstack-management start
> >> > >> Starting cloudstack-management: [ OK ]
> >> > >> [root@cs-east-prod1 ~]#
> >> > >>
> >> > >> Please see attached ZIP of clean management-server.log in DEBUG
> mode.
> >> At
> >> > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the attached
> >> > >> management-server.log*) I tried starting the Virtual Router from
> the
> >> > >> Management UI and received this "Status" message in the UI.
> >> > >>
> >> > >> [image: Inline image 1]
> >> > >>
> >> > >> I can see in the management-server.log that while the
> >> > cloudstack-management
> >> > >> service is initializing, it throws the following ERROR at exactly
> >> > (*2014-11-03
> >> > >> 08:14:36,813 on line 342 of the attached management-server.log*):
> >> > >>
> >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl]
> (main:null)
> >> > Unable
> >> > >> to ping management server at 10.97.38.10:9090 due to
> ConnectException
> >> > >> java.net.ConnectException: Connection refused
> >> > >> at sun.nio.ch.Net.connect0(Native Method)
> >> > >> at sun.nio.ch.Net.connect(Net.java:465)
> >> > >> at sun.nio.ch.Net.connect(Net.java:457)
> >> > >> at
> sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
> >> > >> at
> >> > >>
> >> >
> >>
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
> >> > >> at
> >> > >>
> >> >
> >>
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
> >> > >> at
> >> > >>
> >> >
> >>
> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
> >> > >> at
> >> > >>
> >> >
> >>
> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
> >> > >> at
> >> > >>
> >> >
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
> >> > >> at
> >> > >>
> >> >
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
> >> > >> at
> >> > >>
> >> >
> >>
> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
> >> > >> at
> >> > >>
> >> >
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
> >> > >> at
> >> > >>
> >> >
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
> >> > >> at
> >> > >>
> >> >
> >>
> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
> >> > >> at
> >> > >>
> >> >
> >>
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
> >> > >> at
> >> > >>
> >> >
> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
> >> > >> at
> >> > >>
> >> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
> >> > >> at
> >> > org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
> >> > >> at
> >> > >>
> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
> >> > >> at
> >> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
> >> > >> at
> >> > >>
> >> >
> >>
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> >> > >> at
> >> > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
> >> > >> at
> >> org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
> >> > >> at
> >> > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
> >> > >> at
> >> > >>
> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
> >> > >> at
> >> > >>
> >> org.apache.catalina.core.StandardService.start(StandardService.java:516)
> >> > >> at
> >> > >>
> org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
> >> > >> at org.apache.catalina.startup.Catalina.start(Catalina.java:593)
> >> > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> > >> at
> >> > >>
> >> >
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> > >> at
> >> > >>
> >> >
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> > >> at java.lang.reflect.Method.invoke(Method.java:606)
> >> > >> at
> org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> >> > >> at
> org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> >> > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl]
> (main:null)
> >> > >> Detected that another management node with the same IP 10.97.38.10
> is
> >> > >> considered as running in DB, however it is not pingable, we will
> >> > continue
> >> > >> cluster initialization with this management server node
> >> > >>
> >> > >> I'm not sure what that error means, but more than likely that's the
> >> > reason
> >> > >> why the Virtual Router will not start.
> >> > >>
> >> > >> * PLEASE HELP ! *
> >> > >>
> >> > >> Best Regards,
> >> > >>
> >> > >>
> >> > >>
> >> > >> Adam Scarcella
> >> > >>
> >> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro> wrote:
> >> > >>
> >> > >>> Hi,
> >> > >>>
> >> > >>> More agent and management logs please, make sure they are in
> DEBUG[1]
> >> > mode.
> >> > >>>
> >> > >>> [1] -
> >> > >>>
> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
> >> > >>>
> >> > >>> --
> >> > >>> Sent from the Delta quadrant using Borg technology!
> >> > >>>
> >> > >>> Nux!
> >> > >>> www.nux.ro
> >> > >>>
> >> > >>> ----- Original Message -----
> >> > >>> > From: "Adam" <ad...@gmail.com>
> >> > >>> > To: users@cloudstack.apache.org
> >> > >>> > Sent: Sunday, 2 November, 2014 23:33:22
> >> > >>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >> > >>>
> >> > >>> > I've performed a complete upgrade from 4.3 to 4.4.1 and followed
> >> the
> >> > >>> > instructions perfectly, but my Virtual Router will not start,
> nor
> >> am
> >> > I
> >> > >>> able
> >> > >>> > to "upgrade" it.
> >> > >>> >
> >> > >>> > I tried destroying the Virtual Router, starting any VM Instance,
> >> and
> >> > >>> > allowing CS to recreate the VR. However that didn't help. The VR
> >> > still
> >> > >>> did
> >> > >>> > not start and it was still saying that it required to be
> upgraded.
> >> > >>> >
> >> > >>> > [image: Inline image 1]
> >> > >>> > [image: Inline image 2]
> >> > >>> > [image: Inline image 3]
> >> > >>> >
> >> > >>> > I know there are manual steps to finalize the systemvm upgrade,
> but
> >> > I did
> >> > >>> > those steps:
> >> > >>> >
> >> > >>> >
> >> > >>>
> >> >
> >>
> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
> >> > >>> >
> >> > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
> >> > >>> > Enter password:
> >> > >>> > Welcome to the MySQL monitor. Commands end with ; or \g.
> >> > >>> > Your MySQL connection id is 174
> >> > >>> > Server version: 5.1.73-log Source distribution
> >> > >>> >
> >> > >>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. All
> rights
> >> > >>> reserved.
> >> > >>> >
> >> > >>> > Oracle is a registered trademark of Oracle Corporation and/or
> its
> >> > >>> > affiliates. Other names may be trademarks of their respective
> >> > >>> > owners.
> >> > >>> >
> >> > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the current
> input
> >> > >>> > statement.
> >> > >>> >
> >> > >>> > mysql> select id,name from cloud.vm_template where name =
> >> > >>> > 'systemvm-kvm-4.4';
> >> > >>> > +-----+------------------+
> >> > >>> >| id | name |
> >> > >>> > +-----+------------------+
> >> > >>> >| 274 | systemvm-kvm-4.4 |
> >> > >>> > +-----+------------------+
> >> > >>> > 1 row in set (0.00 sec)
> >> > >>> >
> >> > >>> > mysql> select type from cloud.vm_template where id='274';
> >> > >>> > +--------+
> >> > >>> >| type |
> >> > >>> > +--------+
> >> > >>> >| SYSTEM |
> >> > >>> > +--------+
> >> > >>> > 1 row in set (0.00 sec)
> >> > >>> >
> >> > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance
> where
> >> > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
> >> > >>> > +----------------+
> >> > >>> >| vm_template_id |
> >> > >>> > +----------------+
> >> > >>> >| 274 |
> >> > >>> > +----------------+
> >> > >>> > 1 row in set (0.00 sec)
> >> > >>> >
> >> > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance
> where
> >> > >>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM';
> >> > >>> > +----------------+
> >> > >>> >| vm_template_id |
> >> > >>> > +----------------+
> >> > >>> >| 274 |
> >> > >>> > +----------------+
> >> > >>> > 1 row in set (0.03 sec)
> >> > >>> >
> >> > >>> > mysql> select value from cloud.configuration where name =
> >> > >>> > 'router.template.kvm';
> >> > >>> > +------------------+
> >> > >>> >| value |
> >> > >>> > +------------------+
> >> > >>> >| systemvm-kvm-4.4 |
> >> > >>> > +------------------+
> >> > >>> > 1 row in set (0.00 sec)
> >> > >>> >
> >> > >>> > mysql>
> >> > >>> >
> >> > >>> > * PLEASE HELP ! *
> >> > >>> >
> >> > >>> > Best Regards,
> >> > >>> >
> >> > > >> > Adam Scarcella
> >> >
>
--
Thanks for your support,
Motty
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Nux! <nu...@li.nux.ro>.
Adam,
You can modify the qcow2 file that is in your secondary storage, i.e. you should mount that storage somewhere and go to
/mount/point/template/tmpl/274 and modify the qcow2 file.
Alternatively you could use my already modified copy to overwrite your uuid.qcow2 with (at your own risk of course): http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2
HTH
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Adam" <ad...@gmail.com>
> To: users@cloudstack.apache.org
> Sent: Monday, 3 November, 2014 14:58:58
> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> [root@cs-east-prod1 management]# netstat -a|grep 9090
> unix 3 [ ] STREAM CONNECTED 29090
> [root@cs-east-prod1 management]# nmap localhost
>
> Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47 EST
> Nmap scan report for localhost (127.0.0.1)
> Host is up (0.000015s latency).
> Other addresses for localhost (not scanned): 127.0.0.1
> Not shown: 990 closed ports
> PORT STATE SERVICE
> 22/tcp open ssh
> 25/tcp open smtp
> 80/tcp open http
> 111/tcp open rpcbind
> 199/tcp open smux
> 631/tcp open ipp
> 2049/tcp open nfs
> 3306/tcp open mysql
> 8080/tcp open http-proxy
> 9090/tcp open zeus-admin
>
> Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
> [root@cs-east-prod1 management]# telnet localhost 9090
> Trying ::1...
> Connected to localhost.
> Escape character is '^]'.
> Connection closed by foreign host.
> [root@cs-east-prod1 management]# nmap 10.97.38.10
>
> Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49 EST
> Nmap scan report for cs-east-prod1.na.tibco.com (10.97.38.10)
> Host is up (0.000015s latency).
> Not shown: 993 closed ports
> PORT STATE SERVICE
> 22/tcp open ssh
> 80/tcp open http
> 111/tcp open rpcbind
> 2049/tcp open nfs
> 3306/tcp open mysql
> 8080/tcp open http-proxy
> 9090/tcp open zeus-admin
>
> Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
> [root@cs-east-prod1 management]# telnet 10.97.38.10 9090
> Trying 10.97.38.10...
> Connected to 10.97.38.10.
> Escape character is '^]'.
> Connection closed by foreign host.
> [root@cs-east-prod1 management]#
>
> Also, I checked for the /opt/cloud/bin/ directory on my Management Host,
> but it's not there:
>
> [root@cs-east-prod1 management]# cd /opt/cloud/bin/
> -bash: cd: /opt/cloud/bin/: No such file or directory
> [root@cs-east-prod1 management]# cd /
> [root@cs-east-prod1 /]# find . -name "createipAlias.sh"
> [root@cs-east-prod1 /]# find . -name "createIpAlias.sh"
> [root@cs-east-prod1 /]#
>
> Are you saying that the /opt/cloud/bin/createipAlias.sh script is inside
> the Virtual Router VM? If so, how could I possibly change that since it
> won't start?
>
> Best Regards,
>
>
>
> Adam Scarcella
>
> On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion <pd...@apache.org> wrote:
>
>> Could it be possible the firewall is blocking port 9090 from the
>> management-server?
>>
>> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null) Unable
>> to ping management server at 10.97.38.10:9090 due to ConnectException
>> java.net.ConnectException: Connection refused
>>
>> On Mon, Nov 3, 2014 at 9:02 AM, Nux! <nu...@li.nux.ro> wrote:
>>
>> > BTW the agent logs should help clarify this.
>> > The aforementioned changes must be done either in the current (broken) VR
>> > or in the systemvm (after which you should redeploy the VR to pick up the
>> > change).
>> >
>> > HTH
>> > Lucian
>> >
>> > --
>> > Sent from the Delta quadrant using Borg technology!
>> >
>> > Nux!
>> > www.nux.ro
>> >
>> > ----- Original Message -----
>> > > From: "Nux!" <nu...@li.nux.ro>
>> > > To: users@cloudstack.apache.org
>> > > Sent: Monday, 3 November, 2014 13:53:10
>> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> >
>> > > Hello,
>> > >
>> > > I think you are hitting a problem I also had:
>> > > "2014-11-03 08:16:17,024 ERROR [c.c.v.VirtualMachineManagerImpl]
>> > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556 ctx-fcb072b2)
>> > Failed to
>> > > start instance VM[DomainRouter|r-4758-VM]
>> > > com.cloud.utils.exception.ExecutionException: Unable to start
>> > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart, not retrying"
>> > >
>> > > It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781
>> > >
>> > > Basically the systemvm template has some problems, you need to somehow
>> > copy
>> > > /opt/cloud/bin/createIpAlias.sh to /opt/cloud/bin/createipAlias.sh to
>> > get it
>> > > working.
>> > >
>> > > --
>> > > Sent from the Delta quadrant using Borg technology!
>> > >
>> > > Nux!
>> > > www.nux.ro
>> > >
>> > > ----- Original Message -----
>> > >> From: "Adam" <ad...@gmail.com>
>> > >> To: users@cloudstack.apache.org
>> > >> Sent: Monday, 3 November, 2014 13:32:12
>> > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> > >
>> > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release
>> > >> CentOS release 6.6 (Final)
>> > >> [root@cs-east-prod1 ~]# uname -a
>> > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16
>> UTC
>> > >> 2014 x86_64 x86_64 x86_64 GNU/Linux
>> > >> [root@cs-east-prod1 ~]# virsh --version
>> > >> 0.10.2
>> > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
>> > >> /etc/cloudstack/agent/log4j-cloud.xml
>> > >> [root@cs-east-prod1 ~]# service cloudstack-management stop
>> > >> Stopping cloudstack-management: [ OK ]
>> > >> [root@cs-east-prod1 ~]# mv
>> > >> /var/log/cloudstack/management/management-server.log
>> > >> /var/log/cloudstack/management/management-server.bak
>> > >> [root@cs-east-prod1 ~]# service cloudstack-management start
>> > >> Starting cloudstack-management: [ OK ]
>> > >> [root@cs-east-prod1 ~]#
>> > >>
>> > >> Please see attached ZIP of clean management-server.log in DEBUG mode.
>> At
>> > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the attached
>> > >> management-server.log*) I tried starting the Virtual Router from the
>> > >> Management UI and received this "Status" message in the UI.
>> > >>
>> > >> [image: Inline image 1]
>> > >>
>> > >> I can see in the management-server.log that while the
>> > cloudstack-management
>> > >> service is initializing, it throws the following ERROR at exactly
>> > (*2014-11-03
>> > >> 08:14:36,813 on line 342 of the attached management-server.log*):
>> > >>
>> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null)
>> > Unable
>> > >> to ping management server at 10.97.38.10:9090 due to ConnectException
>> > >> java.net.ConnectException: Connection refused
>> > >> at sun.nio.ch.Net.connect0(Native Method)
>> > >> at sun.nio.ch.Net.connect(Net.java:465)
>> > >> at sun.nio.ch.Net.connect(Net.java:457)
>> > >> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
>> > >> at
>> > >>
>> >
>> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
>> > >> at
>> > >>
>> >
>> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
>> > >> at
>> > >>
>> >
>> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
>> > >> at
>> > >>
>> >
>> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
>> > >> at
>> > >>
>> >
>> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
>> > >> at
>> > >>
>> >
>> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
>> > >> at
>> > >>
>> >
>> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
>> > >> at
>> > >>
>> >
>> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
>> > >> at
>> > >>
>> >
>> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
>> > >> at
>> > >>
>> >
>> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
>> > >> at
>> > >>
>> >
>> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
>> > >> at
>> > >>
>> >
>> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
>> > >> at
>> > >>
>> >
>> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
>> > >> at
>> > >>
>> > org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
>> > >> at
>> > >>
>> >
>> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
>> > >> at
>> > >>
>> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
>> > >> at
>> > org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
>> > >> at
>> > >>
>> >
>> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
>> > >> at
>> > >>
>> >
>> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
>> > >> at
>> > >> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
>> > >> at
>> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
>> > >> at
>> > >>
>> >
>> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
>> > >> at
>> > >>
>> >
>> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
>> > >> at
>> > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
>> > >> at
>> org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
>> > >> at
>> > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
>> > >> at
>> > >> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
>> > >> at
>> > >>
>> org.apache.catalina.core.StandardService.start(StandardService.java:516)
>> > >> at
>> > >> org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
>> > >> at org.apache.catalina.startup.Catalina.start(Catalina.java:593)
>> > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > >> at
>> > >>
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> > >> at
>> > >>
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > >> at java.lang.reflect.Method.invoke(Method.java:606)
>> > >> at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
>> > >> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
>> > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl] (main:null)
>> > >> Detected that another management node with the same IP 10.97.38.10 is
>> > >> considered as running in DB, however it is not pingable, we will
>> > continue
>> > >> cluster initialization with this management server node
>> > >>
>> > >> I'm not sure what that error means, but more than likely that's the
>> > reason
>> > >> why the Virtual Router will not start.
>> > >>
>> > >> * PLEASE HELP ! *
>> > >>
>> > >> Best Regards,
>> > >>
>> > >>
>> > >>
>> > >> Adam Scarcella
>> > >>
>> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro> wrote:
>> > >>
>> > >>> Hi,
>> > >>>
>> > >>> More agent and management logs please, make sure they are in DEBUG[1]
>> > mode.
>> > >>>
>> > >>> [1] -
>> > >>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
>> > >>>
>> > >>> --
>> > >>> Sent from the Delta quadrant using Borg technology!
>> > >>>
>> > >>> Nux!
>> > >>> www.nux.ro
>> > >>>
>> > >>> ----- Original Message -----
>> > >>> > From: "Adam" <ad...@gmail.com>
>> > >>> > To: users@cloudstack.apache.org
>> > >>> > Sent: Sunday, 2 November, 2014 23:33:22
>> > >>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1
>> > >>>
>> > >>> > I've performed a complete upgrade from 4.3 to 4.4.1 and followed
>> the
>> > >>> > instructions perfectly, but my Virtual Router will not start, nor
>> am
>> > I
>> > >>> able
>> > >>> > to "upgrade" it.
>> > >>> >
>> > >>> > I tried destroying the Virtual Router, starting any VM Instance,
>> and
>> > >>> > allowing CS to recreate the VR. However that didn't help. The VR
>> > still
>> > >>> did
>> > >>> > not start and it was still saying that it required to be upgraded.
>> > >>> >
>> > >>> > [image: Inline image 1]
>> > >>> > [image: Inline image 2]
>> > >>> > [image: Inline image 3]
>> > >>> >
>> > >>> > I know there are manual steps to finalize the systemvm upgrade, but
>> > I did
>> > >>> > those steps:
>> > >>> >
>> > >>> >
>> > >>>
>> >
>> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
>> > >>> >
>> > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
>> > >>> > Enter password:
>> > >>> > Welcome to the MySQL monitor. Commands end with ; or \g.
>> > >>> > Your MySQL connection id is 174
>> > >>> > Server version: 5.1.73-log Source distribution
>> > >>> >
>> > >>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights
>> > >>> reserved.
>> > >>> >
>> > >>> > Oracle is a registered trademark of Oracle Corporation and/or its
>> > >>> > affiliates. Other names may be trademarks of their respective
>> > >>> > owners.
>> > >>> >
>> > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the current input
>> > >>> > statement.
>> > >>> >
>> > >>> > mysql> select id,name from cloud.vm_template where name =
>> > >>> > 'systemvm-kvm-4.4';
>> > >>> > +-----+------------------+
>> > >>> >| id | name |
>> > >>> > +-----+------------------+
>> > >>> >| 274 | systemvm-kvm-4.4 |
>> > >>> > +-----+------------------+
>> > >>> > 1 row in set (0.00 sec)
>> > >>> >
>> > >>> > mysql> select type from cloud.vm_template where id='274';
>> > >>> > +--------+
>> > >>> >| type |
>> > >>> > +--------+
>> > >>> >| SYSTEM |
>> > >>> > +--------+
>> > >>> > 1 row in set (0.00 sec)
>> > >>> >
>> > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance where
>> > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
>> > >>> > +----------------+
>> > >>> >| vm_template_id |
>> > >>> > +----------------+
>> > >>> >| 274 |
>> > >>> > +----------------+
>> > >>> > 1 row in set (0.00 sec)
>> > >>> >
>> > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance where
>> > >>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM';
>> > >>> > +----------------+
>> > >>> >| vm_template_id |
>> > >>> > +----------------+
>> > >>> >| 274 |
>> > >>> > +----------------+
>> > >>> > 1 row in set (0.03 sec)
>> > >>> >
>> > >>> > mysql> select value from cloud.configuration where name =
>> > >>> > 'router.template.kvm';
>> > >>> > +------------------+
>> > >>> >| value |
>> > >>> > +------------------+
>> > >>> >| systemvm-kvm-4.4 |
>> > >>> > +------------------+
>> > >>> > 1 row in set (0.00 sec)
>> > >>> >
>> > >>> > mysql>
>> > >>> >
>> > >>> > * PLEASE HELP ! *
>> > >>> >
>> > >>> > Best Regards,
>> > >>> >
>> > > >> > Adam Scarcella
>> >
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Adam <ad...@gmail.com>.
[root@cs-east-prod1 management]# netstat -a|grep 9090
unix 3 [ ] STREAM CONNECTED 29090
[root@cs-east-prod1 management]# nmap localhost
Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47 EST
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000015s latency).
Other addresses for localhost (not scanned): 127.0.0.1
Not shown: 990 closed ports
PORT STATE SERVICE
22/tcp open ssh
25/tcp open smtp
80/tcp open http
111/tcp open rpcbind
199/tcp open smux
631/tcp open ipp
2049/tcp open nfs
3306/tcp open mysql
8080/tcp open http-proxy
9090/tcp open zeus-admin
Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
[root@cs-east-prod1 management]# telnet localhost 9090
Trying ::1...
Connected to localhost.
Escape character is '^]'.
Connection closed by foreign host.
[root@cs-east-prod1 management]# nmap 10.97.38.10
Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49 EST
Nmap scan report for cs-east-prod1.na.tibco.com (10.97.38.10)
Host is up (0.000015s latency).
Not shown: 993 closed ports
PORT STATE SERVICE
22/tcp open ssh
80/tcp open http
111/tcp open rpcbind
2049/tcp open nfs
3306/tcp open mysql
8080/tcp open http-proxy
9090/tcp open zeus-admin
Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds
[root@cs-east-prod1 management]# telnet 10.97.38.10 9090
Trying 10.97.38.10...
Connected to 10.97.38.10.
Escape character is '^]'.
Connection closed by foreign host.
[root@cs-east-prod1 management]#
Also, I checked for the /opt/cloud/bin/ directory on my Management Host,
but it's not there:
[root@cs-east-prod1 management]# cd /opt/cloud/bin/
-bash: cd: /opt/cloud/bin/: No such file or directory
[root@cs-east-prod1 management]# cd /
[root@cs-east-prod1 /]# find . -name "createipAlias.sh"
[root@cs-east-prod1 /]# find . -name "createIpAlias.sh"
[root@cs-east-prod1 /]#
Are you saying that the /opt/cloud/bin/createipAlias.sh script is inside
the Virtual Router VM? If so, how could I possibly change that since it
won't start?
Best Regards,
Adam Scarcella
On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion <pd...@apache.org> wrote:
> Could it be possible the firewall is blocking port 9090 from the
> management-server?
>
> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null) Unable
> to ping management server at 10.97.38.10:9090 due to ConnectException
> java.net.ConnectException: Connection refused
>
> On Mon, Nov 3, 2014 at 9:02 AM, Nux! <nu...@li.nux.ro> wrote:
>
> > BTW the agent logs should help clarify this.
> > The aforementioned changes must be done either in the current (broken) VR
> > or in the systemvm (after which you should redeploy the VR to pick up the
> > change).
> >
> > HTH
> > Lucian
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> > ----- Original Message -----
> > > From: "Nux!" <nu...@li.nux.ro>
> > > To: users@cloudstack.apache.org
> > > Sent: Monday, 3 November, 2014 13:53:10
> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >
> > > Hello,
> > >
> > > I think you are hitting a problem I also had:
> > > "2014-11-03 08:16:17,024 ERROR [c.c.v.VirtualMachineManagerImpl]
> > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556 ctx-fcb072b2)
> > Failed to
> > > start instance VM[DomainRouter|r-4758-VM]
> > > com.cloud.utils.exception.ExecutionException: Unable to start
> > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart, not retrying"
> > >
> > > It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781
> > >
> > > Basically the systemvm template has some problems, you need to somehow
> > copy
> > > /opt/cloud/bin/createIpAlias.sh to /opt/cloud/bin/createipAlias.sh to
> > get it
> > > working.
> > >
> > > --
> > > Sent from the Delta quadrant using Borg technology!
> > >
> > > Nux!
> > > www.nux.ro
> > >
> > > ----- Original Message -----
> > >> From: "Adam" <ad...@gmail.com>
> > >> To: users@cloudstack.apache.org
> > >> Sent: Monday, 3 November, 2014 13:32:12
> > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> > >
> > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release
> > >> CentOS release 6.6 (Final)
> > >> [root@cs-east-prod1 ~]# uname -a
> > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16
> UTC
> > >> 2014 x86_64 x86_64 x86_64 GNU/Linux
> > >> [root@cs-east-prod1 ~]# virsh --version
> > >> 0.10.2
> > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
> > >> /etc/cloudstack/agent/log4j-cloud.xml
> > >> [root@cs-east-prod1 ~]# service cloudstack-management stop
> > >> Stopping cloudstack-management: [ OK ]
> > >> [root@cs-east-prod1 ~]# mv
> > >> /var/log/cloudstack/management/management-server.log
> > >> /var/log/cloudstack/management/management-server.bak
> > >> [root@cs-east-prod1 ~]# service cloudstack-management start
> > >> Starting cloudstack-management: [ OK ]
> > >> [root@cs-east-prod1 ~]#
> > >>
> > >> Please see attached ZIP of clean management-server.log in DEBUG mode.
> At
> > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the attached
> > >> management-server.log*) I tried starting the Virtual Router from the
> > >> Management UI and received this "Status" message in the UI.
> > >>
> > >> [image: Inline image 1]
> > >>
> > >> I can see in the management-server.log that while the
> > cloudstack-management
> > >> service is initializing, it throws the following ERROR at exactly
> > (*2014-11-03
> > >> 08:14:36,813 on line 342 of the attached management-server.log*):
> > >>
> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null)
> > Unable
> > >> to ping management server at 10.97.38.10:9090 due to ConnectException
> > >> java.net.ConnectException: Connection refused
> > >> at sun.nio.ch.Net.connect0(Native Method)
> > >> at sun.nio.ch.Net.connect(Net.java:465)
> > >> at sun.nio.ch.Net.connect(Net.java:457)
> > >> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
> > >> at
> > >>
> >
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
> > >> at
> > >>
> >
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
> > >> at
> > >>
> >
> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
> > >> at
> > >>
> >
> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
> > >> at
> > >>
> >
> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
> > >> at
> > >>
> >
> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
> > >> at
> > >>
> >
> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
> > >> at
> > >>
> >
> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
> > >> at
> > >>
> >
> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
> > >> at
> > >>
> >
> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
> > >> at
> > >>
> >
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
> > >> at
> > >>
> >
> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
> > >> at
> > >>
> >
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
> > >> at
> > >>
> > org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
> > >> at
> > >>
> >
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
> > >> at
> > >>
> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
> > >> at
> > org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
> > >> at
> > >>
> >
> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
> > >> at
> > >>
> >
> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
> > >> at
> > >> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
> > >> at
> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
> > >> at
> > >>
> >
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
> > >> at
> > >>
> >
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> > >> at
> > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
> > >> at
> org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
> > >> at
> > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
> > >> at
> > >> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
> > >> at
> > >>
> org.apache.catalina.core.StandardService.start(StandardService.java:516)
> > >> at
> > >> org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
> > >> at org.apache.catalina.startup.Catalina.start(Catalina.java:593)
> > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >> at
> > >>
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > >> at
> > >>
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > >> at java.lang.reflect.Method.invoke(Method.java:606)
> > >> at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> > >> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl] (main:null)
> > >> Detected that another management node with the same IP 10.97.38.10 is
> > >> considered as running in DB, however it is not pingable, we will
> > continue
> > >> cluster initialization with this management server node
> > >>
> > >> I'm not sure what that error means, but more than likely that's the
> > reason
> > >> why the Virtual Router will not start.
> > >>
> > >> * PLEASE HELP ! *
> > >>
> > >> Best Regards,
> > >>
> > >>
> > >>
> > >> Adam Scarcella
> > >>
> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro> wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> More agent and management logs please, make sure they are in DEBUG[1]
> > mode.
> > >>>
> > >>> [1] -
> > >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
> > >>>
> > >>> --
> > >>> Sent from the Delta quadrant using Borg technology!
> > >>>
> > >>> Nux!
> > >>> www.nux.ro
> > >>>
> > >>> ----- Original Message -----
> > >>> > From: "Adam" <ad...@gmail.com>
> > >>> > To: users@cloudstack.apache.org
> > >>> > Sent: Sunday, 2 November, 2014 23:33:22
> > >>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1
> > >>>
> > >>> > I've performed a complete upgrade from 4.3 to 4.4.1 and followed
> the
> > >>> > instructions perfectly, but my Virtual Router will not start, nor
> am
> > I
> > >>> able
> > >>> > to "upgrade" it.
> > >>> >
> > >>> > I tried destroying the Virtual Router, starting any VM Instance,
> and
> > >>> > allowing CS to recreate the VR. However that didn't help. The VR
> > still
> > >>> did
> > >>> > not start and it was still saying that it required to be upgraded.
> > >>> >
> > >>> > [image: Inline image 1]
> > >>> > [image: Inline image 2]
> > >>> > [image: Inline image 3]
> > >>> >
> > >>> > I know there are manual steps to finalize the systemvm upgrade, but
> > I did
> > >>> > those steps:
> > >>> >
> > >>> >
> > >>>
> >
> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
> > >>> >
> > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
> > >>> > Enter password:
> > >>> > Welcome to the MySQL monitor. Commands end with ; or \g.
> > >>> > Your MySQL connection id is 174
> > >>> > Server version: 5.1.73-log Source distribution
> > >>> >
> > >>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights
> > >>> reserved.
> > >>> >
> > >>> > Oracle is a registered trademark of Oracle Corporation and/or its
> > >>> > affiliates. Other names may be trademarks of their respective
> > >>> > owners.
> > >>> >
> > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the current input
> > >>> > statement.
> > >>> >
> > >>> > mysql> select id,name from cloud.vm_template where name =
> > >>> > 'systemvm-kvm-4.4';
> > >>> > +-----+------------------+
> > >>> >| id | name |
> > >>> > +-----+------------------+
> > >>> >| 274 | systemvm-kvm-4.4 |
> > >>> > +-----+------------------+
> > >>> > 1 row in set (0.00 sec)
> > >>> >
> > >>> > mysql> select type from cloud.vm_template where id='274';
> > >>> > +--------+
> > >>> >| type |
> > >>> > +--------+
> > >>> >| SYSTEM |
> > >>> > +--------+
> > >>> > 1 row in set (0.00 sec)
> > >>> >
> > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance where
> > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
> > >>> > +----------------+
> > >>> >| vm_template_id |
> > >>> > +----------------+
> > >>> >| 274 |
> > >>> > +----------------+
> > >>> > 1 row in set (0.00 sec)
> > >>> >
> > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance where
> > >>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM';
> > >>> > +----------------+
> > >>> >| vm_template_id |
> > >>> > +----------------+
> > >>> >| 274 |
> > >>> > +----------------+
> > >>> > 1 row in set (0.03 sec)
> > >>> >
> > >>> > mysql> select value from cloud.configuration where name =
> > >>> > 'router.template.kvm';
> > >>> > +------------------+
> > >>> >| value |
> > >>> > +------------------+
> > >>> >| systemvm-kvm-4.4 |
> > >>> > +------------------+
> > >>> > 1 row in set (0.00 sec)
> > >>> >
> > >>> > mysql>
> > >>> >
> > >>> > * PLEASE HELP ! *
> > >>> >
> > >>> > Best Regards,
> > >>> >
> > > >> > Adam Scarcella
> >
>
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Pierre-Luc Dion <pd...@apache.org>.
Could it be possible the firewall is blocking port 9090 from the
management-server?
2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null) Unable
to ping management server at 10.97.38.10:9090 due to ConnectException
java.net.ConnectException: Connection refused
On Mon, Nov 3, 2014 at 9:02 AM, Nux! <nu...@li.nux.ro> wrote:
> BTW the agent logs should help clarify this.
> The aforementioned changes must be done either in the current (broken) VR
> or in the systemvm (after which you should redeploy the VR to pick up the
> change).
>
> HTH
> Lucian
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
> > From: "Nux!" <nu...@li.nux.ro>
> > To: users@cloudstack.apache.org
> > Sent: Monday, 3 November, 2014 13:53:10
> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>
> > Hello,
> >
> > I think you are hitting a problem I also had:
> > "2014-11-03 08:16:17,024 ERROR [c.c.v.VirtualMachineManagerImpl]
> > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556 ctx-fcb072b2)
> Failed to
> > start instance VM[DomainRouter|r-4758-VM]
> > com.cloud.utils.exception.ExecutionException: Unable to start
> > VM[DomainRouter|r-4758-VM] due to error in finalizeStart, not retrying"
> >
> > It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781
> >
> > Basically the systemvm template has some problems, you need to somehow
> copy
> > /opt/cloud/bin/createIpAlias.sh to /opt/cloud/bin/createipAlias.sh to
> get it
> > working.
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> > ----- Original Message -----
> >> From: "Adam" <ad...@gmail.com>
> >> To: users@cloudstack.apache.org
> >> Sent: Monday, 3 November, 2014 13:32:12
> >> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >
> >> [root@cs-east-prod1 ~]# cat /etc/redhat-release
> >> CentOS release 6.6 (Final)
> >> [root@cs-east-prod1 ~]# uname -a
> >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16 UTC
> >> 2014 x86_64 x86_64 x86_64 GNU/Linux
> >> [root@cs-east-prod1 ~]# virsh --version
> >> 0.10.2
> >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
> >> /etc/cloudstack/agent/log4j-cloud.xml
> >> [root@cs-east-prod1 ~]# service cloudstack-management stop
> >> Stopping cloudstack-management: [ OK ]
> >> [root@cs-east-prod1 ~]# mv
> >> /var/log/cloudstack/management/management-server.log
> >> /var/log/cloudstack/management/management-server.bak
> >> [root@cs-east-prod1 ~]# service cloudstack-management start
> >> Starting cloudstack-management: [ OK ]
> >> [root@cs-east-prod1 ~]#
> >>
> >> Please see attached ZIP of clean management-server.log in DEBUG mode. At
> >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the attached
> >> management-server.log*) I tried starting the Virtual Router from the
> >> Management UI and received this "Status" message in the UI.
> >>
> >> [image: Inline image 1]
> >>
> >> I can see in the management-server.log that while the
> cloudstack-management
> >> service is initializing, it throws the following ERROR at exactly
> (*2014-11-03
> >> 08:14:36,813 on line 342 of the attached management-server.log*):
> >>
> >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null)
> Unable
> >> to ping management server at 10.97.38.10:9090 due to ConnectException
> >> java.net.ConnectException: Connection refused
> >> at sun.nio.ch.Net.connect0(Native Method)
> >> at sun.nio.ch.Net.connect(Net.java:465)
> >> at sun.nio.ch.Net.connect(Net.java:457)
> >> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
> >> at
> >>
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
> >> at
> >>
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
> >> at
> >>
> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
> >> at
> >>
> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
> >> at
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
> >> at
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
> >> at
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
> >> at
> >>
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
> >> at
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
> >> at
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
> >> at
> >>
> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
> >> at
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
> >> at
> >>
> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
> >> at
> >>
> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
> >> at
> >>
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
> >> at
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
> >> at
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
> >> at
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
> >> at
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> >> at
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> >> at
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
> >> at
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
> >> at
> >>
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
> >> at
> >>
> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
> >> at
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
> >> at
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
> >> at
> >>
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
> >> at
> >>
> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
> >> at
> >>
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
> >> at
> >>
> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
> >> at
> >>
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
> >> at
> >> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
> >> at
> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
> >> at
> >>
> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
> >> at
> >>
> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
> >> at
> >> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
> >> at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
> >> at
> >>
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
> >> at
> >>
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> >> at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
> >> at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
> >> at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
> >> at
> >> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
> >> at
> >> org.apache.catalina.core.StandardService.start(StandardService.java:516)
> >> at
> >> org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
> >> at org.apache.catalina.startup.Catalina.start(Catalina.java:593)
> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> at
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> at
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> at java.lang.reflect.Method.invoke(Method.java:606)
> >> at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> >> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl] (main:null)
> >> Detected that another management node with the same IP 10.97.38.10 is
> >> considered as running in DB, however it is not pingable, we will
> continue
> >> cluster initialization with this management server node
> >>
> >> I'm not sure what that error means, but more than likely that's the
> reason
> >> why the Virtual Router will not start.
> >>
> >> * PLEASE HELP ! *
> >>
> >> Best Regards,
> >>
> >>
> >>
> >> Adam Scarcella
> >>
> >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro> wrote:
> >>
> >>> Hi,
> >>>
> >>> More agent and management logs please, make sure they are in DEBUG[1]
> mode.
> >>>
> >>> [1] -
> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
> >>>
> >>> --
> >>> Sent from the Delta quadrant using Borg technology!
> >>>
> >>> Nux!
> >>> www.nux.ro
> >>>
> >>> ----- Original Message -----
> >>> > From: "Adam" <ad...@gmail.com>
> >>> > To: users@cloudstack.apache.org
> >>> > Sent: Sunday, 2 November, 2014 23:33:22
> >>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1
> >>>
> >>> > I've performed a complete upgrade from 4.3 to 4.4.1 and followed the
> >>> > instructions perfectly, but my Virtual Router will not start, nor am
> I
> >>> able
> >>> > to "upgrade" it.
> >>> >
> >>> > I tried destroying the Virtual Router, starting any VM Instance, and
> >>> > allowing CS to recreate the VR. However that didn't help. The VR
> still
> >>> did
> >>> > not start and it was still saying that it required to be upgraded.
> >>> >
> >>> > [image: Inline image 1]
> >>> > [image: Inline image 2]
> >>> > [image: Inline image 3]
> >>> >
> >>> > I know there are manual steps to finalize the systemvm upgrade, but
> I did
> >>> > those steps:
> >>> >
> >>> >
> >>>
> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
> >>> >
> >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
> >>> > Enter password:
> >>> > Welcome to the MySQL monitor. Commands end with ; or \g.
> >>> > Your MySQL connection id is 174
> >>> > Server version: 5.1.73-log Source distribution
> >>> >
> >>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights
> >>> reserved.
> >>> >
> >>> > Oracle is a registered trademark of Oracle Corporation and/or its
> >>> > affiliates. Other names may be trademarks of their respective
> >>> > owners.
> >>> >
> >>> > Type 'help;' or '\h' for help. Type '\c' to clear the current input
> >>> > statement.
> >>> >
> >>> > mysql> select id,name from cloud.vm_template where name =
> >>> > 'systemvm-kvm-4.4';
> >>> > +-----+------------------+
> >>> >| id | name |
> >>> > +-----+------------------+
> >>> >| 274 | systemvm-kvm-4.4 |
> >>> > +-----+------------------+
> >>> > 1 row in set (0.00 sec)
> >>> >
> >>> > mysql> select type from cloud.vm_template where id='274';
> >>> > +--------+
> >>> >| type |
> >>> > +--------+
> >>> >| SYSTEM |
> >>> > +--------+
> >>> > 1 row in set (0.00 sec)
> >>> >
> >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance where
> >>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
> >>> > +----------------+
> >>> >| vm_template_id |
> >>> > +----------------+
> >>> >| 274 |
> >>> > +----------------+
> >>> > 1 row in set (0.00 sec)
> >>> >
> >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance where
> >>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM';
> >>> > +----------------+
> >>> >| vm_template_id |
> >>> > +----------------+
> >>> >| 274 |
> >>> > +----------------+
> >>> > 1 row in set (0.03 sec)
> >>> >
> >>> > mysql> select value from cloud.configuration where name =
> >>> > 'router.template.kvm';
> >>> > +------------------+
> >>> >| value |
> >>> > +------------------+
> >>> >| systemvm-kvm-4.4 |
> >>> > +------------------+
> >>> > 1 row in set (0.00 sec)
> >>> >
> >>> > mysql>
> >>> >
> >>> > * PLEASE HELP ! *
> >>> >
> >>> > Best Regards,
> >>> >
> > >> > Adam Scarcella
>
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Nux! <nu...@li.nux.ro>.
BTW the agent logs should help clarify this.
The aforementioned changes must be done either in the current (broken) VR or in the systemvm (after which you should redeploy the VR to pick up the change).
HTH
Lucian
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Nux!" <nu...@li.nux.ro>
> To: users@cloudstack.apache.org
> Sent: Monday, 3 November, 2014 13:53:10
> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> Hello,
>
> I think you are hitting a problem I also had:
> "2014-11-03 08:16:17,024 ERROR [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556 ctx-fcb072b2) Failed to
> start instance VM[DomainRouter|r-4758-VM]
> com.cloud.utils.exception.ExecutionException: Unable to start
> VM[DomainRouter|r-4758-VM] due to error in finalizeStart, not retrying"
>
> It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781
>
> Basically the systemvm template has some problems, you need to somehow copy
> /opt/cloud/bin/createIpAlias.sh to /opt/cloud/bin/createipAlias.sh to get it
> working.
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
>> From: "Adam" <ad...@gmail.com>
>> To: users@cloudstack.apache.org
>> Sent: Monday, 3 November, 2014 13:32:12
>> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
>
>> [root@cs-east-prod1 ~]# cat /etc/redhat-release
>> CentOS release 6.6 (Final)
>> [root@cs-east-prod1 ~]# uname -a
>> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16 UTC
>> 2014 x86_64 x86_64 x86_64 GNU/Linux
>> [root@cs-east-prod1 ~]# virsh --version
>> 0.10.2
>> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
>> /etc/cloudstack/agent/log4j-cloud.xml
>> [root@cs-east-prod1 ~]# service cloudstack-management stop
>> Stopping cloudstack-management: [ OK ]
>> [root@cs-east-prod1 ~]# mv
>> /var/log/cloudstack/management/management-server.log
>> /var/log/cloudstack/management/management-server.bak
>> [root@cs-east-prod1 ~]# service cloudstack-management start
>> Starting cloudstack-management: [ OK ]
>> [root@cs-east-prod1 ~]#
>>
>> Please see attached ZIP of clean management-server.log in DEBUG mode. At
>> exactly (*2014-11-03 08:15:35,700 on line 3025 of the attached
>> management-server.log*) I tried starting the Virtual Router from the
>> Management UI and received this "Status" message in the UI.
>>
>> [image: Inline image 1]
>>
>> I can see in the management-server.log that while the cloudstack-management
>> service is initializing, it throws the following ERROR at exactly (*2014-11-03
>> 08:14:36,813 on line 342 of the attached management-server.log*):
>>
>> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null) Unable
>> to ping management server at 10.97.38.10:9090 due to ConnectException
>> java.net.ConnectException: Connection refused
>> at sun.nio.ch.Net.connect0(Native Method)
>> at sun.nio.ch.Net.connect(Net.java:465)
>> at sun.nio.ch.Net.connect(Net.java:457)
>> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
>> at
>> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
>> at
>> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
>> at
>> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
>> at
>> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
>> at
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
>> at
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
>> at
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
>> at
>> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
>> at
>> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
>> at
>> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
>> at
>> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
>> at
>> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
>> at
>> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
>> at
>> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
>> at
>> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
>> at
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
>> at
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
>> at
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
>> at
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
>> at
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
>> at
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
>> at
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
>> at
>> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
>> at
>> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
>> at
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
>> at
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
>> at
>> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
>> at
>> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
>> at
>> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
>> at
>> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
>> at
>> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
>> at
>> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
>> at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
>> at
>> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
>> at
>> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
>> at
>> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
>> at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
>> at
>> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
>> at
>> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
>> at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
>> at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
>> at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
>> at
>> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
>> at
>> org.apache.catalina.core.StandardService.start(StandardService.java:516)
>> at
>> org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
>> at org.apache.catalina.startup.Catalina.start(Catalina.java:593)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
>> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
>> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl] (main:null)
>> Detected that another management node with the same IP 10.97.38.10 is
>> considered as running in DB, however it is not pingable, we will continue
>> cluster initialization with this management server node
>>
>> I'm not sure what that error means, but more than likely that's the reason
>> why the Virtual Router will not start.
>>
>> * PLEASE HELP ! *
>>
>> Best Regards,
>>
>>
>>
>> Adam Scarcella
>>
>> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro> wrote:
>>
>>> Hi,
>>>
>>> More agent and management logs please, make sure they are in DEBUG[1] mode.
>>>
>>> [1] -
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
>>>
>>> --
>>> Sent from the Delta quadrant using Borg technology!
>>>
>>> Nux!
>>> www.nux.ro
>>>
>>> ----- Original Message -----
>>> > From: "Adam" <ad...@gmail.com>
>>> > To: users@cloudstack.apache.org
>>> > Sent: Sunday, 2 November, 2014 23:33:22
>>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1
>>>
>>> > I've performed a complete upgrade from 4.3 to 4.4.1 and followed the
>>> > instructions perfectly, but my Virtual Router will not start, nor am I
>>> able
>>> > to "upgrade" it.
>>> >
>>> > I tried destroying the Virtual Router, starting any VM Instance, and
>>> > allowing CS to recreate the VR. However that didn't help. The VR still
>>> did
>>> > not start and it was still saying that it required to be upgraded.
>>> >
>>> > [image: Inline image 1]
>>> > [image: Inline image 2]
>>> > [image: Inline image 3]
>>> >
>>> > I know there are manual steps to finalize the systemvm upgrade, but I did
>>> > those steps:
>>> >
>>> >
>>> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
>>> >
>>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
>>> > Enter password:
>>> > Welcome to the MySQL monitor. Commands end with ; or \g.
>>> > Your MySQL connection id is 174
>>> > Server version: 5.1.73-log Source distribution
>>> >
>>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights
>>> reserved.
>>> >
>>> > Oracle is a registered trademark of Oracle Corporation and/or its
>>> > affiliates. Other names may be trademarks of their respective
>>> > owners.
>>> >
>>> > Type 'help;' or '\h' for help. Type '\c' to clear the current input
>>> > statement.
>>> >
>>> > mysql> select id,name from cloud.vm_template where name =
>>> > 'systemvm-kvm-4.4';
>>> > +-----+------------------+
>>> >| id | name |
>>> > +-----+------------------+
>>> >| 274 | systemvm-kvm-4.4 |
>>> > +-----+------------------+
>>> > 1 row in set (0.00 sec)
>>> >
>>> > mysql> select type from cloud.vm_template where id='274';
>>> > +--------+
>>> >| type |
>>> > +--------+
>>> >| SYSTEM |
>>> > +--------+
>>> > 1 row in set (0.00 sec)
>>> >
>>> > mysql> select distinct(vm_template_id) from cloud.vm_instance where
>>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
>>> > +----------------+
>>> >| vm_template_id |
>>> > +----------------+
>>> >| 274 |
>>> > +----------------+
>>> > 1 row in set (0.00 sec)
>>> >
>>> > mysql> select distinct(vm_template_id) from cloud.vm_instance where
>>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM';
>>> > +----------------+
>>> >| vm_template_id |
>>> > +----------------+
>>> >| 274 |
>>> > +----------------+
>>> > 1 row in set (0.03 sec)
>>> >
>>> > mysql> select value from cloud.configuration where name =
>>> > 'router.template.kvm';
>>> > +------------------+
>>> >| value |
>>> > +------------------+
>>> >| systemvm-kvm-4.4 |
>>> > +------------------+
>>> > 1 row in set (0.00 sec)
>>> >
>>> > mysql>
>>> >
>>> > * PLEASE HELP ! *
>>> >
>>> > Best Regards,
>>> >
> >> > Adam Scarcella
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Nux! <nu...@li.nux.ro>.
Hello,
I think you are hitting a problem I also had:
"2014-11-03 08:16:17,024 ERROR [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556 ctx-fcb072b2) Failed to start instance VM[DomainRouter|r-4758-VM]
com.cloud.utils.exception.ExecutionException: Unable to start VM[DomainRouter|r-4758-VM] due to error in finalizeStart, not retrying"
It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781
Basically the systemvm template has some problems, you need to somehow copy /opt/cloud/bin/createIpAlias.sh to /opt/cloud/bin/createipAlias.sh to get it working.
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Adam" <ad...@gmail.com>
> To: users@cloudstack.apache.org
> Sent: Monday, 3 November, 2014 13:32:12
> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
> [root@cs-east-prod1 ~]# cat /etc/redhat-release
> CentOS release 6.6 (Final)
> [root@cs-east-prod1 ~]# uname -a
> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16 UTC
> 2014 x86_64 x86_64 x86_64 GNU/Linux
> [root@cs-east-prod1 ~]# virsh --version
> 0.10.2
> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
> /etc/cloudstack/agent/log4j-cloud.xml
> [root@cs-east-prod1 ~]# service cloudstack-management stop
> Stopping cloudstack-management: [ OK ]
> [root@cs-east-prod1 ~]# mv
> /var/log/cloudstack/management/management-server.log
> /var/log/cloudstack/management/management-server.bak
> [root@cs-east-prod1 ~]# service cloudstack-management start
> Starting cloudstack-management: [ OK ]
> [root@cs-east-prod1 ~]#
>
> Please see attached ZIP of clean management-server.log in DEBUG mode. At
> exactly (*2014-11-03 08:15:35,700 on line 3025 of the attached
> management-server.log*) I tried starting the Virtual Router from the
> Management UI and received this "Status" message in the UI.
>
> [image: Inline image 1]
>
> I can see in the management-server.log that while the cloudstack-management
> service is initializing, it throws the following ERROR at exactly (*2014-11-03
> 08:14:36,813 on line 342 of the attached management-server.log*):
>
> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null) Unable
> to ping management server at 10.97.38.10:9090 due to ConnectException
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method)
> at sun.nio.ch.Net.connect(Net.java:465)
> at sun.nio.ch.Net.connect(Net.java:457)
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
> at
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
> at
> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
> at
> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
> at
> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
> at
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
> at
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
> at
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
> at
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
> at
> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
> at
> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
> at
> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
> at
> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
> at
> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
> at
> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
> at
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
> at
> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
> at
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
> at
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
> at
> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
> at
> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
> at
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
> at
> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
> at
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
> at
> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
> at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
> at
> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
> at
> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
> at
> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
> at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
> at
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
> at
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
> at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
> at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
> at
> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
> at
> org.apache.catalina.core.StandardService.start(StandardService.java:516)
> at
> org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
> at org.apache.catalina.startup.Catalina.start(Catalina.java:593)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl] (main:null)
> Detected that another management node with the same IP 10.97.38.10 is
> considered as running in DB, however it is not pingable, we will continue
> cluster initialization with this management server node
>
> I'm not sure what that error means, but more than likely that's the reason
> why the Virtual Router will not start.
>
> * PLEASE HELP ! *
>
> Best Regards,
>
>
>
> Adam Scarcella
>
> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro> wrote:
>
>> Hi,
>>
>> More agent and management logs please, make sure they are in DEBUG[1] mode.
>>
>> [1] -
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
>>
>> ----- Original Message -----
>> > From: "Adam" <ad...@gmail.com>
>> > To: users@cloudstack.apache.org
>> > Sent: Sunday, 2 November, 2014 23:33:22
>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1
>>
>> > I've performed a complete upgrade from 4.3 to 4.4.1 and followed the
>> > instructions perfectly, but my Virtual Router will not start, nor am I
>> able
>> > to "upgrade" it.
>> >
>> > I tried destroying the Virtual Router, starting any VM Instance, and
>> > allowing CS to recreate the VR. However that didn't help. The VR still
>> did
>> > not start and it was still saying that it required to be upgraded.
>> >
>> > [image: Inline image 1]
>> > [image: Inline image 2]
>> > [image: Inline image 3]
>> >
>> > I know there are manual steps to finalize the systemvm upgrade, but I did
>> > those steps:
>> >
>> >
>> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
>> >
>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
>> > Enter password:
>> > Welcome to the MySQL monitor. Commands end with ; or \g.
>> > Your MySQL connection id is 174
>> > Server version: 5.1.73-log Source distribution
>> >
>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights
>> reserved.
>> >
>> > Oracle is a registered trademark of Oracle Corporation and/or its
>> > affiliates. Other names may be trademarks of their respective
>> > owners.
>> >
>> > Type 'help;' or '\h' for help. Type '\c' to clear the current input
>> > statement.
>> >
>> > mysql> select id,name from cloud.vm_template where name =
>> > 'systemvm-kvm-4.4';
>> > +-----+------------------+
>> >| id | name |
>> > +-----+------------------+
>> >| 274 | systemvm-kvm-4.4 |
>> > +-----+------------------+
>> > 1 row in set (0.00 sec)
>> >
>> > mysql> select type from cloud.vm_template where id='274';
>> > +--------+
>> >| type |
>> > +--------+
>> >| SYSTEM |
>> > +--------+
>> > 1 row in set (0.00 sec)
>> >
>> > mysql> select distinct(vm_template_id) from cloud.vm_instance where
>> > type='ConsoleProxy' and hypervisor_type = 'KVM';
>> > +----------------+
>> >| vm_template_id |
>> > +----------------+
>> >| 274 |
>> > +----------------+
>> > 1 row in set (0.00 sec)
>> >
>> > mysql> select distinct(vm_template_id) from cloud.vm_instance where
>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM';
>> > +----------------+
>> >| vm_template_id |
>> > +----------------+
>> >| 274 |
>> > +----------------+
>> > 1 row in set (0.03 sec)
>> >
>> > mysql> select value from cloud.configuration where name =
>> > 'router.template.kvm';
>> > +------------------+
>> >| value |
>> > +------------------+
>> >| systemvm-kvm-4.4 |
>> > +------------------+
>> > 1 row in set (0.00 sec)
>> >
>> > mysql>
>> >
>> > * PLEASE HELP ! *
>> >
>> > Best Regards,
>> >
>> > Adam Scarcella
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Adam <ad...@gmail.com>.
[root@cs-east-prod1 ~]# cat /etc/redhat-release
CentOS release 6.6 (Final)
[root@cs-east-prod1 ~]# uname -a
Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16 UTC
2014 x86_64 x86_64 x86_64 GNU/Linux
[root@cs-east-prod1 ~]# virsh --version
0.10.2
[root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g'
/etc/cloudstack/agent/log4j-cloud.xml
[root@cs-east-prod1 ~]# service cloudstack-management stop
Stopping cloudstack-management: [ OK ]
[root@cs-east-prod1 ~]# mv
/var/log/cloudstack/management/management-server.log
/var/log/cloudstack/management/management-server.bak
[root@cs-east-prod1 ~]# service cloudstack-management start
Starting cloudstack-management: [ OK ]
[root@cs-east-prod1 ~]#
Please see attached ZIP of clean management-server.log in DEBUG mode. At
exactly (*2014-11-03 08:15:35,700 on line 3025 of the attached
management-server.log*) I tried starting the Virtual Router from the
Management UI and received this "Status" message in the UI.
[image: Inline image 1]
I can see in the management-server.log that while the cloudstack-management
service is initializing, it throws the following ERROR at exactly (*2014-11-03
08:14:36,813 on line 342 of the attached management-server.log*):
2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null) Unable
to ping management server at 10.97.38.10:9090 due to ConnectException
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect0(Native Method)
at sun.nio.ch.Net.connect(Net.java:465)
at sun.nio.ch.Net.connect(Net.java:457)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
at
com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123)
at
com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092)
at
com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170)
at
com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057)
at
org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114)
at
org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153)
at
org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110)
at
org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56)
at
org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167)
at
org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
at
org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339)
at
org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
at
org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108)
at
org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
at
org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
at
org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
at
org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
at
org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
at
org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
at
org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
at
org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
at
org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
at
org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
at
org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
at
org.apache.catalina.core.StandardService.start(StandardService.java:516)
at
org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
at org.apache.catalina.startup.Catalina.start(Catalina.java:593)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl] (main:null)
Detected that another management node with the same IP 10.97.38.10 is
considered as running in DB, however it is not pingable, we will continue
cluster initialization with this management server node
I'm not sure what that error means, but more than likely that's the reason
why the Virtual Router will not start.
* PLEASE HELP ! *
Best Regards,
Adam Scarcella
On Sun, Nov 2, 2014 at 8:20 PM, Nux! <nu...@li.nux.ro> wrote:
> Hi,
>
> More agent and management logs please, make sure they are in DEBUG[1] mode.
>
> [1] -
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
> > From: "Adam" <ad...@gmail.com>
> > To: users@cloudstack.apache.org
> > Sent: Sunday, 2 November, 2014 23:33:22
> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1
>
> > I've performed a complete upgrade from 4.3 to 4.4.1 and followed the
> > instructions perfectly, but my Virtual Router will not start, nor am I
> able
> > to "upgrade" it.
> >
> > I tried destroying the Virtual Router, starting any VM Instance, and
> > allowing CS to recreate the VR. However that didn't help. The VR still
> did
> > not start and it was still saying that it required to be upgraded.
> >
> > [image: Inline image 1]
> > [image: Inline image 2]
> > [image: Inline image 3]
> >
> > I know there are manual steps to finalize the systemvm upgrade, but I did
> > those steps:
> >
> >
> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
> >
> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
> > Enter password:
> > Welcome to the MySQL monitor. Commands end with ; or \g.
> > Your MySQL connection id is 174
> > Server version: 5.1.73-log Source distribution
> >
> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights
> reserved.
> >
> > Oracle is a registered trademark of Oracle Corporation and/or its
> > affiliates. Other names may be trademarks of their respective
> > owners.
> >
> > Type 'help;' or '\h' for help. Type '\c' to clear the current input
> > statement.
> >
> > mysql> select id,name from cloud.vm_template where name =
> > 'systemvm-kvm-4.4';
> > +-----+------------------+
> >| id | name |
> > +-----+------------------+
> >| 274 | systemvm-kvm-4.4 |
> > +-----+------------------+
> > 1 row in set (0.00 sec)
> >
> > mysql> select type from cloud.vm_template where id='274';
> > +--------+
> >| type |
> > +--------+
> >| SYSTEM |
> > +--------+
> > 1 row in set (0.00 sec)
> >
> > mysql> select distinct(vm_template_id) from cloud.vm_instance where
> > type='ConsoleProxy' and hypervisor_type = 'KVM';
> > +----------------+
> >| vm_template_id |
> > +----------------+
> >| 274 |
> > +----------------+
> > 1 row in set (0.00 sec)
> >
> > mysql> select distinct(vm_template_id) from cloud.vm_instance where
> > type='SecondaryStorageVm' and hypervisor_type = 'KVM';
> > +----------------+
> >| vm_template_id |
> > +----------------+
> >| 274 |
> > +----------------+
> > 1 row in set (0.03 sec)
> >
> > mysql> select value from cloud.configuration where name =
> > 'router.template.kvm';
> > +------------------+
> >| value |
> > +------------------+
> >| systemvm-kvm-4.4 |
> > +------------------+
> > 1 row in set (0.00 sec)
> >
> > mysql>
> >
> > * PLEASE HELP ! *
> >
> > Best Regards,
> >
> > Adam Scarcella
>
Re: How to Upgrade Virtual Router from 4.3 to 4.4.1
Posted by Nux! <nu...@li.nux.ro>.
Hi,
More agent and management logs please, make sure they are in DEBUG[1] mode.
[1] - https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Adam" <ad...@gmail.com>
> To: users@cloudstack.apache.org
> Sent: Sunday, 2 November, 2014 23:33:22
> Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1
> I've performed a complete upgrade from 4.3 to 4.4.1 and followed the
> instructions perfectly, but my Virtual Router will not start, nor am I able
> to "upgrade" it.
>
> I tried destroying the Virtual Router, starting any VM Instance, and
> allowing CS to recreate the VR. However that didn't help. The VR still did
> not start and it was still saying that it required to be upgraded.
>
> [image: Inline image 1]
> [image: Inline image 2]
> [image: Inline image 3]
>
> I know there are manual steps to finalize the systemvm upgrade, but I did
> those steps:
>
> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix
>
> [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p
> Enter password:
> Welcome to the MySQL monitor. Commands end with ; or \g.
> Your MySQL connection id is 174
> Server version: 5.1.73-log Source distribution
>
> Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
>
> Oracle is a registered trademark of Oracle Corporation and/or its
> affiliates. Other names may be trademarks of their respective
> owners.
>
> Type 'help;' or '\h' for help. Type '\c' to clear the current input
> statement.
>
> mysql> select id,name from cloud.vm_template where name =
> 'systemvm-kvm-4.4';
> +-----+------------------+
>| id | name |
> +-----+------------------+
>| 274 | systemvm-kvm-4.4 |
> +-----+------------------+
> 1 row in set (0.00 sec)
>
> mysql> select type from cloud.vm_template where id='274';
> +--------+
>| type |
> +--------+
>| SYSTEM |
> +--------+
> 1 row in set (0.00 sec)
>
> mysql> select distinct(vm_template_id) from cloud.vm_instance where
> type='ConsoleProxy' and hypervisor_type = 'KVM';
> +----------------+
>| vm_template_id |
> +----------------+
>| 274 |
> +----------------+
> 1 row in set (0.00 sec)
>
> mysql> select distinct(vm_template_id) from cloud.vm_instance where
> type='SecondaryStorageVm' and hypervisor_type = 'KVM';
> +----------------+
>| vm_template_id |
> +----------------+
>| 274 |
> +----------------+
> 1 row in set (0.03 sec)
>
> mysql> select value from cloud.configuration where name =
> 'router.template.kvm';
> +------------------+
>| value |
> +------------------+
>| systemvm-kvm-4.4 |
> +------------------+
> 1 row in set (0.00 sec)
>
> mysql>
>
> * PLEASE HELP ! *
>
> Best Regards,
>
> Adam Scarcella