You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Fariborz Navidan <md...@gmail.com> on 2019/10/02 09:32:36 UTC
Upgrade to 4.13 failed
Hello,
Upgrade to 4.13 was failed and downgrade to 4.12 does not work due to
datbase modifications and failing to roll back. I failed to take backup of
database as well. I have created a rollback sql based on error log step by
step after running manual rollback commands. I do run this sql before every
mgmt restart and add more rollback commands. After last time I went through
this cycle, it seems cloudstack was able to alter all tables and
insert records. Now I' ended up with this error which I don't what does it
mean:
4.13.0.0KVM SystemVm template not found. Cannot upgrade system Vms
Re: Upgrade to 4.13 failed
Posted by Andrija Panic <an...@gmail.com>.
Cool!
On Thu, 3 Oct 2019, 21:02 Fariborz Navidan, <md...@gmail.com> wrote:
> Thank you very much! Finally, I could start management server normally.
>
> Thanks.
>
> On Thu, Oct 3, 2019 at 9:26 PM Andrija Panic <an...@gmail.com>
> wrote:
>
> > It seems easier than I thought...
> >
> > stop mgmt server
> > Rollback DB with your statement shared in some of the previous emails
> (and
> > after that please make DB backup immediately, just for the sake of having
> > one...)
> > Again, perform that manual template install with:
> > - From mgmt, mount the Secondary Storage NFS on /mnt/secondary and
> > execute:
> >
> >
> >
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> > -m /mnt/secondary -u
> >
> >
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> > -h
> > <
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2-h
> >
> > kvm -F
> > umount /mnt/secondary
> >
> > Update some fields for that template:
> > update vm_template set name="systemvm-kvm-4.11.3" where id="3";
> > update vm_template set display_text="systemvm-kvm-4.11.3" where
> id="3";
> > update vm_template set type="USER" where id="3";
> > Check that ID=3 for default KVM system template - but should be identical
> > on everyones env.
> >
> > CloudStack upgrade is looking for specific NAME of the template in order
> to
> > proceed with the upgrade (and initially type=USER, and the upgrade will
> > change that again to SYSTEM)
> > After those SQL changes, start mgmt server - all should be good.
> > If mgmt is started fine, you will have to DESTROY all system VMs and VRs,
> > i.e. NOT reboot - to force using new template - so Destroy SSVM and CPVM
> > (will be recreated automatically) and for the network/VPC - restart them
> > with "Clean Up" option.
> >
> > My previous instructions didn't work, since I was doing more things than
> > what I've shared with you - sorry for that - but those steps are needed
> > anyway, just were incomplete.
> >
> > (We could do different fix by manually inserting DB records for a
> > completely new template, new rows in vm_template, templat_store_ref and
> > template_zone_ref and unpack that BZ2 file/template to specific folder on
> > Secondary Storage - but that would require a bit of MySQL and is more
> > error-prone. This kind of fix would simulate the step of manually
> > registering new systemVM template as regular user template, with the
> > specific name - which would be later recognized by it's name as the new
> > systemVM templat for 4.11.3)
> >
> > Let me know how it goes.
> > Andrija
> >
> > On Thu, 3 Oct 2019 at 19:07, Fariborz Navidan <md...@gmail.com>
> > wrote:
> >
> > > MariaDB [cloud]> select max(id) from vm_template;
> > > +---------+
> > > | max(id) |
> > > +---------+
> > > | 390 |
> > > +---------+
> > >
> > > Can we have a chat session to investigate together?
> > >
> > > On Thu, Oct 3, 2019 at 8:27 PM Andrija Panic <an...@gmail.com>
> > > wrote:
> > >
> > > > do for me:
> > > >
> > > > select max(id) from vm_template:
> > > >
> > > > i might need more info later...
> > > >
> > > > On Thu, 3 Oct 2019 at 18:25, Andrija Panic <an...@gmail.com>
> > > > wrote:
> > > >
> > > > > Can you share logs during this last attempt?
> > > > >
> > > > > I'll try to craft some SQL additions for your vm_template and
> > > > > template_store_ref tables while also manually downloading and
> > unpacking
> > > > the
> > > > > file to specific folder on Secondary Storage...
> > > > >
> > > > > Andrija
> > > > >
> > > > > On Thu, 3 Oct 2019 at 16:55, Fariborz Navidan <
> mdvlinquest@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > >> My cloud is down for 2 days. I would really appreciate if you can
> > help
> > > > me
> > > > >> fix the issue.
> > > > >>
> > > > >> On Thu, Oct 3, 2019 at 4:44 PM Fariborz Navidan <
> > > mdvlinquest@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Any idea?
> > > > >> >
> > > > >> > On Thu, Oct 3, 2019 at 2:09 PM Fariborz Navidan <
> > > > mdvlinquest@gmail.com>
> > > > >> > wrote:
> > > > >> >
> > > > >> >> I did the steps exactly but still same error. It first says
> > cannot
> > > > >> >> upgrade database with no further sql errors stated. Thn says
> > cannot
> > > > >> find
> > > > >> >> System VM template for KVM. It look likes system vm template
> > 4.11.3
> > > > is
> > > > >> >> installed only for router!
> > > > >> >>
> > > > >> >> MariaDB [cloud]> select * from vm_template where name like
> > > > 'SystemVM%'
> > > > >> >> and removed is null;
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> > > > >> >> | id | unique_name | name | uuid
> > > > >> >> | public | featured | type | hvm | bits | url
> > > > >> >>
> > > > |
> > > > >> >> format | created | removed | account_id | checksum
> > > > >> >> | display_text | enable_password |
> > > > >> >> enable_sshkey | guest_os_id | bootable | prepopulate |
> > cross_zones
> > > |
> > > > >> >> extractable | hypervisor_type | source_template_id |
> > template_tag |
> > > > >> >> sort_key | size | state | update_count | updated |
> > > > >> >> dynamically_scalable | parent_template_id | direct_download |
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> > > > >> >> | 1 | routing-1 | SystemVM Template (XenServer) |
> > > > >> >> c1ea2f68-ca75-11e9-8212-d05099d38681 | 0 | 0 |
> > SYSTEM |
> > > > >> 0 |
> > > > >> >> 64 |
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-xen.vhd.bz2
> > > > >> >> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
> > > > >> >> 6e8b3ae84ca8145736d1d7d3f7546e65 | SystemVM Template
> (XenServer)
> > |
> > > > >> >> 0 | 0 | 184 | 1 | 0
> |
> > > > >> >> 1 | 0 | XenServer | NULL | NULL
> > > > |
> > > > >> >> 0 | NULL | Active | 0 | NULL |
> > > > >> 0 |
> > > > >> >> NULL | 0 |
> > > > >> >> | 3 | routing-3 | SystemVM Template (KVM) |
> > > > >> >> e4621347-39b8-427e-a55f-82cf51275b69 | 0 | 0 |
> > SYSTEM |
> > > > >> 0 |
> > > > >> >> 64 |
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> > > > >> >> | QCOW2 | 2019-08-29 17:57:52 | NULL | 1 |
> > > > >> >> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (KVM)
> > |
> > > > >> >> 0 | 0 | 15 | 1 | 0
> |
> > > > >> >> 1 | 0 | KVM | NULL | NULL
> > > > |
> > > > >> >> 0 | 385242624 | Active | 0 | NULL |
> > > > >> 0 |
> > > > >> >> NULL | 0 |
> > > > >> >> | 8 | routing-8 | SystemVM Template (vSphere) |
> > > > >> >> c1ea7e39-ca75-11e9-8212-d05099d38681 | 0 | 0 |
> > SYSTEM |
> > > > >> 0 |
> > > > >> >> 64 |
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-vmware.ova
> > > > >> >> | OVA | 2019-08-29 17:57:52 | NULL | 1 |
> > > > >> >> e981f8cb951688efd93481913198c9cc | SystemVM Template (vSphere)
> > |
> > > > >> >> 0 | 0 | 99 | 1 | 0
> |
> > > > >> >> 1 | 0 | VMware | NULL | NULL
> > > > |
> > > > >> >> 0 | NULL | Active | 0 | NULL |
> > > > >> 1 |
> > > > >> >> NULL | 0 |
> > > > >> >> | 9 | routing-9 | SystemVM Template (HyperV) |
> > > > >> >> c1eaa7b0-ca75-11e9-8212-d05099d38681 | 0 | 0 |
> > SYSTEM |
> > > > >> 0 |
> > > > >> >> 64 |
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-hyperv.vhd.zip
> > > > >> >> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
> > > > >> >> e9032635ffba021371780307162551b9 | SystemVM Template (HyperV)
> > |
> > > > >> >> 0 | 0 | 15 | 1 | 0
> |
> > > > >> >> 1 | 0 | Hyperv | NULL | NULL
> > > > |
> > > > >> >> 0 | NULL | Active | 0 | NULL |
> > > > >> 0 |
> > > > >> >> NULL | 0 |
> > > > >> >> | 10 | routing-10 | SystemVM Template (LXC) |
> > > > >> >> 6706cad5-ca77-11e9-8212-d05099d38681 | 0 | 0 |
> > SYSTEM |
> > > > >> 0 |
> > > > >> >> 64 |
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
> > > > >> >> | QCOW2 | 2019-08-29 18:09:38 | NULL | 1 |
> > > > >> >> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (LXC)
> > |
> > > > >> >> 0 | 0 | 15 | 1 | 0
> |
> > > > >> >> 1 | 0 | LXC | NULL | NULL
> > > > |
> > > > >> >> 0 | NULL | Active | 0 | NULL |
> > > > >> 0 |
> > > > >> >> NULL | 0 |
> > > > >> >> | 12 | routing-12 | SystemVM Template (Ovm3) |
> > > > >> >> 694d5de8-ca77-11e9-8212-d05099d38681 | 0 | 0 |
> > SYSTEM |
> > > > >> 0 |
> > > > >> >> 64 |
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-ovm.raw.bz2
> > > > >> >> | RAW | 2019-08-29 18:09:42 | NULL | 1 |
> > > > >> >> c4a91f8e52e4531a1c2a9a17c530d5fe | SystemVM Template (Ovm3)
> > |
> > > > >> >> 0 | 0 | 183 | 1 | 0
> |
> > > > >> >> 1 | 0 | Ovm3 | NULL | NULL
> > > > |
> > > > >> >> 0 | NULL | Active | NULL | NULL |
> > > > >> 0 |
> > > > >> >> NULL | 0 |
> > > > >> >>
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> > > > >> >> 6 rows in set (0.00 sec)
> > > > >> >>
> > > > >> >> On Thu, Oct 3, 2019 at 1:52 AM Andrija Panic <
> > > > andrija.panic@gmail.com>
> > > > >> >> wrote:
> > > > >> >>
> > > > >> >>> - ake sure mgmt is stopped (you can leave it on 4.13 if not
> > > > >> >>> already downgraded), stop cloudstack-agents on KVM hosts.
> > > > >> >>> - Rollback DB with your SQL statements, that you shared a few
> > > emails
> > > > >> ago
> > > > >> >>> (unfortunately after this 4.12 is NOT possible to work
> > > normally...)
> > > > >> >>> -Download the new systemVM template - it WILL OVERWRITE your
> > old
> > > > one
> > > > >> -
> > > > >> >>> i.e.:
> > > > >> >>> From mgmt, mount the Secondary Storage NFS on
> > > /mnt/secondary
> > > > >> and
> > > > >> >>> execute:
> > > > >> >>>
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> > > > >> >>> -m /mnt/secondary -u
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> > > > >> >>> -h
> > > > >> >>> <
> > > > >>
> > > >
> > >
> >
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2-h
> > > > >> >
> > > > >> >>> kvm -F
> > > > >> >>> This will overwrite existing systemVM template on the
> > > > >> >>> /mnt/secondary/templates/1/3/xxxxxx and will update the DB
> with
> > > new
> > > > >> names
> > > > >> >>> etc...
> > > > >> >>> - Upgrade packages to 4.13 now (if not already having 4.13
> > > > installed)
> > > > >> >>> - start mgmt - upgrade path will find the new systemVM
> template
> > > and
> > > > >> mgmt
> > > > >> >>> server should start fine.
> > > > >> >>> - Destroy your SSVM and CPVM, new ones will be recreated from
> > new
> > > > >> >>> systemVM
> > > > >> >>> template.
> > > > >> >>> - Restart all VRs (or restart networks/VPCs with Clean Up
> > option)
> > > -
> > > > >> new
> > > > >> >>> will be created from the new systemVM template.
> > > > >> >>>
> > > > >> >>> I've just tested all this in my lab - let me know if any
> issues.
> > > > >> >>>
> > > > >> >>>
> > > > >> >>>
> > > > >> >>> On Wed, 2 Oct 2019 at 20:37, Fariborz Navidan <
> > > > mdvlinquest@gmail.com>
> > > > >> >>> wrote:
> > > > >> >>>
> > > > >> >>> > I cannot rollback DB to 4.12. How can I download 4.11.3 and
> > and
> > > > >> >>> register in
> > > > >> >>> > DB?
> > > > >> >>> >
> > > > >> >>> > On Wed, Oct 2, 2019 at 9:43 PM Andrija Panic <
> > > > >> andrija.panic@gmail.com>
> > > > >> >>> > wrote:
> > > > >> >>> >
> > > > >> >>> > > There's a GitHub issue about this (perhaps you or some
> > > > colleague?)
> > > > >> >>> > >
> > > > >> >>> > > Did you register 4.11.3 (not 4.13) systemVM template?
> > > > >> Documentation
> > > > >> >>> has
> > > > >> >>> > > just been updated... Sorry for this.
> > > > >> >>> > >
> > > > >> >>> > > i..e. downgrade packages to 4.12, make sure your DB is
> > rolled
> > > > >> back,
> > > > >> >>> start
> > > > >> >>> > > 4.12 normally, register 4.11.3 systemVM template as per
> the
> > > doc
> > > > >> and
> > > > >> >>> then
> > > > >> >>> > > try again to upgrade.
> > > > >> >>> > >
> > > > >> >>> > > let me know how it goes.
> > > > >> >>> > >
> > > > >> >>> > > Andrija
> > > > >> >>> > >
> > > > >> >>> > > On Wed, 2 Oct 2019, 17:27 Fariborz Navidan, <
> > > > >> mdvlinquest@gmail.com>
> > > > >> >>> > wrote:
> > > > >> >>> > >
> > > > >> >>> > > > Any idea?
> > > > >> >>> > > >
> > > > >> >>> > > > On Wed, Oct 2, 2019 at 3:59 PM Fariborz Navidan <
> > > > >> >>> mdvlinquest@gmail.com
> > > > >> >>> > >
> > > > >> >>> > > > wrote:
> > > > >> >>> > > >
> > > > >> >>> > > > > Yes, It was clean 4.12 installation.
> > > > >> >>> > > > >
> > > > >> >>> > > > > On Wed, Oct 2, 2019 at 3:52 PM Andrija Panic <
> > > > >> >>> > andrija.panic@gmail.com>
> > > > >> >>> > > > > wrote:
> > > > >> >>> > > > >
> > > > >> >>> > > > >> "was this a clean 4.12 install, then later upgrade -
> or
> > > was
> > > > >> it
> > > > >> >>> > > > previously
> > > > >> >>> > > > >> 4.11.x then 4.12 etc?"
> > > > >> >>> > > > >>
> > > > >> >>> > > > >> On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <
> > > > >> >>> > mdvlinquest@gmail.com>
> > > > >> >>> > > > >> wrote:
> > > > >> >>> > > > >>
> > > > >> >>> > > > >> > Which question?
> > > > >> >>> > > > >> >
> > > > >> >>> > > > >> > I have sent pastebin link. I lost 4.12 backup
> > > > >> accidentally. I
> > > > >> >>> am
> > > > >> >>> > > > trying
> > > > >> >>> > > > >> to
> > > > >> >>> > > > >> > make 4.13 upgrade successfully. I do run following
> > > > commands
> > > > >> >>> before
> > > > >> >>> > > > >> upgrade
> > > > >> >>> > > > >> > retry:
> > > > >> >>> > > > >> >
> > > > >> >>> > > > >> > [root@fr-kvm1 ~]# cat rollback.sql
> > > > >> >>> > > > >> > delete from guest_os_hypervisor where guest_os_id
> >=
> > > 277;
> > > > >> >>> > > > >> > delete from guest_os where id >= 277;
> > > > >> >>> > > > >> > SET FOREIGN_KEY_CHECKS=0;
> > > > >> >>> > > > >> > drop table direct_download_certificate;
> > > > >> >>> > > > >> > drop table vpc_offering_details;
> > > > >> >>> > > > >> > drop table direct_download_certificate_host_map;
> > > > >> >>> > > > >> > drop table template_ovf_properties;
> > > > >> >>> > > > >> > SET FOREIGN_KEY_CHECKS=1;
> > > > >> >>> > > > >> > alter table data_center drop column sort_key;
> > > > >> >>> > > > >> > alter table vpc_offerings drop column sort_key;
> > > > >> >>> > > > >> > alter table disk_offering add column domain_id
> > int(32)
> > > > not
> > > > >> >>> null;
> > > > >> >>> > > > >> > alter table service_offering_details add constraint
> > > > >> >>> > > > >> > uk_service_offering_id_name unique (id);
> > > > >> >>> > > > >> > alter table network_offering_details DROP COLUMN
> > > > `display`;
> > > > >> >>> > > > >> > ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP
> > > COLUMN
> > > > >> >>> > > > >> `vm_snapshot_id`;
> > > > >> >>> > > > >> > ALTER TABLE
> `cloud_usage`.`usage_snapshot_on_primary`
> > > > DROP
> > > > >> >>> COLUMN
> > > > >> >>> > > > >> > `vm_snapshot_id`;
> > > > >> >>> > > > >> > [root@fr-kvm1 ~]#
> > > > >> >>> > > > >> >
> > > > >> >>> > > > >> > However I don't understand the error I showed you
> on
> > > log
> > > > in
> > > > >> >>> > > pastebin.
> > > > >> >>> > > > >> > Please check and let me know what does it mean.
> > > > >> >>> > > > >> >
> > > > >> >>> > > > >> > On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <
> > > > >> >>> > > andrija.panic@gmail.com
> > > > >> >>> > > > >
> > > > >> >>> > > > >> > wrote:
> > > > >> >>> > > > >> >
> > > > >> >>> > > > >> > > It was not designed to rollback on its own, you
> are
> > > > >> >>> expected to
> > > > >> >>> > > > >> rollback
> > > > >> >>> > > > >> > DB
> > > > >> >>> > > > >> > > yourself (DB restore) and try again if needed.
> > > > >> >>> > > > >> > >
> > > > >> >>> > > > >> > > But you have not answered my question.
> > > > >> >>> > > > >> > >
> > > > >> >>> > > > >> > > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <
> > > > >> >>> > > mdvlinquest@gmail.com>
> > > > >> >>> > > > >> > wrote:
> > > > >> >>> > > > >> > >
> > > > >> >>> > > > >> > > > In my experience, if DB upgrade fails, it does
> > not
> > > > >> fully
> > > > >> >>> rolls
> > > > >> >>> > > > back
> > > > >> >>> > > > >> > > > remaining 4.13's schema upgraded partially and
> > upon
> > > > >> next
> > > > >> >>> > restart
> > > > >> >>> > > > it
> > > > >> >>> > > > >> > does
> > > > >> >>> > > > >> > > > not check existence of 4.13's table or columns
> > > > causing
> > > > >> SQL
> > > > >> >>> > > errors
> > > > >> >>> > > > >> > > > complaining about tables or columns already
> > exists.
> > > > >> >>> > > > >> > > >
> > > > >> >>> > > > >> > > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <
> > > > >> >>> > > > >> andrija.panic@gmail.com>
> > > > >> >>> > > > >> > > > wrote:
> > > > >> >>> > > > >> > > >
> > > > >> >>> > > > >> > > > > was this a clean 4.12 install, then later
> > > upgrade -
> > > > >> or
> > > > >> >>> was
> > > > >> >>> > it
> > > > >> >>> > > > >> > > previously
> > > > >> >>> > > > >> > > > > 4.11.x then 4.12 etc?
> > > > >> >>> > > > >> > > > >
> > > > >> >>> > > > >> > > > > On Wed, 2 Oct 2019 at 12:21, Fariborz
> Navidan <
> > > > >> >>> > > > >> mdvlinquest@gmail.com
> > > > >> >>> > > > >> > >
> > > > >> >>> > > > >> > > > > wrote:
> > > > >> >>> > > > >> > > > >
> > > > >> >>> > > > >> > > > > > https://pastebin.com/2EFm0jaW
> > > > >> >>> > > > >> > > > > >
> > > > >> >>> > > > >> > > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija
> Panic
> > <
> > > > >> >>> > > > >> > > andrija.panic@gmail.com>
> > > > >> >>> > > > >> > > > > > wrote:
> > > > >> >>> > > > >> > > > > >
> > > > >> >>> > > > >> > > > > > > (backup, backup, backup...)
> > > > >> >>> > > > >> > > > > > >
> > > > >> >>> > > > >> > > > > > > Can you send the full logs (passtebin or
> > > > similar
> > > > >> >>> please,
> > > > >> >>> > > not
> > > > >> >>> > > > >> in
> > > > >> >>> > > > >> > > > email)
> > > > >> >>> > > > >> > > > > ?
> > > > >> >>> > > > >> > > > > > >
> > > > >> >>> > > > >> > > > > > > Andrija
> > > > >> >>> > > > >> > > > > > >
> > > > >> >>> > > > >> > > > > >
> > > > >> >>> > > > >> > > > >
> > > > >> >>> > > > >> > > > >
> > > > >> >>> > > > >> > > > > --
> > > > >> >>> > > > >> > > > >
> > > > >> >>> > > > >> > > > > Andrija Panić
> > > > >> >>> > > > >> > > > >
> > > > >> >>> > > > >> > > >
> > > > >> >>> > > > >> > >
> > > > >> >>> > > > >> >
> > > > >> >>> > > > >>
> > > > >> >>> > > > >>
> > > > >> >>> > > > >> --
> > > > >> >>> > > > >>
> > > > >> >>> > > > >> Andrija Panić
> > > > >> >>> > > > >>
> > > > >> >>> > > > >
> > > > >> >>> > > >
> > > > >> >>> > >
> > > > >> >>> >
> > > > >> >>>
> > > > >> >>>
> > > > >> >>> --
> > > > >> >>>
> > > > >> >>> Andrija Panić
> > > > >> >>>
> > > > >> >>
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Andrija Panić
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> > Andrija Panić
> >
>
Re: Upgrade to 4.13 failed
Posted by Fariborz Navidan <md...@gmail.com>.
Thank you very much! Finally, I could start management server normally.
Thanks.
On Thu, Oct 3, 2019 at 9:26 PM Andrija Panic <an...@gmail.com>
wrote:
> It seems easier than I thought...
>
> stop mgmt server
> Rollback DB with your statement shared in some of the previous emails (and
> after that please make DB backup immediately, just for the sake of having
> one...)
> Again, perform that manual template install with:
> - From mgmt, mount the Secondary Storage NFS on /mnt/secondary and
> execute:
>
>
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> -m /mnt/secondary -u
>
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> -h
> <http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2-h>
> kvm -F
> umount /mnt/secondary
>
> Update some fields for that template:
> update vm_template set name="systemvm-kvm-4.11.3" where id="3";
> update vm_template set display_text="systemvm-kvm-4.11.3" where id="3";
> update vm_template set type="USER" where id="3";
> Check that ID=3 for default KVM system template - but should be identical
> on everyones env.
>
> CloudStack upgrade is looking for specific NAME of the template in order to
> proceed with the upgrade (and initially type=USER, and the upgrade will
> change that again to SYSTEM)
> After those SQL changes, start mgmt server - all should be good.
> If mgmt is started fine, you will have to DESTROY all system VMs and VRs,
> i.e. NOT reboot - to force using new template - so Destroy SSVM and CPVM
> (will be recreated automatically) and for the network/VPC - restart them
> with "Clean Up" option.
>
> My previous instructions didn't work, since I was doing more things than
> what I've shared with you - sorry for that - but those steps are needed
> anyway, just were incomplete.
>
> (We could do different fix by manually inserting DB records for a
> completely new template, new rows in vm_template, templat_store_ref and
> template_zone_ref and unpack that BZ2 file/template to specific folder on
> Secondary Storage - but that would require a bit of MySQL and is more
> error-prone. This kind of fix would simulate the step of manually
> registering new systemVM template as regular user template, with the
> specific name - which would be later recognized by it's name as the new
> systemVM templat for 4.11.3)
>
> Let me know how it goes.
> Andrija
>
> On Thu, 3 Oct 2019 at 19:07, Fariborz Navidan <md...@gmail.com>
> wrote:
>
> > MariaDB [cloud]> select max(id) from vm_template;
> > +---------+
> > | max(id) |
> > +---------+
> > | 390 |
> > +---------+
> >
> > Can we have a chat session to investigate together?
> >
> > On Thu, Oct 3, 2019 at 8:27 PM Andrija Panic <an...@gmail.com>
> > wrote:
> >
> > > do for me:
> > >
> > > select max(id) from vm_template:
> > >
> > > i might need more info later...
> > >
> > > On Thu, 3 Oct 2019 at 18:25, Andrija Panic <an...@gmail.com>
> > > wrote:
> > >
> > > > Can you share logs during this last attempt?
> > > >
> > > > I'll try to craft some SQL additions for your vm_template and
> > > > template_store_ref tables while also manually downloading and
> unpacking
> > > the
> > > > file to specific folder on Secondary Storage...
> > > >
> > > > Andrija
> > > >
> > > > On Thu, 3 Oct 2019 at 16:55, Fariborz Navidan <mdvlinquest@gmail.com
> >
> > > > wrote:
> > > >
> > > >> My cloud is down for 2 days. I would really appreciate if you can
> help
> > > me
> > > >> fix the issue.
> > > >>
> > > >> On Thu, Oct 3, 2019 at 4:44 PM Fariborz Navidan <
> > mdvlinquest@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Any idea?
> > > >> >
> > > >> > On Thu, Oct 3, 2019 at 2:09 PM Fariborz Navidan <
> > > mdvlinquest@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> >> I did the steps exactly but still same error. It first says
> cannot
> > > >> >> upgrade database with no further sql errors stated. Thn says
> cannot
> > > >> find
> > > >> >> System VM template for KVM. It look likes system vm template
> 4.11.3
> > > is
> > > >> >> installed only for router!
> > > >> >>
> > > >> >> MariaDB [cloud]> select * from vm_template where name like
> > > 'SystemVM%'
> > > >> >> and removed is null;
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> > > >> >> | id | unique_name | name | uuid
> > > >> >> | public | featured | type | hvm | bits | url
> > > >> >>
> > > |
> > > >> >> format | created | removed | account_id | checksum
> > > >> >> | display_text | enable_password |
> > > >> >> enable_sshkey | guest_os_id | bootable | prepopulate |
> cross_zones
> > |
> > > >> >> extractable | hypervisor_type | source_template_id |
> template_tag |
> > > >> >> sort_key | size | state | update_count | updated |
> > > >> >> dynamically_scalable | parent_template_id | direct_download |
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> > > >> >> | 1 | routing-1 | SystemVM Template (XenServer) |
> > > >> >> c1ea2f68-ca75-11e9-8212-d05099d38681 | 0 | 0 |
> SYSTEM |
> > > >> 0 |
> > > >> >> 64 |
> > > >> >>
> > > >>
> > >
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-xen.vhd.bz2
> > > >> >> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
> > > >> >> 6e8b3ae84ca8145736d1d7d3f7546e65 | SystemVM Template (XenServer)
> |
> > > >> >> 0 | 0 | 184 | 1 | 0 |
> > > >> >> 1 | 0 | XenServer | NULL | NULL
> > > |
> > > >> >> 0 | NULL | Active | 0 | NULL |
> > > >> 0 |
> > > >> >> NULL | 0 |
> > > >> >> | 3 | routing-3 | SystemVM Template (KVM) |
> > > >> >> e4621347-39b8-427e-a55f-82cf51275b69 | 0 | 0 |
> SYSTEM |
> > > >> 0 |
> > > >> >> 64 |
> > > >> >>
> > > >>
> > >
> >
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> > > >> >> | QCOW2 | 2019-08-29 17:57:52 | NULL | 1 |
> > > >> >> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (KVM)
> |
> > > >> >> 0 | 0 | 15 | 1 | 0 |
> > > >> >> 1 | 0 | KVM | NULL | NULL
> > > |
> > > >> >> 0 | 385242624 | Active | 0 | NULL |
> > > >> 0 |
> > > >> >> NULL | 0 |
> > > >> >> | 8 | routing-8 | SystemVM Template (vSphere) |
> > > >> >> c1ea7e39-ca75-11e9-8212-d05099d38681 | 0 | 0 |
> SYSTEM |
> > > >> 0 |
> > > >> >> 64 |
> > > >> >>
> > > >>
> > >
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-vmware.ova
> > > >> >> | OVA | 2019-08-29 17:57:52 | NULL | 1 |
> > > >> >> e981f8cb951688efd93481913198c9cc | SystemVM Template (vSphere)
> |
> > > >> >> 0 | 0 | 99 | 1 | 0 |
> > > >> >> 1 | 0 | VMware | NULL | NULL
> > > |
> > > >> >> 0 | NULL | Active | 0 | NULL |
> > > >> 1 |
> > > >> >> NULL | 0 |
> > > >> >> | 9 | routing-9 | SystemVM Template (HyperV) |
> > > >> >> c1eaa7b0-ca75-11e9-8212-d05099d38681 | 0 | 0 |
> SYSTEM |
> > > >> 0 |
> > > >> >> 64 |
> > > >> >>
> > > >>
> > >
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-hyperv.vhd.zip
> > > >> >> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
> > > >> >> e9032635ffba021371780307162551b9 | SystemVM Template (HyperV)
> |
> > > >> >> 0 | 0 | 15 | 1 | 0 |
> > > >> >> 1 | 0 | Hyperv | NULL | NULL
> > > |
> > > >> >> 0 | NULL | Active | 0 | NULL |
> > > >> 0 |
> > > >> >> NULL | 0 |
> > > >> >> | 10 | routing-10 | SystemVM Template (LXC) |
> > > >> >> 6706cad5-ca77-11e9-8212-d05099d38681 | 0 | 0 |
> SYSTEM |
> > > >> 0 |
> > > >> >> 64 |
> > > >> >>
> > > >>
> > >
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
> > > >> >> | QCOW2 | 2019-08-29 18:09:38 | NULL | 1 |
> > > >> >> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (LXC)
> |
> > > >> >> 0 | 0 | 15 | 1 | 0 |
> > > >> >> 1 | 0 | LXC | NULL | NULL
> > > |
> > > >> >> 0 | NULL | Active | 0 | NULL |
> > > >> 0 |
> > > >> >> NULL | 0 |
> > > >> >> | 12 | routing-12 | SystemVM Template (Ovm3) |
> > > >> >> 694d5de8-ca77-11e9-8212-d05099d38681 | 0 | 0 |
> SYSTEM |
> > > >> 0 |
> > > >> >> 64 |
> > > >> >>
> > > >>
> > >
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-ovm.raw.bz2
> > > >> >> | RAW | 2019-08-29 18:09:42 | NULL | 1 |
> > > >> >> c4a91f8e52e4531a1c2a9a17c530d5fe | SystemVM Template (Ovm3)
> |
> > > >> >> 0 | 0 | 183 | 1 | 0 |
> > > >> >> 1 | 0 | Ovm3 | NULL | NULL
> > > |
> > > >> >> 0 | NULL | Active | NULL | NULL |
> > > >> 0 |
> > > >> >> NULL | 0 |
> > > >> >>
> > > >> >>
> > > >>
> > >
> >
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> > > >> >> 6 rows in set (0.00 sec)
> > > >> >>
> > > >> >> On Thu, Oct 3, 2019 at 1:52 AM Andrija Panic <
> > > andrija.panic@gmail.com>
> > > >> >> wrote:
> > > >> >>
> > > >> >>> - ake sure mgmt is stopped (you can leave it on 4.13 if not
> > > >> >>> already downgraded), stop cloudstack-agents on KVM hosts.
> > > >> >>> - Rollback DB with your SQL statements, that you shared a few
> > emails
> > > >> ago
> > > >> >>> (unfortunately after this 4.12 is NOT possible to work
> > normally...)
> > > >> >>> -Download the new systemVM template - it WILL OVERWRITE your
> old
> > > one
> > > >> -
> > > >> >>> i.e.:
> > > >> >>> From mgmt, mount the Secondary Storage NFS on
> > /mnt/secondary
> > > >> and
> > > >> >>> execute:
> > > >> >>>
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> > > >> >>> -m /mnt/secondary -u
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> >
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> > > >> >>> -h
> > > >> >>> <
> > > >>
> > >
> >
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2-h
> > > >> >
> > > >> >>> kvm -F
> > > >> >>> This will overwrite existing systemVM template on the
> > > >> >>> /mnt/secondary/templates/1/3/xxxxxx and will update the DB with
> > new
> > > >> names
> > > >> >>> etc...
> > > >> >>> - Upgrade packages to 4.13 now (if not already having 4.13
> > > installed)
> > > >> >>> - start mgmt - upgrade path will find the new systemVM template
> > and
> > > >> mgmt
> > > >> >>> server should start fine.
> > > >> >>> - Destroy your SSVM and CPVM, new ones will be recreated from
> new
> > > >> >>> systemVM
> > > >> >>> template.
> > > >> >>> - Restart all VRs (or restart networks/VPCs with Clean Up
> option)
> > -
> > > >> new
> > > >> >>> will be created from the new systemVM template.
> > > >> >>>
> > > >> >>> I've just tested all this in my lab - let me know if any issues.
> > > >> >>>
> > > >> >>>
> > > >> >>>
> > > >> >>> On Wed, 2 Oct 2019 at 20:37, Fariborz Navidan <
> > > mdvlinquest@gmail.com>
> > > >> >>> wrote:
> > > >> >>>
> > > >> >>> > I cannot rollback DB to 4.12. How can I download 4.11.3 and
> and
> > > >> >>> register in
> > > >> >>> > DB?
> > > >> >>> >
> > > >> >>> > On Wed, Oct 2, 2019 at 9:43 PM Andrija Panic <
> > > >> andrija.panic@gmail.com>
> > > >> >>> > wrote:
> > > >> >>> >
> > > >> >>> > > There's a GitHub issue about this (perhaps you or some
> > > colleague?)
> > > >> >>> > >
> > > >> >>> > > Did you register 4.11.3 (not 4.13) systemVM template?
> > > >> Documentation
> > > >> >>> has
> > > >> >>> > > just been updated... Sorry for this.
> > > >> >>> > >
> > > >> >>> > > i..e. downgrade packages to 4.12, make sure your DB is
> rolled
> > > >> back,
> > > >> >>> start
> > > >> >>> > > 4.12 normally, register 4.11.3 systemVM template as per the
> > doc
> > > >> and
> > > >> >>> then
> > > >> >>> > > try again to upgrade.
> > > >> >>> > >
> > > >> >>> > > let me know how it goes.
> > > >> >>> > >
> > > >> >>> > > Andrija
> > > >> >>> > >
> > > >> >>> > > On Wed, 2 Oct 2019, 17:27 Fariborz Navidan, <
> > > >> mdvlinquest@gmail.com>
> > > >> >>> > wrote:
> > > >> >>> > >
> > > >> >>> > > > Any idea?
> > > >> >>> > > >
> > > >> >>> > > > On Wed, Oct 2, 2019 at 3:59 PM Fariborz Navidan <
> > > >> >>> mdvlinquest@gmail.com
> > > >> >>> > >
> > > >> >>> > > > wrote:
> > > >> >>> > > >
> > > >> >>> > > > > Yes, It was clean 4.12 installation.
> > > >> >>> > > > >
> > > >> >>> > > > > On Wed, Oct 2, 2019 at 3:52 PM Andrija Panic <
> > > >> >>> > andrija.panic@gmail.com>
> > > >> >>> > > > > wrote:
> > > >> >>> > > > >
> > > >> >>> > > > >> "was this a clean 4.12 install, then later upgrade - or
> > was
> > > >> it
> > > >> >>> > > > previously
> > > >> >>> > > > >> 4.11.x then 4.12 etc?"
> > > >> >>> > > > >>
> > > >> >>> > > > >> On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <
> > > >> >>> > mdvlinquest@gmail.com>
> > > >> >>> > > > >> wrote:
> > > >> >>> > > > >>
> > > >> >>> > > > >> > Which question?
> > > >> >>> > > > >> >
> > > >> >>> > > > >> > I have sent pastebin link. I lost 4.12 backup
> > > >> accidentally. I
> > > >> >>> am
> > > >> >>> > > > trying
> > > >> >>> > > > >> to
> > > >> >>> > > > >> > make 4.13 upgrade successfully. I do run following
> > > commands
> > > >> >>> before
> > > >> >>> > > > >> upgrade
> > > >> >>> > > > >> > retry:
> > > >> >>> > > > >> >
> > > >> >>> > > > >> > [root@fr-kvm1 ~]# cat rollback.sql
> > > >> >>> > > > >> > delete from guest_os_hypervisor where guest_os_id >=
> > 277;
> > > >> >>> > > > >> > delete from guest_os where id >= 277;
> > > >> >>> > > > >> > SET FOREIGN_KEY_CHECKS=0;
> > > >> >>> > > > >> > drop table direct_download_certificate;
> > > >> >>> > > > >> > drop table vpc_offering_details;
> > > >> >>> > > > >> > drop table direct_download_certificate_host_map;
> > > >> >>> > > > >> > drop table template_ovf_properties;
> > > >> >>> > > > >> > SET FOREIGN_KEY_CHECKS=1;
> > > >> >>> > > > >> > alter table data_center drop column sort_key;
> > > >> >>> > > > >> > alter table vpc_offerings drop column sort_key;
> > > >> >>> > > > >> > alter table disk_offering add column domain_id
> int(32)
> > > not
> > > >> >>> null;
> > > >> >>> > > > >> > alter table service_offering_details add constraint
> > > >> >>> > > > >> > uk_service_offering_id_name unique (id);
> > > >> >>> > > > >> > alter table network_offering_details DROP COLUMN
> > > `display`;
> > > >> >>> > > > >> > ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP
> > COLUMN
> > > >> >>> > > > >> `vm_snapshot_id`;
> > > >> >>> > > > >> > ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary`
> > > DROP
> > > >> >>> COLUMN
> > > >> >>> > > > >> > `vm_snapshot_id`;
> > > >> >>> > > > >> > [root@fr-kvm1 ~]#
> > > >> >>> > > > >> >
> > > >> >>> > > > >> > However I don't understand the error I showed you on
> > log
> > > in
> > > >> >>> > > pastebin.
> > > >> >>> > > > >> > Please check and let me know what does it mean.
> > > >> >>> > > > >> >
> > > >> >>> > > > >> > On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <
> > > >> >>> > > andrija.panic@gmail.com
> > > >> >>> > > > >
> > > >> >>> > > > >> > wrote:
> > > >> >>> > > > >> >
> > > >> >>> > > > >> > > It was not designed to rollback on its own, you are
> > > >> >>> expected to
> > > >> >>> > > > >> rollback
> > > >> >>> > > > >> > DB
> > > >> >>> > > > >> > > yourself (DB restore) and try again if needed.
> > > >> >>> > > > >> > >
> > > >> >>> > > > >> > > But you have not answered my question.
> > > >> >>> > > > >> > >
> > > >> >>> > > > >> > > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <
> > > >> >>> > > mdvlinquest@gmail.com>
> > > >> >>> > > > >> > wrote:
> > > >> >>> > > > >> > >
> > > >> >>> > > > >> > > > In my experience, if DB upgrade fails, it does
> not
> > > >> fully
> > > >> >>> rolls
> > > >> >>> > > > back
> > > >> >>> > > > >> > > > remaining 4.13's schema upgraded partially and
> upon
> > > >> next
> > > >> >>> > restart
> > > >> >>> > > > it
> > > >> >>> > > > >> > does
> > > >> >>> > > > >> > > > not check existence of 4.13's table or columns
> > > causing
> > > >> SQL
> > > >> >>> > > errors
> > > >> >>> > > > >> > > > complaining about tables or columns already
> exists.
> > > >> >>> > > > >> > > >
> > > >> >>> > > > >> > > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <
> > > >> >>> > > > >> andrija.panic@gmail.com>
> > > >> >>> > > > >> > > > wrote:
> > > >> >>> > > > >> > > >
> > > >> >>> > > > >> > > > > was this a clean 4.12 install, then later
> > upgrade -
> > > >> or
> > > >> >>> was
> > > >> >>> > it
> > > >> >>> > > > >> > > previously
> > > >> >>> > > > >> > > > > 4.11.x then 4.12 etc?
> > > >> >>> > > > >> > > > >
> > > >> >>> > > > >> > > > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <
> > > >> >>> > > > >> mdvlinquest@gmail.com
> > > >> >>> > > > >> > >
> > > >> >>> > > > >> > > > > wrote:
> > > >> >>> > > > >> > > > >
> > > >> >>> > > > >> > > > > > https://pastebin.com/2EFm0jaW
> > > >> >>> > > > >> > > > > >
> > > >> >>> > > > >> > > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic
> <
> > > >> >>> > > > >> > > andrija.panic@gmail.com>
> > > >> >>> > > > >> > > > > > wrote:
> > > >> >>> > > > >> > > > > >
> > > >> >>> > > > >> > > > > > > (backup, backup, backup...)
> > > >> >>> > > > >> > > > > > >
> > > >> >>> > > > >> > > > > > > Can you send the full logs (passtebin or
> > > similar
> > > >> >>> please,
> > > >> >>> > > not
> > > >> >>> > > > >> in
> > > >> >>> > > > >> > > > email)
> > > >> >>> > > > >> > > > > ?
> > > >> >>> > > > >> > > > > > >
> > > >> >>> > > > >> > > > > > > Andrija
> > > >> >>> > > > >> > > > > > >
> > > >> >>> > > > >> > > > > >
> > > >> >>> > > > >> > > > >
> > > >> >>> > > > >> > > > >
> > > >> >>> > > > >> > > > > --
> > > >> >>> > > > >> > > > >
> > > >> >>> > > > >> > > > > Andrija Panić
> > > >> >>> > > > >> > > > >
> > > >> >>> > > > >> > > >
> > > >> >>> > > > >> > >
> > > >> >>> > > > >> >
> > > >> >>> > > > >>
> > > >> >>> > > > >>
> > > >> >>> > > > >> --
> > > >> >>> > > > >>
> > > >> >>> > > > >> Andrija Panić
> > > >> >>> > > > >>
> > > >> >>> > > > >
> > > >> >>> > > >
> > > >> >>> > >
> > > >> >>> >
> > > >> >>>
> > > >> >>>
> > > >> >>> --
> > > >> >>>
> > > >> >>> Andrija Panić
> > > >> >>>
> > > >> >>
> > > >>
> > > >
> > > >
> > > > --
> > > >
> > > > Andrija Panić
> > > >
> > >
> >
>
>
> --
>
> Andrija Panić
>
Re: Upgrade to 4.13 failed
Posted by Andrija Panic <an...@gmail.com>.
It seems easier than I thought...
stop mgmt server
Rollback DB with your statement shared in some of the previous emails (and
after that please make DB backup immediately, just for the sake of having
one...)
Again, perform that manual template install with:
- From mgmt, mount the Secondary Storage NFS on /mnt/secondary and
execute:
/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
-m /mnt/secondary -u
http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
-h
kvm -F
umount /mnt/secondary
Update some fields for that template:
update vm_template set name="systemvm-kvm-4.11.3" where id="3";
update vm_template set display_text="systemvm-kvm-4.11.3" where id="3";
update vm_template set type="USER" where id="3";
Check that ID=3 for default KVM system template - but should be identical
on everyones env.
CloudStack upgrade is looking for specific NAME of the template in order to
proceed with the upgrade (and initially type=USER, and the upgrade will
change that again to SYSTEM)
After those SQL changes, start mgmt server - all should be good.
If mgmt is started fine, you will have to DESTROY all system VMs and VRs,
i.e. NOT reboot - to force using new template - so Destroy SSVM and CPVM
(will be recreated automatically) and for the network/VPC - restart them
with "Clean Up" option.
My previous instructions didn't work, since I was doing more things than
what I've shared with you - sorry for that - but those steps are needed
anyway, just were incomplete.
(We could do different fix by manually inserting DB records for a
completely new template, new rows in vm_template, templat_store_ref and
template_zone_ref and unpack that BZ2 file/template to specific folder on
Secondary Storage - but that would require a bit of MySQL and is more
error-prone. This kind of fix would simulate the step of manually
registering new systemVM template as regular user template, with the
specific name - which would be later recognized by it's name as the new
systemVM templat for 4.11.3)
Let me know how it goes.
Andrija
On Thu, 3 Oct 2019 at 19:07, Fariborz Navidan <md...@gmail.com> wrote:
> MariaDB [cloud]> select max(id) from vm_template;
> +---------+
> | max(id) |
> +---------+
> | 390 |
> +---------+
>
> Can we have a chat session to investigate together?
>
> On Thu, Oct 3, 2019 at 8:27 PM Andrija Panic <an...@gmail.com>
> wrote:
>
> > do for me:
> >
> > select max(id) from vm_template:
> >
> > i might need more info later...
> >
> > On Thu, 3 Oct 2019 at 18:25, Andrija Panic <an...@gmail.com>
> > wrote:
> >
> > > Can you share logs during this last attempt?
> > >
> > > I'll try to craft some SQL additions for your vm_template and
> > > template_store_ref tables while also manually downloading and unpacking
> > the
> > > file to specific folder on Secondary Storage...
> > >
> > > Andrija
> > >
> > > On Thu, 3 Oct 2019 at 16:55, Fariborz Navidan <md...@gmail.com>
> > > wrote:
> > >
> > >> My cloud is down for 2 days. I would really appreciate if you can help
> > me
> > >> fix the issue.
> > >>
> > >> On Thu, Oct 3, 2019 at 4:44 PM Fariborz Navidan <
> mdvlinquest@gmail.com>
> > >> wrote:
> > >>
> > >> > Any idea?
> > >> >
> > >> > On Thu, Oct 3, 2019 at 2:09 PM Fariborz Navidan <
> > mdvlinquest@gmail.com>
> > >> > wrote:
> > >> >
> > >> >> I did the steps exactly but still same error. It first says cannot
> > >> >> upgrade database with no further sql errors stated. Thn says cannot
> > >> find
> > >> >> System VM template for KVM. It look likes system vm template 4.11.3
> > is
> > >> >> installed only for router!
> > >> >>
> > >> >> MariaDB [cloud]> select * from vm_template where name like
> > 'SystemVM%'
> > >> >> and removed is null;
> > >> >>
> > >> >>
> > >>
> >
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> > >> >> | id | unique_name | name | uuid
> > >> >> | public | featured | type | hvm | bits | url
> > >> >>
> > |
> > >> >> format | created | removed | account_id | checksum
> > >> >> | display_text | enable_password |
> > >> >> enable_sshkey | guest_os_id | bootable | prepopulate | cross_zones
> |
> > >> >> extractable | hypervisor_type | source_template_id | template_tag |
> > >> >> sort_key | size | state | update_count | updated |
> > >> >> dynamically_scalable | parent_template_id | direct_download |
> > >> >>
> > >> >>
> > >>
> >
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> > >> >> | 1 | routing-1 | SystemVM Template (XenServer) |
> > >> >> c1ea2f68-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
> > >> 0 |
> > >> >> 64 |
> > >> >>
> > >>
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-xen.vhd.bz2
> > >> >> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
> > >> >> 6e8b3ae84ca8145736d1d7d3f7546e65 | SystemVM Template (XenServer) |
> > >> >> 0 | 0 | 184 | 1 | 0 |
> > >> >> 1 | 0 | XenServer | NULL | NULL
> > |
> > >> >> 0 | NULL | Active | 0 | NULL |
> > >> 0 |
> > >> >> NULL | 0 |
> > >> >> | 3 | routing-3 | SystemVM Template (KVM) |
> > >> >> e4621347-39b8-427e-a55f-82cf51275b69 | 0 | 0 | SYSTEM |
> > >> 0 |
> > >> >> 64 |
> > >> >>
> > >>
> >
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> > >> >> | QCOW2 | 2019-08-29 17:57:52 | NULL | 1 |
> > >> >> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (KVM) |
> > >> >> 0 | 0 | 15 | 1 | 0 |
> > >> >> 1 | 0 | KVM | NULL | NULL
> > |
> > >> >> 0 | 385242624 | Active | 0 | NULL |
> > >> 0 |
> > >> >> NULL | 0 |
> > >> >> | 8 | routing-8 | SystemVM Template (vSphere) |
> > >> >> c1ea7e39-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
> > >> 0 |
> > >> >> 64 |
> > >> >>
> > >>
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-vmware.ova
> > >> >> | OVA | 2019-08-29 17:57:52 | NULL | 1 |
> > >> >> e981f8cb951688efd93481913198c9cc | SystemVM Template (vSphere) |
> > >> >> 0 | 0 | 99 | 1 | 0 |
> > >> >> 1 | 0 | VMware | NULL | NULL
> > |
> > >> >> 0 | NULL | Active | 0 | NULL |
> > >> 1 |
> > >> >> NULL | 0 |
> > >> >> | 9 | routing-9 | SystemVM Template (HyperV) |
> > >> >> c1eaa7b0-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
> > >> 0 |
> > >> >> 64 |
> > >> >>
> > >>
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-hyperv.vhd.zip
> > >> >> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
> > >> >> e9032635ffba021371780307162551b9 | SystemVM Template (HyperV) |
> > >> >> 0 | 0 | 15 | 1 | 0 |
> > >> >> 1 | 0 | Hyperv | NULL | NULL
> > |
> > >> >> 0 | NULL | Active | 0 | NULL |
> > >> 0 |
> > >> >> NULL | 0 |
> > >> >> | 10 | routing-10 | SystemVM Template (LXC) |
> > >> >> 6706cad5-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
> > >> 0 |
> > >> >> 64 |
> > >> >>
> > >>
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
> > >> >> | QCOW2 | 2019-08-29 18:09:38 | NULL | 1 |
> > >> >> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (LXC) |
> > >> >> 0 | 0 | 15 | 1 | 0 |
> > >> >> 1 | 0 | LXC | NULL | NULL
> > |
> > >> >> 0 | NULL | Active | 0 | NULL |
> > >> 0 |
> > >> >> NULL | 0 |
> > >> >> | 12 | routing-12 | SystemVM Template (Ovm3) |
> > >> >> 694d5de8-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
> > >> 0 |
> > >> >> 64 |
> > >> >>
> > >>
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-ovm.raw.bz2
> > >> >> | RAW | 2019-08-29 18:09:42 | NULL | 1 |
> > >> >> c4a91f8e52e4531a1c2a9a17c530d5fe | SystemVM Template (Ovm3) |
> > >> >> 0 | 0 | 183 | 1 | 0 |
> > >> >> 1 | 0 | Ovm3 | NULL | NULL
> > |
> > >> >> 0 | NULL | Active | NULL | NULL |
> > >> 0 |
> > >> >> NULL | 0 |
> > >> >>
> > >> >>
> > >>
> >
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> > >> >> 6 rows in set (0.00 sec)
> > >> >>
> > >> >> On Thu, Oct 3, 2019 at 1:52 AM Andrija Panic <
> > andrija.panic@gmail.com>
> > >> >> wrote:
> > >> >>
> > >> >>> - ake sure mgmt is stopped (you can leave it on 4.13 if not
> > >> >>> already downgraded), stop cloudstack-agents on KVM hosts.
> > >> >>> - Rollback DB with your SQL statements, that you shared a few
> emails
> > >> ago
> > >> >>> (unfortunately after this 4.12 is NOT possible to work
> normally...)
> > >> >>> -Download the new systemVM template - it WILL OVERWRITE your old
> > one
> > >> -
> > >> >>> i.e.:
> > >> >>> From mgmt, mount the Secondary Storage NFS on
> /mnt/secondary
> > >> and
> > >> >>> execute:
> > >> >>>
> > >> >>>
> > >> >>>
> > >>
> >
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> > >> >>> -m /mnt/secondary -u
> > >> >>>
> > >> >>>
> > >>
> >
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> > >> >>> -h
> > >> >>> <
> > >>
> >
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2-h
> > >> >
> > >> >>> kvm -F
> > >> >>> This will overwrite existing systemVM template on the
> > >> >>> /mnt/secondary/templates/1/3/xxxxxx and will update the DB with
> new
> > >> names
> > >> >>> etc...
> > >> >>> - Upgrade packages to 4.13 now (if not already having 4.13
> > installed)
> > >> >>> - start mgmt - upgrade path will find the new systemVM template
> and
> > >> mgmt
> > >> >>> server should start fine.
> > >> >>> - Destroy your SSVM and CPVM, new ones will be recreated from new
> > >> >>> systemVM
> > >> >>> template.
> > >> >>> - Restart all VRs (or restart networks/VPCs with Clean Up option)
> -
> > >> new
> > >> >>> will be created from the new systemVM template.
> > >> >>>
> > >> >>> I've just tested all this in my lab - let me know if any issues.
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >>> On Wed, 2 Oct 2019 at 20:37, Fariborz Navidan <
> > mdvlinquest@gmail.com>
> > >> >>> wrote:
> > >> >>>
> > >> >>> > I cannot rollback DB to 4.12. How can I download 4.11.3 and and
> > >> >>> register in
> > >> >>> > DB?
> > >> >>> >
> > >> >>> > On Wed, Oct 2, 2019 at 9:43 PM Andrija Panic <
> > >> andrija.panic@gmail.com>
> > >> >>> > wrote:
> > >> >>> >
> > >> >>> > > There's a GitHub issue about this (perhaps you or some
> > colleague?)
> > >> >>> > >
> > >> >>> > > Did you register 4.11.3 (not 4.13) systemVM template?
> > >> Documentation
> > >> >>> has
> > >> >>> > > just been updated... Sorry for this.
> > >> >>> > >
> > >> >>> > > i..e. downgrade packages to 4.12, make sure your DB is rolled
> > >> back,
> > >> >>> start
> > >> >>> > > 4.12 normally, register 4.11.3 systemVM template as per the
> doc
> > >> and
> > >> >>> then
> > >> >>> > > try again to upgrade.
> > >> >>> > >
> > >> >>> > > let me know how it goes.
> > >> >>> > >
> > >> >>> > > Andrija
> > >> >>> > >
> > >> >>> > > On Wed, 2 Oct 2019, 17:27 Fariborz Navidan, <
> > >> mdvlinquest@gmail.com>
> > >> >>> > wrote:
> > >> >>> > >
> > >> >>> > > > Any idea?
> > >> >>> > > >
> > >> >>> > > > On Wed, Oct 2, 2019 at 3:59 PM Fariborz Navidan <
> > >> >>> mdvlinquest@gmail.com
> > >> >>> > >
> > >> >>> > > > wrote:
> > >> >>> > > >
> > >> >>> > > > > Yes, It was clean 4.12 installation.
> > >> >>> > > > >
> > >> >>> > > > > On Wed, Oct 2, 2019 at 3:52 PM Andrija Panic <
> > >> >>> > andrija.panic@gmail.com>
> > >> >>> > > > > wrote:
> > >> >>> > > > >
> > >> >>> > > > >> "was this a clean 4.12 install, then later upgrade - or
> was
> > >> it
> > >> >>> > > > previously
> > >> >>> > > > >> 4.11.x then 4.12 etc?"
> > >> >>> > > > >>
> > >> >>> > > > >> On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <
> > >> >>> > mdvlinquest@gmail.com>
> > >> >>> > > > >> wrote:
> > >> >>> > > > >>
> > >> >>> > > > >> > Which question?
> > >> >>> > > > >> >
> > >> >>> > > > >> > I have sent pastebin link. I lost 4.12 backup
> > >> accidentally. I
> > >> >>> am
> > >> >>> > > > trying
> > >> >>> > > > >> to
> > >> >>> > > > >> > make 4.13 upgrade successfully. I do run following
> > commands
> > >> >>> before
> > >> >>> > > > >> upgrade
> > >> >>> > > > >> > retry:
> > >> >>> > > > >> >
> > >> >>> > > > >> > [root@fr-kvm1 ~]# cat rollback.sql
> > >> >>> > > > >> > delete from guest_os_hypervisor where guest_os_id >=
> 277;
> > >> >>> > > > >> > delete from guest_os where id >= 277;
> > >> >>> > > > >> > SET FOREIGN_KEY_CHECKS=0;
> > >> >>> > > > >> > drop table direct_download_certificate;
> > >> >>> > > > >> > drop table vpc_offering_details;
> > >> >>> > > > >> > drop table direct_download_certificate_host_map;
> > >> >>> > > > >> > drop table template_ovf_properties;
> > >> >>> > > > >> > SET FOREIGN_KEY_CHECKS=1;
> > >> >>> > > > >> > alter table data_center drop column sort_key;
> > >> >>> > > > >> > alter table vpc_offerings drop column sort_key;
> > >> >>> > > > >> > alter table disk_offering add column domain_id int(32)
> > not
> > >> >>> null;
> > >> >>> > > > >> > alter table service_offering_details add constraint
> > >> >>> > > > >> > uk_service_offering_id_name unique (id);
> > >> >>> > > > >> > alter table network_offering_details DROP COLUMN
> > `display`;
> > >> >>> > > > >> > ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP
> COLUMN
> > >> >>> > > > >> `vm_snapshot_id`;
> > >> >>> > > > >> > ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary`
> > DROP
> > >> >>> COLUMN
> > >> >>> > > > >> > `vm_snapshot_id`;
> > >> >>> > > > >> > [root@fr-kvm1 ~]#
> > >> >>> > > > >> >
> > >> >>> > > > >> > However I don't understand the error I showed you on
> log
> > in
> > >> >>> > > pastebin.
> > >> >>> > > > >> > Please check and let me know what does it mean.
> > >> >>> > > > >> >
> > >> >>> > > > >> > On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <
> > >> >>> > > andrija.panic@gmail.com
> > >> >>> > > > >
> > >> >>> > > > >> > wrote:
> > >> >>> > > > >> >
> > >> >>> > > > >> > > It was not designed to rollback on its own, you are
> > >> >>> expected to
> > >> >>> > > > >> rollback
> > >> >>> > > > >> > DB
> > >> >>> > > > >> > > yourself (DB restore) and try again if needed.
> > >> >>> > > > >> > >
> > >> >>> > > > >> > > But you have not answered my question.
> > >> >>> > > > >> > >
> > >> >>> > > > >> > > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <
> > >> >>> > > mdvlinquest@gmail.com>
> > >> >>> > > > >> > wrote:
> > >> >>> > > > >> > >
> > >> >>> > > > >> > > > In my experience, if DB upgrade fails, it does not
> > >> fully
> > >> >>> rolls
> > >> >>> > > > back
> > >> >>> > > > >> > > > remaining 4.13's schema upgraded partially and upon
> > >> next
> > >> >>> > restart
> > >> >>> > > > it
> > >> >>> > > > >> > does
> > >> >>> > > > >> > > > not check existence of 4.13's table or columns
> > causing
> > >> SQL
> > >> >>> > > errors
> > >> >>> > > > >> > > > complaining about tables or columns already exists.
> > >> >>> > > > >> > > >
> > >> >>> > > > >> > > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <
> > >> >>> > > > >> andrija.panic@gmail.com>
> > >> >>> > > > >> > > > wrote:
> > >> >>> > > > >> > > >
> > >> >>> > > > >> > > > > was this a clean 4.12 install, then later
> upgrade -
> > >> or
> > >> >>> was
> > >> >>> > it
> > >> >>> > > > >> > > previously
> > >> >>> > > > >> > > > > 4.11.x then 4.12 etc?
> > >> >>> > > > >> > > > >
> > >> >>> > > > >> > > > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <
> > >> >>> > > > >> mdvlinquest@gmail.com
> > >> >>> > > > >> > >
> > >> >>> > > > >> > > > > wrote:
> > >> >>> > > > >> > > > >
> > >> >>> > > > >> > > > > > https://pastebin.com/2EFm0jaW
> > >> >>> > > > >> > > > > >
> > >> >>> > > > >> > > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <
> > >> >>> > > > >> > > andrija.panic@gmail.com>
> > >> >>> > > > >> > > > > > wrote:
> > >> >>> > > > >> > > > > >
> > >> >>> > > > >> > > > > > > (backup, backup, backup...)
> > >> >>> > > > >> > > > > > >
> > >> >>> > > > >> > > > > > > Can you send the full logs (passtebin or
> > similar
> > >> >>> please,
> > >> >>> > > not
> > >> >>> > > > >> in
> > >> >>> > > > >> > > > email)
> > >> >>> > > > >> > > > > ?
> > >> >>> > > > >> > > > > > >
> > >> >>> > > > >> > > > > > > Andrija
> > >> >>> > > > >> > > > > > >
> > >> >>> > > > >> > > > > >
> > >> >>> > > > >> > > > >
> > >> >>> > > > >> > > > >
> > >> >>> > > > >> > > > > --
> > >> >>> > > > >> > > > >
> > >> >>> > > > >> > > > > Andrija Panić
> > >> >>> > > > >> > > > >
> > >> >>> > > > >> > > >
> > >> >>> > > > >> > >
> > >> >>> > > > >> >
> > >> >>> > > > >>
> > >> >>> > > > >>
> > >> >>> > > > >> --
> > >> >>> > > > >>
> > >> >>> > > > >> Andrija Panić
> > >> >>> > > > >>
> > >> >>> > > > >
> > >> >>> > > >
> > >> >>> > >
> > >> >>> >
> > >> >>>
> > >> >>>
> > >> >>> --
> > >> >>>
> > >> >>> Andrija Panić
> > >> >>>
> > >> >>
> > >>
> > >
> > >
> > > --
> > >
> > > Andrija Panić
> > >
> >
>
--
Andrija Panić
Re: Upgrade to 4.13 failed
Posted by Fariborz Navidan <md...@gmail.com>.
MariaDB [cloud]> select max(id) from vm_template;
+---------+
| max(id) |
+---------+
| 390 |
+---------+
Can we have a chat session to investigate together?
On Thu, Oct 3, 2019 at 8:27 PM Andrija Panic <an...@gmail.com>
wrote:
> do for me:
>
> select max(id) from vm_template:
>
> i might need more info later...
>
> On Thu, 3 Oct 2019 at 18:25, Andrija Panic <an...@gmail.com>
> wrote:
>
> > Can you share logs during this last attempt?
> >
> > I'll try to craft some SQL additions for your vm_template and
> > template_store_ref tables while also manually downloading and unpacking
> the
> > file to specific folder on Secondary Storage...
> >
> > Andrija
> >
> > On Thu, 3 Oct 2019 at 16:55, Fariborz Navidan <md...@gmail.com>
> > wrote:
> >
> >> My cloud is down for 2 days. I would really appreciate if you can help
> me
> >> fix the issue.
> >>
> >> On Thu, Oct 3, 2019 at 4:44 PM Fariborz Navidan <md...@gmail.com>
> >> wrote:
> >>
> >> > Any idea?
> >> >
> >> > On Thu, Oct 3, 2019 at 2:09 PM Fariborz Navidan <
> mdvlinquest@gmail.com>
> >> > wrote:
> >> >
> >> >> I did the steps exactly but still same error. It first says cannot
> >> >> upgrade database with no further sql errors stated. Thn says cannot
> >> find
> >> >> System VM template for KVM. It look likes system vm template 4.11.3
> is
> >> >> installed only for router!
> >> >>
> >> >> MariaDB [cloud]> select * from vm_template where name like
> 'SystemVM%'
> >> >> and removed is null;
> >> >>
> >> >>
> >>
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> >> >> | id | unique_name | name | uuid
> >> >> | public | featured | type | hvm | bits | url
> >> >>
> |
> >> >> format | created | removed | account_id | checksum
> >> >> | display_text | enable_password |
> >> >> enable_sshkey | guest_os_id | bootable | prepopulate | cross_zones |
> >> >> extractable | hypervisor_type | source_template_id | template_tag |
> >> >> sort_key | size | state | update_count | updated |
> >> >> dynamically_scalable | parent_template_id | direct_download |
> >> >>
> >> >>
> >>
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> >> >> | 1 | routing-1 | SystemVM Template (XenServer) |
> >> >> c1ea2f68-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
> >> 0 |
> >> >> 64 |
> >> >>
> >>
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-xen.vhd.bz2
> >> >> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
> >> >> 6e8b3ae84ca8145736d1d7d3f7546e65 | SystemVM Template (XenServer) |
> >> >> 0 | 0 | 184 | 1 | 0 |
> >> >> 1 | 0 | XenServer | NULL | NULL
> |
> >> >> 0 | NULL | Active | 0 | NULL |
> >> 0 |
> >> >> NULL | 0 |
> >> >> | 3 | routing-3 | SystemVM Template (KVM) |
> >> >> e4621347-39b8-427e-a55f-82cf51275b69 | 0 | 0 | SYSTEM |
> >> 0 |
> >> >> 64 |
> >> >>
> >>
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> >> >> | QCOW2 | 2019-08-29 17:57:52 | NULL | 1 |
> >> >> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (KVM) |
> >> >> 0 | 0 | 15 | 1 | 0 |
> >> >> 1 | 0 | KVM | NULL | NULL
> |
> >> >> 0 | 385242624 | Active | 0 | NULL |
> >> 0 |
> >> >> NULL | 0 |
> >> >> | 8 | routing-8 | SystemVM Template (vSphere) |
> >> >> c1ea7e39-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
> >> 0 |
> >> >> 64 |
> >> >>
> >>
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-vmware.ova
> >> >> | OVA | 2019-08-29 17:57:52 | NULL | 1 |
> >> >> e981f8cb951688efd93481913198c9cc | SystemVM Template (vSphere) |
> >> >> 0 | 0 | 99 | 1 | 0 |
> >> >> 1 | 0 | VMware | NULL | NULL
> |
> >> >> 0 | NULL | Active | 0 | NULL |
> >> 1 |
> >> >> NULL | 0 |
> >> >> | 9 | routing-9 | SystemVM Template (HyperV) |
> >> >> c1eaa7b0-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
> >> 0 |
> >> >> 64 |
> >> >>
> >>
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-hyperv.vhd.zip
> >> >> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
> >> >> e9032635ffba021371780307162551b9 | SystemVM Template (HyperV) |
> >> >> 0 | 0 | 15 | 1 | 0 |
> >> >> 1 | 0 | Hyperv | NULL | NULL
> |
> >> >> 0 | NULL | Active | 0 | NULL |
> >> 0 |
> >> >> NULL | 0 |
> >> >> | 10 | routing-10 | SystemVM Template (LXC) |
> >> >> 6706cad5-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
> >> 0 |
> >> >> 64 |
> >> >>
> >>
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
> >> >> | QCOW2 | 2019-08-29 18:09:38 | NULL | 1 |
> >> >> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (LXC) |
> >> >> 0 | 0 | 15 | 1 | 0 |
> >> >> 1 | 0 | LXC | NULL | NULL
> |
> >> >> 0 | NULL | Active | 0 | NULL |
> >> 0 |
> >> >> NULL | 0 |
> >> >> | 12 | routing-12 | SystemVM Template (Ovm3) |
> >> >> 694d5de8-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
> >> 0 |
> >> >> 64 |
> >> >>
> >>
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-ovm.raw.bz2
> >> >> | RAW | 2019-08-29 18:09:42 | NULL | 1 |
> >> >> c4a91f8e52e4531a1c2a9a17c530d5fe | SystemVM Template (Ovm3) |
> >> >> 0 | 0 | 183 | 1 | 0 |
> >> >> 1 | 0 | Ovm3 | NULL | NULL
> |
> >> >> 0 | NULL | Active | NULL | NULL |
> >> 0 |
> >> >> NULL | 0 |
> >> >>
> >> >>
> >>
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> >> >> 6 rows in set (0.00 sec)
> >> >>
> >> >> On Thu, Oct 3, 2019 at 1:52 AM Andrija Panic <
> andrija.panic@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> - ake sure mgmt is stopped (you can leave it on 4.13 if not
> >> >>> already downgraded), stop cloudstack-agents on KVM hosts.
> >> >>> - Rollback DB with your SQL statements, that you shared a few emails
> >> ago
> >> >>> (unfortunately after this 4.12 is NOT possible to work normally...)
> >> >>> -Download the new systemVM template - it WILL OVERWRITE your old
> one
> >> -
> >> >>> i.e.:
> >> >>> From mgmt, mount the Secondary Storage NFS on /mnt/secondary
> >> and
> >> >>> execute:
> >> >>>
> >> >>>
> >> >>>
> >>
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> >> >>> -m /mnt/secondary -u
> >> >>>
> >> >>>
> >>
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> >> >>> -h
> >> >>> <
> >>
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2-h
> >> >
> >> >>> kvm -F
> >> >>> This will overwrite existing systemVM template on the
> >> >>> /mnt/secondary/templates/1/3/xxxxxx and will update the DB with new
> >> names
> >> >>> etc...
> >> >>> - Upgrade packages to 4.13 now (if not already having 4.13
> installed)
> >> >>> - start mgmt - upgrade path will find the new systemVM template and
> >> mgmt
> >> >>> server should start fine.
> >> >>> - Destroy your SSVM and CPVM, new ones will be recreated from new
> >> >>> systemVM
> >> >>> template.
> >> >>> - Restart all VRs (or restart networks/VPCs with Clean Up option) -
> >> new
> >> >>> will be created from the new systemVM template.
> >> >>>
> >> >>> I've just tested all this in my lab - let me know if any issues.
> >> >>>
> >> >>>
> >> >>>
> >> >>> On Wed, 2 Oct 2019 at 20:37, Fariborz Navidan <
> mdvlinquest@gmail.com>
> >> >>> wrote:
> >> >>>
> >> >>> > I cannot rollback DB to 4.12. How can I download 4.11.3 and and
> >> >>> register in
> >> >>> > DB?
> >> >>> >
> >> >>> > On Wed, Oct 2, 2019 at 9:43 PM Andrija Panic <
> >> andrija.panic@gmail.com>
> >> >>> > wrote:
> >> >>> >
> >> >>> > > There's a GitHub issue about this (perhaps you or some
> colleague?)
> >> >>> > >
> >> >>> > > Did you register 4.11.3 (not 4.13) systemVM template?
> >> Documentation
> >> >>> has
> >> >>> > > just been updated... Sorry for this.
> >> >>> > >
> >> >>> > > i..e. downgrade packages to 4.12, make sure your DB is rolled
> >> back,
> >> >>> start
> >> >>> > > 4.12 normally, register 4.11.3 systemVM template as per the doc
> >> and
> >> >>> then
> >> >>> > > try again to upgrade.
> >> >>> > >
> >> >>> > > let me know how it goes.
> >> >>> > >
> >> >>> > > Andrija
> >> >>> > >
> >> >>> > > On Wed, 2 Oct 2019, 17:27 Fariborz Navidan, <
> >> mdvlinquest@gmail.com>
> >> >>> > wrote:
> >> >>> > >
> >> >>> > > > Any idea?
> >> >>> > > >
> >> >>> > > > On Wed, Oct 2, 2019 at 3:59 PM Fariborz Navidan <
> >> >>> mdvlinquest@gmail.com
> >> >>> > >
> >> >>> > > > wrote:
> >> >>> > > >
> >> >>> > > > > Yes, It was clean 4.12 installation.
> >> >>> > > > >
> >> >>> > > > > On Wed, Oct 2, 2019 at 3:52 PM Andrija Panic <
> >> >>> > andrija.panic@gmail.com>
> >> >>> > > > > wrote:
> >> >>> > > > >
> >> >>> > > > >> "was this a clean 4.12 install, then later upgrade - or was
> >> it
> >> >>> > > > previously
> >> >>> > > > >> 4.11.x then 4.12 etc?"
> >> >>> > > > >>
> >> >>> > > > >> On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <
> >> >>> > mdvlinquest@gmail.com>
> >> >>> > > > >> wrote:
> >> >>> > > > >>
> >> >>> > > > >> > Which question?
> >> >>> > > > >> >
> >> >>> > > > >> > I have sent pastebin link. I lost 4.12 backup
> >> accidentally. I
> >> >>> am
> >> >>> > > > trying
> >> >>> > > > >> to
> >> >>> > > > >> > make 4.13 upgrade successfully. I do run following
> commands
> >> >>> before
> >> >>> > > > >> upgrade
> >> >>> > > > >> > retry:
> >> >>> > > > >> >
> >> >>> > > > >> > [root@fr-kvm1 ~]# cat rollback.sql
> >> >>> > > > >> > delete from guest_os_hypervisor where guest_os_id >= 277;
> >> >>> > > > >> > delete from guest_os where id >= 277;
> >> >>> > > > >> > SET FOREIGN_KEY_CHECKS=0;
> >> >>> > > > >> > drop table direct_download_certificate;
> >> >>> > > > >> > drop table vpc_offering_details;
> >> >>> > > > >> > drop table direct_download_certificate_host_map;
> >> >>> > > > >> > drop table template_ovf_properties;
> >> >>> > > > >> > SET FOREIGN_KEY_CHECKS=1;
> >> >>> > > > >> > alter table data_center drop column sort_key;
> >> >>> > > > >> > alter table vpc_offerings drop column sort_key;
> >> >>> > > > >> > alter table disk_offering add column domain_id int(32)
> not
> >> >>> null;
> >> >>> > > > >> > alter table service_offering_details add constraint
> >> >>> > > > >> > uk_service_offering_id_name unique (id);
> >> >>> > > > >> > alter table network_offering_details DROP COLUMN
> `display`;
> >> >>> > > > >> > ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP COLUMN
> >> >>> > > > >> `vm_snapshot_id`;
> >> >>> > > > >> > ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary`
> DROP
> >> >>> COLUMN
> >> >>> > > > >> > `vm_snapshot_id`;
> >> >>> > > > >> > [root@fr-kvm1 ~]#
> >> >>> > > > >> >
> >> >>> > > > >> > However I don't understand the error I showed you on log
> in
> >> >>> > > pastebin.
> >> >>> > > > >> > Please check and let me know what does it mean.
> >> >>> > > > >> >
> >> >>> > > > >> > On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <
> >> >>> > > andrija.panic@gmail.com
> >> >>> > > > >
> >> >>> > > > >> > wrote:
> >> >>> > > > >> >
> >> >>> > > > >> > > It was not designed to rollback on its own, you are
> >> >>> expected to
> >> >>> > > > >> rollback
> >> >>> > > > >> > DB
> >> >>> > > > >> > > yourself (DB restore) and try again if needed.
> >> >>> > > > >> > >
> >> >>> > > > >> > > But you have not answered my question.
> >> >>> > > > >> > >
> >> >>> > > > >> > > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <
> >> >>> > > mdvlinquest@gmail.com>
> >> >>> > > > >> > wrote:
> >> >>> > > > >> > >
> >> >>> > > > >> > > > In my experience, if DB upgrade fails, it does not
> >> fully
> >> >>> rolls
> >> >>> > > > back
> >> >>> > > > >> > > > remaining 4.13's schema upgraded partially and upon
> >> next
> >> >>> > restart
> >> >>> > > > it
> >> >>> > > > >> > does
> >> >>> > > > >> > > > not check existence of 4.13's table or columns
> causing
> >> SQL
> >> >>> > > errors
> >> >>> > > > >> > > > complaining about tables or columns already exists.
> >> >>> > > > >> > > >
> >> >>> > > > >> > > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <
> >> >>> > > > >> andrija.panic@gmail.com>
> >> >>> > > > >> > > > wrote:
> >> >>> > > > >> > > >
> >> >>> > > > >> > > > > was this a clean 4.12 install, then later upgrade -
> >> or
> >> >>> was
> >> >>> > it
> >> >>> > > > >> > > previously
> >> >>> > > > >> > > > > 4.11.x then 4.12 etc?
> >> >>> > > > >> > > > >
> >> >>> > > > >> > > > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <
> >> >>> > > > >> mdvlinquest@gmail.com
> >> >>> > > > >> > >
> >> >>> > > > >> > > > > wrote:
> >> >>> > > > >> > > > >
> >> >>> > > > >> > > > > > https://pastebin.com/2EFm0jaW
> >> >>> > > > >> > > > > >
> >> >>> > > > >> > > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <
> >> >>> > > > >> > > andrija.panic@gmail.com>
> >> >>> > > > >> > > > > > wrote:
> >> >>> > > > >> > > > > >
> >> >>> > > > >> > > > > > > (backup, backup, backup...)
> >> >>> > > > >> > > > > > >
> >> >>> > > > >> > > > > > > Can you send the full logs (passtebin or
> similar
> >> >>> please,
> >> >>> > > not
> >> >>> > > > >> in
> >> >>> > > > >> > > > email)
> >> >>> > > > >> > > > > ?
> >> >>> > > > >> > > > > > >
> >> >>> > > > >> > > > > > > Andrija
> >> >>> > > > >> > > > > > >
> >> >>> > > > >> > > > > >
> >> >>> > > > >> > > > >
> >> >>> > > > >> > > > >
> >> >>> > > > >> > > > > --
> >> >>> > > > >> > > > >
> >> >>> > > > >> > > > > Andrija Panić
> >> >>> > > > >> > > > >
> >> >>> > > > >> > > >
> >> >>> > > > >> > >
> >> >>> > > > >> >
> >> >>> > > > >>
> >> >>> > > > >>
> >> >>> > > > >> --
> >> >>> > > > >>
> >> >>> > > > >> Andrija Panić
> >> >>> > > > >>
> >> >>> > > > >
> >> >>> > > >
> >> >>> > >
> >> >>> >
> >> >>>
> >> >>>
> >> >>> --
> >> >>>
> >> >>> Andrija Panić
> >> >>>
> >> >>
> >>
> >
> >
> > --
> >
> > Andrija Panić
> >
>
Re: Upgrade to 4.13 failed
Posted by Andrija Panic <an...@gmail.com>.
do for me:
select max(id) from vm_template:
i might need more info later...
On Thu, 3 Oct 2019 at 18:25, Andrija Panic <an...@gmail.com> wrote:
> Can you share logs during this last attempt?
>
> I'll try to craft some SQL additions for your vm_template and
> template_store_ref tables while also manually downloading and unpacking the
> file to specific folder on Secondary Storage...
>
> Andrija
>
> On Thu, 3 Oct 2019 at 16:55, Fariborz Navidan <md...@gmail.com>
> wrote:
>
>> My cloud is down for 2 days. I would really appreciate if you can help me
>> fix the issue.
>>
>> On Thu, Oct 3, 2019 at 4:44 PM Fariborz Navidan <md...@gmail.com>
>> wrote:
>>
>> > Any idea?
>> >
>> > On Thu, Oct 3, 2019 at 2:09 PM Fariborz Navidan <md...@gmail.com>
>> > wrote:
>> >
>> >> I did the steps exactly but still same error. It first says cannot
>> >> upgrade database with no further sql errors stated. Thn says cannot
>> find
>> >> System VM template for KVM. It look likes system vm template 4.11.3 is
>> >> installed only for router!
>> >>
>> >> MariaDB [cloud]> select * from vm_template where name like 'SystemVM%'
>> >> and removed is null;
>> >>
>> >>
>> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
>> >> | id | unique_name | name | uuid
>> >> | public | featured | type | hvm | bits | url
>> >> |
>> >> format | created | removed | account_id | checksum
>> >> | display_text | enable_password |
>> >> enable_sshkey | guest_os_id | bootable | prepopulate | cross_zones |
>> >> extractable | hypervisor_type | source_template_id | template_tag |
>> >> sort_key | size | state | update_count | updated |
>> >> dynamically_scalable | parent_template_id | direct_download |
>> >>
>> >>
>> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
>> >> | 1 | routing-1 | SystemVM Template (XenServer) |
>> >> c1ea2f68-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
>> 0 |
>> >> 64 |
>> >>
>> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-xen.vhd.bz2
>> >> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
>> >> 6e8b3ae84ca8145736d1d7d3f7546e65 | SystemVM Template (XenServer) |
>> >> 0 | 0 | 184 | 1 | 0 |
>> >> 1 | 0 | XenServer | NULL | NULL |
>> >> 0 | NULL | Active | 0 | NULL |
>> 0 |
>> >> NULL | 0 |
>> >> | 3 | routing-3 | SystemVM Template (KVM) |
>> >> e4621347-39b8-427e-a55f-82cf51275b69 | 0 | 0 | SYSTEM |
>> 0 |
>> >> 64 |
>> >>
>> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
>> >> | QCOW2 | 2019-08-29 17:57:52 | NULL | 1 |
>> >> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (KVM) |
>> >> 0 | 0 | 15 | 1 | 0 |
>> >> 1 | 0 | KVM | NULL | NULL |
>> >> 0 | 385242624 | Active | 0 | NULL |
>> 0 |
>> >> NULL | 0 |
>> >> | 8 | routing-8 | SystemVM Template (vSphere) |
>> >> c1ea7e39-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
>> 0 |
>> >> 64 |
>> >>
>> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-vmware.ova
>> >> | OVA | 2019-08-29 17:57:52 | NULL | 1 |
>> >> e981f8cb951688efd93481913198c9cc | SystemVM Template (vSphere) |
>> >> 0 | 0 | 99 | 1 | 0 |
>> >> 1 | 0 | VMware | NULL | NULL |
>> >> 0 | NULL | Active | 0 | NULL |
>> 1 |
>> >> NULL | 0 |
>> >> | 9 | routing-9 | SystemVM Template (HyperV) |
>> >> c1eaa7b0-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
>> 0 |
>> >> 64 |
>> >>
>> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-hyperv.vhd.zip
>> >> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
>> >> e9032635ffba021371780307162551b9 | SystemVM Template (HyperV) |
>> >> 0 | 0 | 15 | 1 | 0 |
>> >> 1 | 0 | Hyperv | NULL | NULL |
>> >> 0 | NULL | Active | 0 | NULL |
>> 0 |
>> >> NULL | 0 |
>> >> | 10 | routing-10 | SystemVM Template (LXC) |
>> >> 6706cad5-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
>> 0 |
>> >> 64 |
>> >>
>> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
>> >> | QCOW2 | 2019-08-29 18:09:38 | NULL | 1 |
>> >> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (LXC) |
>> >> 0 | 0 | 15 | 1 | 0 |
>> >> 1 | 0 | LXC | NULL | NULL |
>> >> 0 | NULL | Active | 0 | NULL |
>> 0 |
>> >> NULL | 0 |
>> >> | 12 | routing-12 | SystemVM Template (Ovm3) |
>> >> 694d5de8-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
>> 0 |
>> >> 64 |
>> >>
>> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-ovm.raw.bz2
>> >> | RAW | 2019-08-29 18:09:42 | NULL | 1 |
>> >> c4a91f8e52e4531a1c2a9a17c530d5fe | SystemVM Template (Ovm3) |
>> >> 0 | 0 | 183 | 1 | 0 |
>> >> 1 | 0 | Ovm3 | NULL | NULL |
>> >> 0 | NULL | Active | NULL | NULL |
>> 0 |
>> >> NULL | 0 |
>> >>
>> >>
>> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
>> >> 6 rows in set (0.00 sec)
>> >>
>> >> On Thu, Oct 3, 2019 at 1:52 AM Andrija Panic <an...@gmail.com>
>> >> wrote:
>> >>
>> >>> - ake sure mgmt is stopped (you can leave it on 4.13 if not
>> >>> already downgraded), stop cloudstack-agents on KVM hosts.
>> >>> - Rollback DB with your SQL statements, that you shared a few emails
>> ago
>> >>> (unfortunately after this 4.12 is NOT possible to work normally...)
>> >>> -Download the new systemVM template - it WILL OVERWRITE your old one
>> -
>> >>> i.e.:
>> >>> From mgmt, mount the Secondary Storage NFS on /mnt/secondary
>> and
>> >>> execute:
>> >>>
>> >>>
>> >>>
>> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
>> >>> -m /mnt/secondary -u
>> >>>
>> >>>
>> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
>> >>> -h
>> >>> <
>> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2-h
>> >
>> >>> kvm -F
>> >>> This will overwrite existing systemVM template on the
>> >>> /mnt/secondary/templates/1/3/xxxxxx and will update the DB with new
>> names
>> >>> etc...
>> >>> - Upgrade packages to 4.13 now (if not already having 4.13 installed)
>> >>> - start mgmt - upgrade path will find the new systemVM template and
>> mgmt
>> >>> server should start fine.
>> >>> - Destroy your SSVM and CPVM, new ones will be recreated from new
>> >>> systemVM
>> >>> template.
>> >>> - Restart all VRs (or restart networks/VPCs with Clean Up option) -
>> new
>> >>> will be created from the new systemVM template.
>> >>>
>> >>> I've just tested all this in my lab - let me know if any issues.
>> >>>
>> >>>
>> >>>
>> >>> On Wed, 2 Oct 2019 at 20:37, Fariborz Navidan <md...@gmail.com>
>> >>> wrote:
>> >>>
>> >>> > I cannot rollback DB to 4.12. How can I download 4.11.3 and and
>> >>> register in
>> >>> > DB?
>> >>> >
>> >>> > On Wed, Oct 2, 2019 at 9:43 PM Andrija Panic <
>> andrija.panic@gmail.com>
>> >>> > wrote:
>> >>> >
>> >>> > > There's a GitHub issue about this (perhaps you or some colleague?)
>> >>> > >
>> >>> > > Did you register 4.11.3 (not 4.13) systemVM template?
>> Documentation
>> >>> has
>> >>> > > just been updated... Sorry for this.
>> >>> > >
>> >>> > > i..e. downgrade packages to 4.12, make sure your DB is rolled
>> back,
>> >>> start
>> >>> > > 4.12 normally, register 4.11.3 systemVM template as per the doc
>> and
>> >>> then
>> >>> > > try again to upgrade.
>> >>> > >
>> >>> > > let me know how it goes.
>> >>> > >
>> >>> > > Andrija
>> >>> > >
>> >>> > > On Wed, 2 Oct 2019, 17:27 Fariborz Navidan, <
>> mdvlinquest@gmail.com>
>> >>> > wrote:
>> >>> > >
>> >>> > > > Any idea?
>> >>> > > >
>> >>> > > > On Wed, Oct 2, 2019 at 3:59 PM Fariborz Navidan <
>> >>> mdvlinquest@gmail.com
>> >>> > >
>> >>> > > > wrote:
>> >>> > > >
>> >>> > > > > Yes, It was clean 4.12 installation.
>> >>> > > > >
>> >>> > > > > On Wed, Oct 2, 2019 at 3:52 PM Andrija Panic <
>> >>> > andrija.panic@gmail.com>
>> >>> > > > > wrote:
>> >>> > > > >
>> >>> > > > >> "was this a clean 4.12 install, then later upgrade - or was
>> it
>> >>> > > > previously
>> >>> > > > >> 4.11.x then 4.12 etc?"
>> >>> > > > >>
>> >>> > > > >> On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <
>> >>> > mdvlinquest@gmail.com>
>> >>> > > > >> wrote:
>> >>> > > > >>
>> >>> > > > >> > Which question?
>> >>> > > > >> >
>> >>> > > > >> > I have sent pastebin link. I lost 4.12 backup
>> accidentally. I
>> >>> am
>> >>> > > > trying
>> >>> > > > >> to
>> >>> > > > >> > make 4.13 upgrade successfully. I do run following commands
>> >>> before
>> >>> > > > >> upgrade
>> >>> > > > >> > retry:
>> >>> > > > >> >
>> >>> > > > >> > [root@fr-kvm1 ~]# cat rollback.sql
>> >>> > > > >> > delete from guest_os_hypervisor where guest_os_id >= 277;
>> >>> > > > >> > delete from guest_os where id >= 277;
>> >>> > > > >> > SET FOREIGN_KEY_CHECKS=0;
>> >>> > > > >> > drop table direct_download_certificate;
>> >>> > > > >> > drop table vpc_offering_details;
>> >>> > > > >> > drop table direct_download_certificate_host_map;
>> >>> > > > >> > drop table template_ovf_properties;
>> >>> > > > >> > SET FOREIGN_KEY_CHECKS=1;
>> >>> > > > >> > alter table data_center drop column sort_key;
>> >>> > > > >> > alter table vpc_offerings drop column sort_key;
>> >>> > > > >> > alter table disk_offering add column domain_id int(32) not
>> >>> null;
>> >>> > > > >> > alter table service_offering_details add constraint
>> >>> > > > >> > uk_service_offering_id_name unique (id);
>> >>> > > > >> > alter table network_offering_details DROP COLUMN `display`;
>> >>> > > > >> > ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP COLUMN
>> >>> > > > >> `vm_snapshot_id`;
>> >>> > > > >> > ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary` DROP
>> >>> COLUMN
>> >>> > > > >> > `vm_snapshot_id`;
>> >>> > > > >> > [root@fr-kvm1 ~]#
>> >>> > > > >> >
>> >>> > > > >> > However I don't understand the error I showed you on log in
>> >>> > > pastebin.
>> >>> > > > >> > Please check and let me know what does it mean.
>> >>> > > > >> >
>> >>> > > > >> > On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <
>> >>> > > andrija.panic@gmail.com
>> >>> > > > >
>> >>> > > > >> > wrote:
>> >>> > > > >> >
>> >>> > > > >> > > It was not designed to rollback on its own, you are
>> >>> expected to
>> >>> > > > >> rollback
>> >>> > > > >> > DB
>> >>> > > > >> > > yourself (DB restore) and try again if needed.
>> >>> > > > >> > >
>> >>> > > > >> > > But you have not answered my question.
>> >>> > > > >> > >
>> >>> > > > >> > > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <
>> >>> > > mdvlinquest@gmail.com>
>> >>> > > > >> > wrote:
>> >>> > > > >> > >
>> >>> > > > >> > > > In my experience, if DB upgrade fails, it does not
>> fully
>> >>> rolls
>> >>> > > > back
>> >>> > > > >> > > > remaining 4.13's schema upgraded partially and upon
>> next
>> >>> > restart
>> >>> > > > it
>> >>> > > > >> > does
>> >>> > > > >> > > > not check existence of 4.13's table or columns causing
>> SQL
>> >>> > > errors
>> >>> > > > >> > > > complaining about tables or columns already exists.
>> >>> > > > >> > > >
>> >>> > > > >> > > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <
>> >>> > > > >> andrija.panic@gmail.com>
>> >>> > > > >> > > > wrote:
>> >>> > > > >> > > >
>> >>> > > > >> > > > > was this a clean 4.12 install, then later upgrade -
>> or
>> >>> was
>> >>> > it
>> >>> > > > >> > > previously
>> >>> > > > >> > > > > 4.11.x then 4.12 etc?
>> >>> > > > >> > > > >
>> >>> > > > >> > > > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <
>> >>> > > > >> mdvlinquest@gmail.com
>> >>> > > > >> > >
>> >>> > > > >> > > > > wrote:
>> >>> > > > >> > > > >
>> >>> > > > >> > > > > > https://pastebin.com/2EFm0jaW
>> >>> > > > >> > > > > >
>> >>> > > > >> > > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <
>> >>> > > > >> > > andrija.panic@gmail.com>
>> >>> > > > >> > > > > > wrote:
>> >>> > > > >> > > > > >
>> >>> > > > >> > > > > > > (backup, backup, backup...)
>> >>> > > > >> > > > > > >
>> >>> > > > >> > > > > > > Can you send the full logs (passtebin or similar
>> >>> please,
>> >>> > > not
>> >>> > > > >> in
>> >>> > > > >> > > > email)
>> >>> > > > >> > > > > ?
>> >>> > > > >> > > > > > >
>> >>> > > > >> > > > > > > Andrija
>> >>> > > > >> > > > > > >
>> >>> > > > >> > > > > >
>> >>> > > > >> > > > >
>> >>> > > > >> > > > >
>> >>> > > > >> > > > > --
>> >>> > > > >> > > > >
>> >>> > > > >> > > > > Andrija Panić
>> >>> > > > >> > > > >
>> >>> > > > >> > > >
>> >>> > > > >> > >
>> >>> > > > >> >
>> >>> > > > >>
>> >>> > > > >>
>> >>> > > > >> --
>> >>> > > > >>
>> >>> > > > >> Andrija Panić
>> >>> > > > >>
>> >>> > > > >
>> >>> > > >
>> >>> > >
>> >>> >
>> >>>
>> >>>
>> >>> --
>> >>>
>> >>> Andrija Panić
>> >>>
>> >>
>>
>
>
> --
>
> Andrija Panić
>
Re: Upgrade to 4.13 failed
Posted by Fariborz Navidan <md...@gmail.com>.
The ID for SystemVM Template (KVM) is 3 and there is two references in
template_store_ref:
MariaDB [cloud]> select * from template_store_ref where template_id=3;
+-----+----------+-------------+---------------------+---------------------+--------------------------------------+--------------+-----------+------------+---------------+----------------+-----------+----------------------------------------------------------------------------------------------------+--------------------------------------------------------------+------------------------------------------------------------------------------------+-------+-----------+---------+--------------+---------+---------------------+----------------------+--------------+
| id | store_id | template_id | created | last_updated
| job_id | download_pct | size |
store_role | physical_size | download_state | error_str | local_path
| install_path | url
| state | destroyed | is_copy | update_count | ref_cnt | updated
| download_url_created | download_url |
+-----+----------+-------------+---------------------+---------------------+--------------------------------------+--------------+-----------+------------+---------------+----------------+-----------+----------------------------------------------------------------------------------------------------+--------------------------------------------------------------+------------------------------------------------------------------------------------+-------+-----------+---------+--------------+---------+---------------------+----------------------+--------------+
| 5 | 1 | 3 | 2019-08-29 16:55:07 | 2019-10-01 09:24:55
| NULL | 100 | 385242624 | Image
| 385242624 | DOWNLOADED | NULL | NULL
|
template/tmpl/1/3/4b1fdeba-f3f5-45c8-b5ba-851e8bfd7f2b.qcow2 |
http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
| Ready | 0 | 0 | 0 | 0 | NULL
| NULL | NULL |
| 233 | 2 | 3 | 2019-09-15 13:01:12 | 2019-10-01 09:24:56
| 755ab14c-c4df-4dd2-ae39-2a9ff7bb0177 | 100 | 385242624 | Image
| 385242624 | DOWNLOADED | |
/mnt/SecStorage/aa312459-a38e-3d25-a531-791cf3c1bd35/template/tmpl/1/3/dnld6265824134994923385tmp_
| template/tmpl/1/3/4b1fdeba-f3f5-45c8-b5ba-851e8bfd7f2b.qcow2 | NULL
|
Ready | 0 | 0 | 2 | 0 | 2019-09-15 13:01:25
| NULL | NULL |
+-----+----------+-------------+---------------------+---------------------+--------------------------------------+--------------+-----------+------------+---------------+----------------+-----------+----------------------------------------------------------------------------------------------------+--------------------------------------------------------------+------------------------------------------------------------------------------------+-------+-----------+---------+--------------+---------+---------------------+----------------------+--------------+
2 rows in set (0.00 sec)
MariaDB [cloud]>
On Thu, Oct 3, 2019 at 7:56 PM Andrija Panic <an...@gmail.com>
wrote:
> Can you share logs during this last attempt?
>
> I'll try to craft some SQL additions for your vm_template and
> template_store_ref tables while also manually downloading and unpacking the
> file to specific folder on Secondary Storage...
>
> Andrija
>
> On Thu, 3 Oct 2019 at 16:55, Fariborz Navidan <md...@gmail.com>
> wrote:
>
> > My cloud is down for 2 days. I would really appreciate if you can help me
> > fix the issue.
> >
> > On Thu, Oct 3, 2019 at 4:44 PM Fariborz Navidan <md...@gmail.com>
> > wrote:
> >
> > > Any idea?
> > >
> > > On Thu, Oct 3, 2019 at 2:09 PM Fariborz Navidan <mdvlinquest@gmail.com
> >
> > > wrote:
> > >
> > >> I did the steps exactly but still same error. It first says cannot
> > >> upgrade database with no further sql errors stated. Thn says cannot
> find
> > >> System VM template for KVM. It look likes system vm template 4.11.3 is
> > >> installed only for router!
> > >>
> > >> MariaDB [cloud]> select * from vm_template where name like 'SystemVM%'
> > >> and removed is null;
> > >>
> > >>
> >
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> > >> | id | unique_name | name | uuid
> > >> | public | featured | type | hvm | bits | url
> > >> |
> > >> format | created | removed | account_id | checksum
> > >> | display_text | enable_password |
> > >> enable_sshkey | guest_os_id | bootable | prepopulate | cross_zones |
> > >> extractable | hypervisor_type | source_template_id | template_tag |
> > >> sort_key | size | state | update_count | updated |
> > >> dynamically_scalable | parent_template_id | direct_download |
> > >>
> > >>
> >
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> > >> | 1 | routing-1 | SystemVM Template (XenServer) |
> > >> c1ea2f68-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
> 0
> > |
> > >> 64 |
> > >>
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-xen.vhd.bz2
> > >> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
> > >> 6e8b3ae84ca8145736d1d7d3f7546e65 | SystemVM Template (XenServer) |
> > >> 0 | 0 | 184 | 1 | 0 |
> > >> 1 | 0 | XenServer | NULL | NULL
> |
> > >> 0 | NULL | Active | 0 | NULL |
> > 0 |
> > >> NULL | 0 |
> > >> | 3 | routing-3 | SystemVM Template (KVM) |
> > >> e4621347-39b8-427e-a55f-82cf51275b69 | 0 | 0 | SYSTEM |
> 0
> > |
> > >> 64 |
> > >>
> >
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> > >> | QCOW2 | 2019-08-29 17:57:52 | NULL | 1 |
> > >> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (KVM) |
> > >> 0 | 0 | 15 | 1 | 0 |
> > >> 1 | 0 | KVM | NULL | NULL
> |
> > >> 0 | 385242624 | Active | 0 | NULL |
> > 0 |
> > >> NULL | 0 |
> > >> | 8 | routing-8 | SystemVM Template (vSphere) |
> > >> c1ea7e39-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
> 0
> > |
> > >> 64 |
> > >>
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-vmware.ova
> > >> | OVA | 2019-08-29 17:57:52 | NULL | 1 |
> > >> e981f8cb951688efd93481913198c9cc | SystemVM Template (vSphere) |
> > >> 0 | 0 | 99 | 1 | 0 |
> > >> 1 | 0 | VMware | NULL | NULL
> |
> > >> 0 | NULL | Active | 0 | NULL |
> > 1 |
> > >> NULL | 0 |
> > >> | 9 | routing-9 | SystemVM Template (HyperV) |
> > >> c1eaa7b0-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
> 0
> > |
> > >> 64 |
> > >>
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-hyperv.vhd.zip
> > >> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
> > >> e9032635ffba021371780307162551b9 | SystemVM Template (HyperV) |
> > >> 0 | 0 | 15 | 1 | 0 |
> > >> 1 | 0 | Hyperv | NULL | NULL
> |
> > >> 0 | NULL | Active | 0 | NULL |
> > 0 |
> > >> NULL | 0 |
> > >> | 10 | routing-10 | SystemVM Template (LXC) |
> > >> 6706cad5-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
> 0
> > |
> > >> 64 |
> > >>
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
> > >> | QCOW2 | 2019-08-29 18:09:38 | NULL | 1 |
> > >> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (LXC) |
> > >> 0 | 0 | 15 | 1 | 0 |
> > >> 1 | 0 | LXC | NULL | NULL
> |
> > >> 0 | NULL | Active | 0 | NULL |
> > 0 |
> > >> NULL | 0 |
> > >> | 12 | routing-12 | SystemVM Template (Ovm3) |
> > >> 694d5de8-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM |
> 0
> > |
> > >> 64 |
> > >>
> >
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-ovm.raw.bz2
> > >> | RAW | 2019-08-29 18:09:42 | NULL | 1 |
> > >> c4a91f8e52e4531a1c2a9a17c530d5fe | SystemVM Template (Ovm3) |
> > >> 0 | 0 | 183 | 1 | 0 |
> > >> 1 | 0 | Ovm3 | NULL | NULL
> |
> > >> 0 | NULL | Active | NULL | NULL |
> > 0 |
> > >> NULL | 0 |
> > >>
> > >>
> >
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> > >> 6 rows in set (0.00 sec)
> > >>
> > >> On Thu, Oct 3, 2019 at 1:52 AM Andrija Panic <andrija.panic@gmail.com
> >
> > >> wrote:
> > >>
> > >>> - ake sure mgmt is stopped (you can leave it on 4.13 if not
> > >>> already downgraded), stop cloudstack-agents on KVM hosts.
> > >>> - Rollback DB with your SQL statements, that you shared a few emails
> > ago
> > >>> (unfortunately after this 4.12 is NOT possible to work normally...)
> > >>> -Download the new systemVM template - it WILL OVERWRITE your old
> one -
> > >>> i.e.:
> > >>> From mgmt, mount the Secondary Storage NFS on /mnt/secondary
> > and
> > >>> execute:
> > >>>
> > >>>
> > >>>
> >
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> > >>> -m /mnt/secondary -u
> > >>>
> > >>>
> >
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> > >>> -h
> > >>> <
> >
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2-h
> > >
> > >>> kvm -F
> > >>> This will overwrite existing systemVM template on the
> > >>> /mnt/secondary/templates/1/3/xxxxxx and will update the DB with new
> > names
> > >>> etc...
> > >>> - Upgrade packages to 4.13 now (if not already having 4.13 installed)
> > >>> - start mgmt - upgrade path will find the new systemVM template and
> > mgmt
> > >>> server should start fine.
> > >>> - Destroy your SSVM and CPVM, new ones will be recreated from new
> > >>> systemVM
> > >>> template.
> > >>> - Restart all VRs (or restart networks/VPCs with Clean Up option) -
> new
> > >>> will be created from the new systemVM template.
> > >>>
> > >>> I've just tested all this in my lab - let me know if any issues.
> > >>>
> > >>>
> > >>>
> > >>> On Wed, 2 Oct 2019 at 20:37, Fariborz Navidan <mdvlinquest@gmail.com
> >
> > >>> wrote:
> > >>>
> > >>> > I cannot rollback DB to 4.12. How can I download 4.11.3 and and
> > >>> register in
> > >>> > DB?
> > >>> >
> > >>> > On Wed, Oct 2, 2019 at 9:43 PM Andrija Panic <
> > andrija.panic@gmail.com>
> > >>> > wrote:
> > >>> >
> > >>> > > There's a GitHub issue about this (perhaps you or some
> colleague?)
> > >>> > >
> > >>> > > Did you register 4.11.3 (not 4.13) systemVM template?
> Documentation
> > >>> has
> > >>> > > just been updated... Sorry for this.
> > >>> > >
> > >>> > > i..e. downgrade packages to 4.12, make sure your DB is rolled
> back,
> > >>> start
> > >>> > > 4.12 normally, register 4.11.3 systemVM template as per the doc
> and
> > >>> then
> > >>> > > try again to upgrade.
> > >>> > >
> > >>> > > let me know how it goes.
> > >>> > >
> > >>> > > Andrija
> > >>> > >
> > >>> > > On Wed, 2 Oct 2019, 17:27 Fariborz Navidan, <
> mdvlinquest@gmail.com
> > >
> > >>> > wrote:
> > >>> > >
> > >>> > > > Any idea?
> > >>> > > >
> > >>> > > > On Wed, Oct 2, 2019 at 3:59 PM Fariborz Navidan <
> > >>> mdvlinquest@gmail.com
> > >>> > >
> > >>> > > > wrote:
> > >>> > > >
> > >>> > > > > Yes, It was clean 4.12 installation.
> > >>> > > > >
> > >>> > > > > On Wed, Oct 2, 2019 at 3:52 PM Andrija Panic <
> > >>> > andrija.panic@gmail.com>
> > >>> > > > > wrote:
> > >>> > > > >
> > >>> > > > >> "was this a clean 4.12 install, then later upgrade - or was
> it
> > >>> > > > previously
> > >>> > > > >> 4.11.x then 4.12 etc?"
> > >>> > > > >>
> > >>> > > > >> On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <
> > >>> > mdvlinquest@gmail.com>
> > >>> > > > >> wrote:
> > >>> > > > >>
> > >>> > > > >> > Which question?
> > >>> > > > >> >
> > >>> > > > >> > I have sent pastebin link. I lost 4.12 backup
> accidentally.
> > I
> > >>> am
> > >>> > > > trying
> > >>> > > > >> to
> > >>> > > > >> > make 4.13 upgrade successfully. I do run following
> commands
> > >>> before
> > >>> > > > >> upgrade
> > >>> > > > >> > retry:
> > >>> > > > >> >
> > >>> > > > >> > [root@fr-kvm1 ~]# cat rollback.sql
> > >>> > > > >> > delete from guest_os_hypervisor where guest_os_id >= 277;
> > >>> > > > >> > delete from guest_os where id >= 277;
> > >>> > > > >> > SET FOREIGN_KEY_CHECKS=0;
> > >>> > > > >> > drop table direct_download_certificate;
> > >>> > > > >> > drop table vpc_offering_details;
> > >>> > > > >> > drop table direct_download_certificate_host_map;
> > >>> > > > >> > drop table template_ovf_properties;
> > >>> > > > >> > SET FOREIGN_KEY_CHECKS=1;
> > >>> > > > >> > alter table data_center drop column sort_key;
> > >>> > > > >> > alter table vpc_offerings drop column sort_key;
> > >>> > > > >> > alter table disk_offering add column domain_id int(32) not
> > >>> null;
> > >>> > > > >> > alter table service_offering_details add constraint
> > >>> > > > >> > uk_service_offering_id_name unique (id);
> > >>> > > > >> > alter table network_offering_details DROP COLUMN
> `display`;
> > >>> > > > >> > ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP COLUMN
> > >>> > > > >> `vm_snapshot_id`;
> > >>> > > > >> > ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary` DROP
> > >>> COLUMN
> > >>> > > > >> > `vm_snapshot_id`;
> > >>> > > > >> > [root@fr-kvm1 ~]#
> > >>> > > > >> >
> > >>> > > > >> > However I don't understand the error I showed you on log
> in
> > >>> > > pastebin.
> > >>> > > > >> > Please check and let me know what does it mean.
> > >>> > > > >> >
> > >>> > > > >> > On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <
> > >>> > > andrija.panic@gmail.com
> > >>> > > > >
> > >>> > > > >> > wrote:
> > >>> > > > >> >
> > >>> > > > >> > > It was not designed to rollback on its own, you are
> > >>> expected to
> > >>> > > > >> rollback
> > >>> > > > >> > DB
> > >>> > > > >> > > yourself (DB restore) and try again if needed.
> > >>> > > > >> > >
> > >>> > > > >> > > But you have not answered my question.
> > >>> > > > >> > >
> > >>> > > > >> > > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <
> > >>> > > mdvlinquest@gmail.com>
> > >>> > > > >> > wrote:
> > >>> > > > >> > >
> > >>> > > > >> > > > In my experience, if DB upgrade fails, it does not
> fully
> > >>> rolls
> > >>> > > > back
> > >>> > > > >> > > > remaining 4.13's schema upgraded partially and upon
> next
> > >>> > restart
> > >>> > > > it
> > >>> > > > >> > does
> > >>> > > > >> > > > not check existence of 4.13's table or columns causing
> > SQL
> > >>> > > errors
> > >>> > > > >> > > > complaining about tables or columns already exists.
> > >>> > > > >> > > >
> > >>> > > > >> > > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <
> > >>> > > > >> andrija.panic@gmail.com>
> > >>> > > > >> > > > wrote:
> > >>> > > > >> > > >
> > >>> > > > >> > > > > was this a clean 4.12 install, then later upgrade -
> or
> > >>> was
> > >>> > it
> > >>> > > > >> > > previously
> > >>> > > > >> > > > > 4.11.x then 4.12 etc?
> > >>> > > > >> > > > >
> > >>> > > > >> > > > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <
> > >>> > > > >> mdvlinquest@gmail.com
> > >>> > > > >> > >
> > >>> > > > >> > > > > wrote:
> > >>> > > > >> > > > >
> > >>> > > > >> > > > > > https://pastebin.com/2EFm0jaW
> > >>> > > > >> > > > > >
> > >>> > > > >> > > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <
> > >>> > > > >> > > andrija.panic@gmail.com>
> > >>> > > > >> > > > > > wrote:
> > >>> > > > >> > > > > >
> > >>> > > > >> > > > > > > (backup, backup, backup...)
> > >>> > > > >> > > > > > >
> > >>> > > > >> > > > > > > Can you send the full logs (passtebin or similar
> > >>> please,
> > >>> > > not
> > >>> > > > >> in
> > >>> > > > >> > > > email)
> > >>> > > > >> > > > > ?
> > >>> > > > >> > > > > > >
> > >>> > > > >> > > > > > > Andrija
> > >>> > > > >> > > > > > >
> > >>> > > > >> > > > > >
> > >>> > > > >> > > > >
> > >>> > > > >> > > > >
> > >>> > > > >> > > > > --
> > >>> > > > >> > > > >
> > >>> > > > >> > > > > Andrija Panić
> > >>> > > > >> > > > >
> > >>> > > > >> > > >
> > >>> > > > >> > >
> > >>> > > > >> >
> > >>> > > > >>
> > >>> > > > >>
> > >>> > > > >> --
> > >>> > > > >>
> > >>> > > > >> Andrija Panić
> > >>> > > > >>
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> > >>>
> > >>> --
> > >>>
> > >>> Andrija Panić
> > >>>
> > >>
> >
>
>
> --
>
> Andrija Panić
>
Re: Upgrade to 4.13 failed
Posted by Andrija Panic <an...@gmail.com>.
Can you share logs during this last attempt?
I'll try to craft some SQL additions for your vm_template and
template_store_ref tables while also manually downloading and unpacking the
file to specific folder on Secondary Storage...
Andrija
On Thu, 3 Oct 2019 at 16:55, Fariborz Navidan <md...@gmail.com> wrote:
> My cloud is down for 2 days. I would really appreciate if you can help me
> fix the issue.
>
> On Thu, Oct 3, 2019 at 4:44 PM Fariborz Navidan <md...@gmail.com>
> wrote:
>
> > Any idea?
> >
> > On Thu, Oct 3, 2019 at 2:09 PM Fariborz Navidan <md...@gmail.com>
> > wrote:
> >
> >> I did the steps exactly but still same error. It first says cannot
> >> upgrade database with no further sql errors stated. Thn says cannot find
> >> System VM template for KVM. It look likes system vm template 4.11.3 is
> >> installed only for router!
> >>
> >> MariaDB [cloud]> select * from vm_template where name like 'SystemVM%'
> >> and removed is null;
> >>
> >>
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> >> | id | unique_name | name | uuid
> >> | public | featured | type | hvm | bits | url
> >> |
> >> format | created | removed | account_id | checksum
> >> | display_text | enable_password |
> >> enable_sshkey | guest_os_id | bootable | prepopulate | cross_zones |
> >> extractable | hypervisor_type | source_template_id | template_tag |
> >> sort_key | size | state | update_count | updated |
> >> dynamically_scalable | parent_template_id | direct_download |
> >>
> >>
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> >> | 1 | routing-1 | SystemVM Template (XenServer) |
> >> c1ea2f68-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0
> |
> >> 64 |
> >>
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-xen.vhd.bz2
> >> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
> >> 6e8b3ae84ca8145736d1d7d3f7546e65 | SystemVM Template (XenServer) |
> >> 0 | 0 | 184 | 1 | 0 |
> >> 1 | 0 | XenServer | NULL | NULL |
> >> 0 | NULL | Active | 0 | NULL |
> 0 |
> >> NULL | 0 |
> >> | 3 | routing-3 | SystemVM Template (KVM) |
> >> e4621347-39b8-427e-a55f-82cf51275b69 | 0 | 0 | SYSTEM | 0
> |
> >> 64 |
> >>
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> >> | QCOW2 | 2019-08-29 17:57:52 | NULL | 1 |
> >> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (KVM) |
> >> 0 | 0 | 15 | 1 | 0 |
> >> 1 | 0 | KVM | NULL | NULL |
> >> 0 | 385242624 | Active | 0 | NULL |
> 0 |
> >> NULL | 0 |
> >> | 8 | routing-8 | SystemVM Template (vSphere) |
> >> c1ea7e39-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0
> |
> >> 64 |
> >>
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-vmware.ova
> >> | OVA | 2019-08-29 17:57:52 | NULL | 1 |
> >> e981f8cb951688efd93481913198c9cc | SystemVM Template (vSphere) |
> >> 0 | 0 | 99 | 1 | 0 |
> >> 1 | 0 | VMware | NULL | NULL |
> >> 0 | NULL | Active | 0 | NULL |
> 1 |
> >> NULL | 0 |
> >> | 9 | routing-9 | SystemVM Template (HyperV) |
> >> c1eaa7b0-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0
> |
> >> 64 |
> >>
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-hyperv.vhd.zip
> >> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
> >> e9032635ffba021371780307162551b9 | SystemVM Template (HyperV) |
> >> 0 | 0 | 15 | 1 | 0 |
> >> 1 | 0 | Hyperv | NULL | NULL |
> >> 0 | NULL | Active | 0 | NULL |
> 0 |
> >> NULL | 0 |
> >> | 10 | routing-10 | SystemVM Template (LXC) |
> >> 6706cad5-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0
> |
> >> 64 |
> >>
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
> >> | QCOW2 | 2019-08-29 18:09:38 | NULL | 1 |
> >> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (LXC) |
> >> 0 | 0 | 15 | 1 | 0 |
> >> 1 | 0 | LXC | NULL | NULL |
> >> 0 | NULL | Active | 0 | NULL |
> 0 |
> >> NULL | 0 |
> >> | 12 | routing-12 | SystemVM Template (Ovm3) |
> >> 694d5de8-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0
> |
> >> 64 |
> >>
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-ovm.raw.bz2
> >> | RAW | 2019-08-29 18:09:42 | NULL | 1 |
> >> c4a91f8e52e4531a1c2a9a17c530d5fe | SystemVM Template (Ovm3) |
> >> 0 | 0 | 183 | 1 | 0 |
> >> 1 | 0 | Ovm3 | NULL | NULL |
> >> 0 | NULL | Active | NULL | NULL |
> 0 |
> >> NULL | 0 |
> >>
> >>
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> >> 6 rows in set (0.00 sec)
> >>
> >> On Thu, Oct 3, 2019 at 1:52 AM Andrija Panic <an...@gmail.com>
> >> wrote:
> >>
> >>> - ake sure mgmt is stopped (you can leave it on 4.13 if not
> >>> already downgraded), stop cloudstack-agents on KVM hosts.
> >>> - Rollback DB with your SQL statements, that you shared a few emails
> ago
> >>> (unfortunately after this 4.12 is NOT possible to work normally...)
> >>> -Download the new systemVM template - it WILL OVERWRITE your old one -
> >>> i.e.:
> >>> From mgmt, mount the Secondary Storage NFS on /mnt/secondary
> and
> >>> execute:
> >>>
> >>>
> >>>
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> >>> -m /mnt/secondary -u
> >>>
> >>>
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> >>> -h
> >>> <
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2-h
> >
> >>> kvm -F
> >>> This will overwrite existing systemVM template on the
> >>> /mnt/secondary/templates/1/3/xxxxxx and will update the DB with new
> names
> >>> etc...
> >>> - Upgrade packages to 4.13 now (if not already having 4.13 installed)
> >>> - start mgmt - upgrade path will find the new systemVM template and
> mgmt
> >>> server should start fine.
> >>> - Destroy your SSVM and CPVM, new ones will be recreated from new
> >>> systemVM
> >>> template.
> >>> - Restart all VRs (or restart networks/VPCs with Clean Up option) - new
> >>> will be created from the new systemVM template.
> >>>
> >>> I've just tested all this in my lab - let me know if any issues.
> >>>
> >>>
> >>>
> >>> On Wed, 2 Oct 2019 at 20:37, Fariborz Navidan <md...@gmail.com>
> >>> wrote:
> >>>
> >>> > I cannot rollback DB to 4.12. How can I download 4.11.3 and and
> >>> register in
> >>> > DB?
> >>> >
> >>> > On Wed, Oct 2, 2019 at 9:43 PM Andrija Panic <
> andrija.panic@gmail.com>
> >>> > wrote:
> >>> >
> >>> > > There's a GitHub issue about this (perhaps you or some colleague?)
> >>> > >
> >>> > > Did you register 4.11.3 (not 4.13) systemVM template? Documentation
> >>> has
> >>> > > just been updated... Sorry for this.
> >>> > >
> >>> > > i..e. downgrade packages to 4.12, make sure your DB is rolled back,
> >>> start
> >>> > > 4.12 normally, register 4.11.3 systemVM template as per the doc and
> >>> then
> >>> > > try again to upgrade.
> >>> > >
> >>> > > let me know how it goes.
> >>> > >
> >>> > > Andrija
> >>> > >
> >>> > > On Wed, 2 Oct 2019, 17:27 Fariborz Navidan, <mdvlinquest@gmail.com
> >
> >>> > wrote:
> >>> > >
> >>> > > > Any idea?
> >>> > > >
> >>> > > > On Wed, Oct 2, 2019 at 3:59 PM Fariborz Navidan <
> >>> mdvlinquest@gmail.com
> >>> > >
> >>> > > > wrote:
> >>> > > >
> >>> > > > > Yes, It was clean 4.12 installation.
> >>> > > > >
> >>> > > > > On Wed, Oct 2, 2019 at 3:52 PM Andrija Panic <
> >>> > andrija.panic@gmail.com>
> >>> > > > > wrote:
> >>> > > > >
> >>> > > > >> "was this a clean 4.12 install, then later upgrade - or was it
> >>> > > > previously
> >>> > > > >> 4.11.x then 4.12 etc?"
> >>> > > > >>
> >>> > > > >> On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <
> >>> > mdvlinquest@gmail.com>
> >>> > > > >> wrote:
> >>> > > > >>
> >>> > > > >> > Which question?
> >>> > > > >> >
> >>> > > > >> > I have sent pastebin link. I lost 4.12 backup accidentally.
> I
> >>> am
> >>> > > > trying
> >>> > > > >> to
> >>> > > > >> > make 4.13 upgrade successfully. I do run following commands
> >>> before
> >>> > > > >> upgrade
> >>> > > > >> > retry:
> >>> > > > >> >
> >>> > > > >> > [root@fr-kvm1 ~]# cat rollback.sql
> >>> > > > >> > delete from guest_os_hypervisor where guest_os_id >= 277;
> >>> > > > >> > delete from guest_os where id >= 277;
> >>> > > > >> > SET FOREIGN_KEY_CHECKS=0;
> >>> > > > >> > drop table direct_download_certificate;
> >>> > > > >> > drop table vpc_offering_details;
> >>> > > > >> > drop table direct_download_certificate_host_map;
> >>> > > > >> > drop table template_ovf_properties;
> >>> > > > >> > SET FOREIGN_KEY_CHECKS=1;
> >>> > > > >> > alter table data_center drop column sort_key;
> >>> > > > >> > alter table vpc_offerings drop column sort_key;
> >>> > > > >> > alter table disk_offering add column domain_id int(32) not
> >>> null;
> >>> > > > >> > alter table service_offering_details add constraint
> >>> > > > >> > uk_service_offering_id_name unique (id);
> >>> > > > >> > alter table network_offering_details DROP COLUMN `display`;
> >>> > > > >> > ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP COLUMN
> >>> > > > >> `vm_snapshot_id`;
> >>> > > > >> > ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary` DROP
> >>> COLUMN
> >>> > > > >> > `vm_snapshot_id`;
> >>> > > > >> > [root@fr-kvm1 ~]#
> >>> > > > >> >
> >>> > > > >> > However I don't understand the error I showed you on log in
> >>> > > pastebin.
> >>> > > > >> > Please check and let me know what does it mean.
> >>> > > > >> >
> >>> > > > >> > On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <
> >>> > > andrija.panic@gmail.com
> >>> > > > >
> >>> > > > >> > wrote:
> >>> > > > >> >
> >>> > > > >> > > It was not designed to rollback on its own, you are
> >>> expected to
> >>> > > > >> rollback
> >>> > > > >> > DB
> >>> > > > >> > > yourself (DB restore) and try again if needed.
> >>> > > > >> > >
> >>> > > > >> > > But you have not answered my question.
> >>> > > > >> > >
> >>> > > > >> > > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <
> >>> > > mdvlinquest@gmail.com>
> >>> > > > >> > wrote:
> >>> > > > >> > >
> >>> > > > >> > > > In my experience, if DB upgrade fails, it does not fully
> >>> rolls
> >>> > > > back
> >>> > > > >> > > > remaining 4.13's schema upgraded partially and upon next
> >>> > restart
> >>> > > > it
> >>> > > > >> > does
> >>> > > > >> > > > not check existence of 4.13's table or columns causing
> SQL
> >>> > > errors
> >>> > > > >> > > > complaining about tables or columns already exists.
> >>> > > > >> > > >
> >>> > > > >> > > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <
> >>> > > > >> andrija.panic@gmail.com>
> >>> > > > >> > > > wrote:
> >>> > > > >> > > >
> >>> > > > >> > > > > was this a clean 4.12 install, then later upgrade - or
> >>> was
> >>> > it
> >>> > > > >> > > previously
> >>> > > > >> > > > > 4.11.x then 4.12 etc?
> >>> > > > >> > > > >
> >>> > > > >> > > > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <
> >>> > > > >> mdvlinquest@gmail.com
> >>> > > > >> > >
> >>> > > > >> > > > > wrote:
> >>> > > > >> > > > >
> >>> > > > >> > > > > > https://pastebin.com/2EFm0jaW
> >>> > > > >> > > > > >
> >>> > > > >> > > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <
> >>> > > > >> > > andrija.panic@gmail.com>
> >>> > > > >> > > > > > wrote:
> >>> > > > >> > > > > >
> >>> > > > >> > > > > > > (backup, backup, backup...)
> >>> > > > >> > > > > > >
> >>> > > > >> > > > > > > Can you send the full logs (passtebin or similar
> >>> please,
> >>> > > not
> >>> > > > >> in
> >>> > > > >> > > > email)
> >>> > > > >> > > > > ?
> >>> > > > >> > > > > > >
> >>> > > > >> > > > > > > Andrija
> >>> > > > >> > > > > > >
> >>> > > > >> > > > > >
> >>> > > > >> > > > >
> >>> > > > >> > > > >
> >>> > > > >> > > > > --
> >>> > > > >> > > > >
> >>> > > > >> > > > > Andrija Panić
> >>> > > > >> > > > >
> >>> > > > >> > > >
> >>> > > > >> > >
> >>> > > > >> >
> >>> > > > >>
> >>> > > > >>
> >>> > > > >> --
> >>> > > > >>
> >>> > > > >> Andrija Panić
> >>> > > > >>
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>>
> >>> --
> >>>
> >>> Andrija Panić
> >>>
> >>
>
--
Andrija Panić
Re: Upgrade to 4.13 failed
Posted by Fariborz Navidan <md...@gmail.com>.
My cloud is down for 2 days. I would really appreciate if you can help me
fix the issue.
On Thu, Oct 3, 2019 at 4:44 PM Fariborz Navidan <md...@gmail.com>
wrote:
> Any idea?
>
> On Thu, Oct 3, 2019 at 2:09 PM Fariborz Navidan <md...@gmail.com>
> wrote:
>
>> I did the steps exactly but still same error. It first says cannot
>> upgrade database with no further sql errors stated. Thn says cannot find
>> System VM template for KVM. It look likes system vm template 4.11.3 is
>> installed only for router!
>>
>> MariaDB [cloud]> select * from vm_template where name like 'SystemVM%'
>> and removed is null;
>>
>> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
>> | id | unique_name | name | uuid
>> | public | featured | type | hvm | bits | url
>> |
>> format | created | removed | account_id | checksum
>> | display_text | enable_password |
>> enable_sshkey | guest_os_id | bootable | prepopulate | cross_zones |
>> extractable | hypervisor_type | source_template_id | template_tag |
>> sort_key | size | state | update_count | updated |
>> dynamically_scalable | parent_template_id | direct_download |
>>
>> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
>> | 1 | routing-1 | SystemVM Template (XenServer) |
>> c1ea2f68-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0 |
>> 64 |
>> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-xen.vhd.bz2
>> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
>> 6e8b3ae84ca8145736d1d7d3f7546e65 | SystemVM Template (XenServer) |
>> 0 | 0 | 184 | 1 | 0 |
>> 1 | 0 | XenServer | NULL | NULL |
>> 0 | NULL | Active | 0 | NULL | 0 |
>> NULL | 0 |
>> | 3 | routing-3 | SystemVM Template (KVM) |
>> e4621347-39b8-427e-a55f-82cf51275b69 | 0 | 0 | SYSTEM | 0 |
>> 64 |
>> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
>> | QCOW2 | 2019-08-29 17:57:52 | NULL | 1 |
>> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (KVM) |
>> 0 | 0 | 15 | 1 | 0 |
>> 1 | 0 | KVM | NULL | NULL |
>> 0 | 385242624 | Active | 0 | NULL | 0 |
>> NULL | 0 |
>> | 8 | routing-8 | SystemVM Template (vSphere) |
>> c1ea7e39-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0 |
>> 64 |
>> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-vmware.ova
>> | OVA | 2019-08-29 17:57:52 | NULL | 1 |
>> e981f8cb951688efd93481913198c9cc | SystemVM Template (vSphere) |
>> 0 | 0 | 99 | 1 | 0 |
>> 1 | 0 | VMware | NULL | NULL |
>> 0 | NULL | Active | 0 | NULL | 1 |
>> NULL | 0 |
>> | 9 | routing-9 | SystemVM Template (HyperV) |
>> c1eaa7b0-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0 |
>> 64 |
>> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-hyperv.vhd.zip
>> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
>> e9032635ffba021371780307162551b9 | SystemVM Template (HyperV) |
>> 0 | 0 | 15 | 1 | 0 |
>> 1 | 0 | Hyperv | NULL | NULL |
>> 0 | NULL | Active | 0 | NULL | 0 |
>> NULL | 0 |
>> | 10 | routing-10 | SystemVM Template (LXC) |
>> 6706cad5-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0 |
>> 64 |
>> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
>> | QCOW2 | 2019-08-29 18:09:38 | NULL | 1 |
>> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (LXC) |
>> 0 | 0 | 15 | 1 | 0 |
>> 1 | 0 | LXC | NULL | NULL |
>> 0 | NULL | Active | 0 | NULL | 0 |
>> NULL | 0 |
>> | 12 | routing-12 | SystemVM Template (Ovm3) |
>> 694d5de8-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0 |
>> 64 |
>> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-ovm.raw.bz2
>> | RAW | 2019-08-29 18:09:42 | NULL | 1 |
>> c4a91f8e52e4531a1c2a9a17c530d5fe | SystemVM Template (Ovm3) |
>> 0 | 0 | 183 | 1 | 0 |
>> 1 | 0 | Ovm3 | NULL | NULL |
>> 0 | NULL | Active | NULL | NULL | 0 |
>> NULL | 0 |
>>
>> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
>> 6 rows in set (0.00 sec)
>>
>> On Thu, Oct 3, 2019 at 1:52 AM Andrija Panic <an...@gmail.com>
>> wrote:
>>
>>> - ake sure mgmt is stopped (you can leave it on 4.13 if not
>>> already downgraded), stop cloudstack-agents on KVM hosts.
>>> - Rollback DB with your SQL statements, that you shared a few emails ago
>>> (unfortunately after this 4.12 is NOT possible to work normally...)
>>> -Download the new systemVM template - it WILL OVERWRITE your old one -
>>> i.e.:
>>> From mgmt, mount the Secondary Storage NFS on /mnt/secondary and
>>> execute:
>>>
>>>
>>> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
>>> -m /mnt/secondary -u
>>>
>>> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
>>> -h
>>> <http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2-h>
>>> kvm -F
>>> This will overwrite existing systemVM template on the
>>> /mnt/secondary/templates/1/3/xxxxxx and will update the DB with new names
>>> etc...
>>> - Upgrade packages to 4.13 now (if not already having 4.13 installed)
>>> - start mgmt - upgrade path will find the new systemVM template and mgmt
>>> server should start fine.
>>> - Destroy your SSVM and CPVM, new ones will be recreated from new
>>> systemVM
>>> template.
>>> - Restart all VRs (or restart networks/VPCs with Clean Up option) - new
>>> will be created from the new systemVM template.
>>>
>>> I've just tested all this in my lab - let me know if any issues.
>>>
>>>
>>>
>>> On Wed, 2 Oct 2019 at 20:37, Fariborz Navidan <md...@gmail.com>
>>> wrote:
>>>
>>> > I cannot rollback DB to 4.12. How can I download 4.11.3 and and
>>> register in
>>> > DB?
>>> >
>>> > On Wed, Oct 2, 2019 at 9:43 PM Andrija Panic <an...@gmail.com>
>>> > wrote:
>>> >
>>> > > There's a GitHub issue about this (perhaps you or some colleague?)
>>> > >
>>> > > Did you register 4.11.3 (not 4.13) systemVM template? Documentation
>>> has
>>> > > just been updated... Sorry for this.
>>> > >
>>> > > i..e. downgrade packages to 4.12, make sure your DB is rolled back,
>>> start
>>> > > 4.12 normally, register 4.11.3 systemVM template as per the doc and
>>> then
>>> > > try again to upgrade.
>>> > >
>>> > > let me know how it goes.
>>> > >
>>> > > Andrija
>>> > >
>>> > > On Wed, 2 Oct 2019, 17:27 Fariborz Navidan, <md...@gmail.com>
>>> > wrote:
>>> > >
>>> > > > Any idea?
>>> > > >
>>> > > > On Wed, Oct 2, 2019 at 3:59 PM Fariborz Navidan <
>>> mdvlinquest@gmail.com
>>> > >
>>> > > > wrote:
>>> > > >
>>> > > > > Yes, It was clean 4.12 installation.
>>> > > > >
>>> > > > > On Wed, Oct 2, 2019 at 3:52 PM Andrija Panic <
>>> > andrija.panic@gmail.com>
>>> > > > > wrote:
>>> > > > >
>>> > > > >> "was this a clean 4.12 install, then later upgrade - or was it
>>> > > > previously
>>> > > > >> 4.11.x then 4.12 etc?"
>>> > > > >>
>>> > > > >> On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <
>>> > mdvlinquest@gmail.com>
>>> > > > >> wrote:
>>> > > > >>
>>> > > > >> > Which question?
>>> > > > >> >
>>> > > > >> > I have sent pastebin link. I lost 4.12 backup accidentally. I
>>> am
>>> > > > trying
>>> > > > >> to
>>> > > > >> > make 4.13 upgrade successfully. I do run following commands
>>> before
>>> > > > >> upgrade
>>> > > > >> > retry:
>>> > > > >> >
>>> > > > >> > [root@fr-kvm1 ~]# cat rollback.sql
>>> > > > >> > delete from guest_os_hypervisor where guest_os_id >= 277;
>>> > > > >> > delete from guest_os where id >= 277;
>>> > > > >> > SET FOREIGN_KEY_CHECKS=0;
>>> > > > >> > drop table direct_download_certificate;
>>> > > > >> > drop table vpc_offering_details;
>>> > > > >> > drop table direct_download_certificate_host_map;
>>> > > > >> > drop table template_ovf_properties;
>>> > > > >> > SET FOREIGN_KEY_CHECKS=1;
>>> > > > >> > alter table data_center drop column sort_key;
>>> > > > >> > alter table vpc_offerings drop column sort_key;
>>> > > > >> > alter table disk_offering add column domain_id int(32) not
>>> null;
>>> > > > >> > alter table service_offering_details add constraint
>>> > > > >> > uk_service_offering_id_name unique (id);
>>> > > > >> > alter table network_offering_details DROP COLUMN `display`;
>>> > > > >> > ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP COLUMN
>>> > > > >> `vm_snapshot_id`;
>>> > > > >> > ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary` DROP
>>> COLUMN
>>> > > > >> > `vm_snapshot_id`;
>>> > > > >> > [root@fr-kvm1 ~]#
>>> > > > >> >
>>> > > > >> > However I don't understand the error I showed you on log in
>>> > > pastebin.
>>> > > > >> > Please check and let me know what does it mean.
>>> > > > >> >
>>> > > > >> > On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <
>>> > > andrija.panic@gmail.com
>>> > > > >
>>> > > > >> > wrote:
>>> > > > >> >
>>> > > > >> > > It was not designed to rollback on its own, you are
>>> expected to
>>> > > > >> rollback
>>> > > > >> > DB
>>> > > > >> > > yourself (DB restore) and try again if needed.
>>> > > > >> > >
>>> > > > >> > > But you have not answered my question.
>>> > > > >> > >
>>> > > > >> > > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <
>>> > > mdvlinquest@gmail.com>
>>> > > > >> > wrote:
>>> > > > >> > >
>>> > > > >> > > > In my experience, if DB upgrade fails, it does not fully
>>> rolls
>>> > > > back
>>> > > > >> > > > remaining 4.13's schema upgraded partially and upon next
>>> > restart
>>> > > > it
>>> > > > >> > does
>>> > > > >> > > > not check existence of 4.13's table or columns causing SQL
>>> > > errors
>>> > > > >> > > > complaining about tables or columns already exists.
>>> > > > >> > > >
>>> > > > >> > > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <
>>> > > > >> andrija.panic@gmail.com>
>>> > > > >> > > > wrote:
>>> > > > >> > > >
>>> > > > >> > > > > was this a clean 4.12 install, then later upgrade - or
>>> was
>>> > it
>>> > > > >> > > previously
>>> > > > >> > > > > 4.11.x then 4.12 etc?
>>> > > > >> > > > >
>>> > > > >> > > > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <
>>> > > > >> mdvlinquest@gmail.com
>>> > > > >> > >
>>> > > > >> > > > > wrote:
>>> > > > >> > > > >
>>> > > > >> > > > > > https://pastebin.com/2EFm0jaW
>>> > > > >> > > > > >
>>> > > > >> > > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <
>>> > > > >> > > andrija.panic@gmail.com>
>>> > > > >> > > > > > wrote:
>>> > > > >> > > > > >
>>> > > > >> > > > > > > (backup, backup, backup...)
>>> > > > >> > > > > > >
>>> > > > >> > > > > > > Can you send the full logs (passtebin or similar
>>> please,
>>> > > not
>>> > > > >> in
>>> > > > >> > > > email)
>>> > > > >> > > > > ?
>>> > > > >> > > > > > >
>>> > > > >> > > > > > > Andrija
>>> > > > >> > > > > > >
>>> > > > >> > > > > >
>>> > > > >> > > > >
>>> > > > >> > > > >
>>> > > > >> > > > > --
>>> > > > >> > > > >
>>> > > > >> > > > > Andrija Panić
>>> > > > >> > > > >
>>> > > > >> > > >
>>> > > > >> > >
>>> > > > >> >
>>> > > > >>
>>> > > > >>
>>> > > > >> --
>>> > > > >>
>>> > > > >> Andrija Panić
>>> > > > >>
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>>
>>> --
>>>
>>> Andrija Panić
>>>
>>
Re: Upgrade to 4.13 failed
Posted by Fariborz Navidan <md...@gmail.com>.
Any idea?
On Thu, Oct 3, 2019 at 2:09 PM Fariborz Navidan <md...@gmail.com>
wrote:
> I did the steps exactly but still same error. It first says cannot upgrade
> database with no further sql errors stated. Thn says cannot find System VM
> template for KVM. It look likes system vm template 4.11.3 is installed only
> for router!
>
> MariaDB [cloud]> select * from vm_template where name like 'SystemVM%' and
> removed is null;
>
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> | id | unique_name | name | uuid
> | public | featured | type | hvm | bits | url
> |
> format | created | removed | account_id | checksum
> | display_text | enable_password |
> enable_sshkey | guest_os_id | bootable | prepopulate | cross_zones |
> extractable | hypervisor_type | source_template_id | template_tag |
> sort_key | size | state | update_count | updated |
> dynamically_scalable | parent_template_id | direct_download |
>
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> | 1 | routing-1 | SystemVM Template (XenServer) |
> c1ea2f68-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0 |
> 64 |
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-xen.vhd.bz2
> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
> 6e8b3ae84ca8145736d1d7d3f7546e65 | SystemVM Template (XenServer) |
> 0 | 0 | 184 | 1 | 0 |
> 1 | 0 | XenServer | NULL | NULL |
> 0 | NULL | Active | 0 | NULL | 0 |
> NULL | 0 |
> | 3 | routing-3 | SystemVM Template (KVM) |
> e4621347-39b8-427e-a55f-82cf51275b69 | 0 | 0 | SYSTEM | 0 |
> 64 |
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> | QCOW2 | 2019-08-29 17:57:52 | NULL | 1 |
> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (KVM) |
> 0 | 0 | 15 | 1 | 0 |
> 1 | 0 | KVM | NULL | NULL |
> 0 | 385242624 | Active | 0 | NULL | 0 |
> NULL | 0 |
> | 8 | routing-8 | SystemVM Template (vSphere) |
> c1ea7e39-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0 |
> 64 |
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-vmware.ova
> | OVA | 2019-08-29 17:57:52 | NULL | 1 |
> e981f8cb951688efd93481913198c9cc | SystemVM Template (vSphere) |
> 0 | 0 | 99 | 1 | 0 |
> 1 | 0 | VMware | NULL | NULL |
> 0 | NULL | Active | 0 | NULL | 1 |
> NULL | 0 |
> | 9 | routing-9 | SystemVM Template (HyperV) |
> c1eaa7b0-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0 |
> 64 |
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-hyperv.vhd.zip
> | VHD | 2019-08-29 17:57:52 | NULL | 1 |
> e9032635ffba021371780307162551b9 | SystemVM Template (HyperV) |
> 0 | 0 | 15 | 1 | 0 |
> 1 | 0 | Hyperv | NULL | NULL |
> 0 | NULL | Active | 0 | NULL | 0 |
> NULL | 0 |
> | 10 | routing-10 | SystemVM Template (LXC) |
> 6706cad5-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0 |
> 64 |
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
> | QCOW2 | 2019-08-29 18:09:38 | NULL | 1 |
> 6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (LXC) |
> 0 | 0 | 15 | 1 | 0 |
> 1 | 0 | LXC | NULL | NULL |
> 0 | NULL | Active | 0 | NULL | 0 |
> NULL | 0 |
> | 12 | routing-12 | SystemVM Template (Ovm3) |
> 694d5de8-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0 |
> 64 |
> https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-ovm.raw.bz2
> | RAW | 2019-08-29 18:09:42 | NULL | 1 |
> c4a91f8e52e4531a1c2a9a17c530d5fe | SystemVM Template (Ovm3) |
> 0 | 0 | 183 | 1 | 0 |
> 1 | 0 | Ovm3 | NULL | NULL |
> 0 | NULL | Active | NULL | NULL | 0 |
> NULL | 0 |
>
> +----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
> 6 rows in set (0.00 sec)
>
> On Thu, Oct 3, 2019 at 1:52 AM Andrija Panic <an...@gmail.com>
> wrote:
>
>> - ake sure mgmt is stopped (you can leave it on 4.13 if not
>> already downgraded), stop cloudstack-agents on KVM hosts.
>> - Rollback DB with your SQL statements, that you shared a few emails ago
>> (unfortunately after this 4.12 is NOT possible to work normally...)
>> -Download the new systemVM template - it WILL OVERWRITE your old one -
>> i.e.:
>> From mgmt, mount the Secondary Storage NFS on /mnt/secondary and
>> execute:
>>
>>
>> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
>> -m /mnt/secondary -u
>>
>> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
>> -h
>> <http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2-h>
>> kvm -F
>> This will overwrite existing systemVM template on the
>> /mnt/secondary/templates/1/3/xxxxxx and will update the DB with new names
>> etc...
>> - Upgrade packages to 4.13 now (if not already having 4.13 installed)
>> - start mgmt - upgrade path will find the new systemVM template and mgmt
>> server should start fine.
>> - Destroy your SSVM and CPVM, new ones will be recreated from new systemVM
>> template.
>> - Restart all VRs (or restart networks/VPCs with Clean Up option) - new
>> will be created from the new systemVM template.
>>
>> I've just tested all this in my lab - let me know if any issues.
>>
>>
>>
>> On Wed, 2 Oct 2019 at 20:37, Fariborz Navidan <md...@gmail.com>
>> wrote:
>>
>> > I cannot rollback DB to 4.12. How can I download 4.11.3 and and
>> register in
>> > DB?
>> >
>> > On Wed, Oct 2, 2019 at 9:43 PM Andrija Panic <an...@gmail.com>
>> > wrote:
>> >
>> > > There's a GitHub issue about this (perhaps you or some colleague?)
>> > >
>> > > Did you register 4.11.3 (not 4.13) systemVM template? Documentation
>> has
>> > > just been updated... Sorry for this.
>> > >
>> > > i..e. downgrade packages to 4.12, make sure your DB is rolled back,
>> start
>> > > 4.12 normally, register 4.11.3 systemVM template as per the doc and
>> then
>> > > try again to upgrade.
>> > >
>> > > let me know how it goes.
>> > >
>> > > Andrija
>> > >
>> > > On Wed, 2 Oct 2019, 17:27 Fariborz Navidan, <md...@gmail.com>
>> > wrote:
>> > >
>> > > > Any idea?
>> > > >
>> > > > On Wed, Oct 2, 2019 at 3:59 PM Fariborz Navidan <
>> mdvlinquest@gmail.com
>> > >
>> > > > wrote:
>> > > >
>> > > > > Yes, It was clean 4.12 installation.
>> > > > >
>> > > > > On Wed, Oct 2, 2019 at 3:52 PM Andrija Panic <
>> > andrija.panic@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > >> "was this a clean 4.12 install, then later upgrade - or was it
>> > > > previously
>> > > > >> 4.11.x then 4.12 etc?"
>> > > > >>
>> > > > >> On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <
>> > mdvlinquest@gmail.com>
>> > > > >> wrote:
>> > > > >>
>> > > > >> > Which question?
>> > > > >> >
>> > > > >> > I have sent pastebin link. I lost 4.12 backup accidentally. I
>> am
>> > > > trying
>> > > > >> to
>> > > > >> > make 4.13 upgrade successfully. I do run following commands
>> before
>> > > > >> upgrade
>> > > > >> > retry:
>> > > > >> >
>> > > > >> > [root@fr-kvm1 ~]# cat rollback.sql
>> > > > >> > delete from guest_os_hypervisor where guest_os_id >= 277;
>> > > > >> > delete from guest_os where id >= 277;
>> > > > >> > SET FOREIGN_KEY_CHECKS=0;
>> > > > >> > drop table direct_download_certificate;
>> > > > >> > drop table vpc_offering_details;
>> > > > >> > drop table direct_download_certificate_host_map;
>> > > > >> > drop table template_ovf_properties;
>> > > > >> > SET FOREIGN_KEY_CHECKS=1;
>> > > > >> > alter table data_center drop column sort_key;
>> > > > >> > alter table vpc_offerings drop column sort_key;
>> > > > >> > alter table disk_offering add column domain_id int(32) not
>> null;
>> > > > >> > alter table service_offering_details add constraint
>> > > > >> > uk_service_offering_id_name unique (id);
>> > > > >> > alter table network_offering_details DROP COLUMN `display`;
>> > > > >> > ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP COLUMN
>> > > > >> `vm_snapshot_id`;
>> > > > >> > ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary` DROP
>> COLUMN
>> > > > >> > `vm_snapshot_id`;
>> > > > >> > [root@fr-kvm1 ~]#
>> > > > >> >
>> > > > >> > However I don't understand the error I showed you on log in
>> > > pastebin.
>> > > > >> > Please check and let me know what does it mean.
>> > > > >> >
>> > > > >> > On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <
>> > > andrija.panic@gmail.com
>> > > > >
>> > > > >> > wrote:
>> > > > >> >
>> > > > >> > > It was not designed to rollback on its own, you are expected
>> to
>> > > > >> rollback
>> > > > >> > DB
>> > > > >> > > yourself (DB restore) and try again if needed.
>> > > > >> > >
>> > > > >> > > But you have not answered my question.
>> > > > >> > >
>> > > > >> > > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <
>> > > mdvlinquest@gmail.com>
>> > > > >> > wrote:
>> > > > >> > >
>> > > > >> > > > In my experience, if DB upgrade fails, it does not fully
>> rolls
>> > > > back
>> > > > >> > > > remaining 4.13's schema upgraded partially and upon next
>> > restart
>> > > > it
>> > > > >> > does
>> > > > >> > > > not check existence of 4.13's table or columns causing SQL
>> > > errors
>> > > > >> > > > complaining about tables or columns already exists.
>> > > > >> > > >
>> > > > >> > > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <
>> > > > >> andrija.panic@gmail.com>
>> > > > >> > > > wrote:
>> > > > >> > > >
>> > > > >> > > > > was this a clean 4.12 install, then later upgrade - or
>> was
>> > it
>> > > > >> > > previously
>> > > > >> > > > > 4.11.x then 4.12 etc?
>> > > > >> > > > >
>> > > > >> > > > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <
>> > > > >> mdvlinquest@gmail.com
>> > > > >> > >
>> > > > >> > > > > wrote:
>> > > > >> > > > >
>> > > > >> > > > > > https://pastebin.com/2EFm0jaW
>> > > > >> > > > > >
>> > > > >> > > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <
>> > > > >> > > andrija.panic@gmail.com>
>> > > > >> > > > > > wrote:
>> > > > >> > > > > >
>> > > > >> > > > > > > (backup, backup, backup...)
>> > > > >> > > > > > >
>> > > > >> > > > > > > Can you send the full logs (passtebin or similar
>> please,
>> > > not
>> > > > >> in
>> > > > >> > > > email)
>> > > > >> > > > > ?
>> > > > >> > > > > > >
>> > > > >> > > > > > > Andrija
>> > > > >> > > > > > >
>> > > > >> > > > > >
>> > > > >> > > > >
>> > > > >> > > > >
>> > > > >> > > > > --
>> > > > >> > > > >
>> > > > >> > > > > Andrija Panić
>> > > > >> > > > >
>> > > > >> > > >
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > > >>
>> > > > >> --
>> > > > >>
>> > > > >> Andrija Panić
>> > > > >>
>> > > > >
>> > > >
>> > >
>> >
>>
>>
>> --
>>
>> Andrija Panić
>>
>
Re: Upgrade to 4.13 failed
Posted by Fariborz Navidan <md...@gmail.com>.
I did the steps exactly but still same error. It first says cannot upgrade
database with no further sql errors stated. Thn says cannot find System VM
template for KVM. It look likes system vm template 4.11.3 is installed only
for router!
MariaDB [cloud]> select * from vm_template where name like 'SystemVM%' and
removed is null;
+----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
| id | unique_name | name | uuid
| public | featured | type | hvm | bits | url
| format
| created | removed | account_id | checksum
| display_text | enable_password | enable_sshkey |
guest_os_id | bootable | prepopulate | cross_zones | extractable |
hypervisor_type | source_template_id | template_tag | sort_key | size
| state | update_count | updated | dynamically_scalable |
parent_template_id | direct_download |
+----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
| 1 | routing-1 | SystemVM Template (XenServer) |
c1ea2f68-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0 |
64 |
https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-xen.vhd.bz2
| VHD | 2019-08-29 17:57:52 | NULL | 1 |
6e8b3ae84ca8145736d1d7d3f7546e65 | SystemVM Template (XenServer) |
0 | 0 | 184 | 1 | 0 |
1 | 0 | XenServer | NULL | NULL |
0 | NULL | Active | 0 | NULL | 0 |
NULL | 0 |
| 3 | routing-3 | SystemVM Template (KVM) |
e4621347-39b8-427e-a55f-82cf51275b69 | 0 | 0 | SYSTEM | 0 |
64 |
http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
| QCOW2 | 2019-08-29 17:57:52 | NULL | 1 |
6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (KVM) |
0 | 0 | 15 | 1 | 0 |
1 | 0 | KVM | NULL | NULL |
0 | 385242624 | Active | 0 | NULL | 0 |
NULL | 0 |
| 8 | routing-8 | SystemVM Template (vSphere) |
c1ea7e39-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0 |
64 |
https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-vmware.ova
| OVA | 2019-08-29 17:57:52 | NULL | 1 |
e981f8cb951688efd93481913198c9cc | SystemVM Template (vSphere) |
0 | 0 | 99 | 1 | 0 |
1 | 0 | VMware | NULL | NULL |
0 | NULL | Active | 0 | NULL | 1 |
NULL | 0 |
| 9 | routing-9 | SystemVM Template (HyperV) |
c1eaa7b0-ca75-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0 |
64 |
https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-hyperv.vhd.zip
| VHD | 2019-08-29 17:57:52 | NULL | 1 |
e9032635ffba021371780307162551b9 | SystemVM Template (HyperV) |
0 | 0 | 15 | 1 | 0 |
1 | 0 | Hyperv | NULL | NULL |
0 | NULL | Active | 0 | NULL | 0 |
NULL | 0 |
| 10 | routing-10 | SystemVM Template (LXC) |
6706cad5-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0 |
64 |
https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-kvm.qcow2.bz2
| QCOW2 | 2019-08-29 18:09:38 | NULL | 1 |
6d12cc764cd7d64112d8c35d70923eb1 | SystemVM Template (LXC) |
0 | 0 | 15 | 1 | 0 |
1 | 0 | LXC | NULL | NULL |
0 | NULL | Active | 0 | NULL | 0 |
NULL | 0 |
| 12 | routing-12 | SystemVM Template (Ovm3) |
694d5de8-ca77-11e9-8212-d05099d38681 | 0 | 0 | SYSTEM | 0 |
64 |
https://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.2-ovm.raw.bz2
| RAW | 2019-08-29 18:09:42 | NULL | 1 |
c4a91f8e52e4531a1c2a9a17c530d5fe | SystemVM Template (Ovm3) |
0 | 0 | 183 | 1 | 0 |
1 | 0 | Ovm3 | NULL | NULL |
0 | NULL | Active | NULL | NULL | 0 |
NULL | 0 |
+----+-------------+-------------------------------+--------------------------------------+--------+----------+--------+-----+------+--------------------------------------------------------------------------------------+--------+---------------------+---------+------------+----------------------------------+-------------------------------+-----------------+---------------+-------------+----------+-------------+-------------+-------------+-----------------+--------------------+--------------+----------+-----------+--------+--------------+---------+----------------------+--------------------+-----------------+
6 rows in set (0.00 sec)
On Thu, Oct 3, 2019 at 1:52 AM Andrija Panic <an...@gmail.com>
wrote:
> - ake sure mgmt is stopped (you can leave it on 4.13 if not
> already downgraded), stop cloudstack-agents on KVM hosts.
> - Rollback DB with your SQL statements, that you shared a few emails ago
> (unfortunately after this 4.12 is NOT possible to work normally...)
> -Download the new systemVM template - it WILL OVERWRITE your old one -
> i.e.:
> From mgmt, mount the Secondary Storage NFS on /mnt/secondary and
> execute:
>
>
> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
> -m /mnt/secondary -u
>
> http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
> -h
> <http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2-h>
> kvm -F
> This will overwrite existing systemVM template on the
> /mnt/secondary/templates/1/3/xxxxxx and will update the DB with new names
> etc...
> - Upgrade packages to 4.13 now (if not already having 4.13 installed)
> - start mgmt - upgrade path will find the new systemVM template and mgmt
> server should start fine.
> - Destroy your SSVM and CPVM, new ones will be recreated from new systemVM
> template.
> - Restart all VRs (or restart networks/VPCs with Clean Up option) - new
> will be created from the new systemVM template.
>
> I've just tested all this in my lab - let me know if any issues.
>
>
>
> On Wed, 2 Oct 2019 at 20:37, Fariborz Navidan <md...@gmail.com>
> wrote:
>
> > I cannot rollback DB to 4.12. How can I download 4.11.3 and and register
> in
> > DB?
> >
> > On Wed, Oct 2, 2019 at 9:43 PM Andrija Panic <an...@gmail.com>
> > wrote:
> >
> > > There's a GitHub issue about this (perhaps you or some colleague?)
> > >
> > > Did you register 4.11.3 (not 4.13) systemVM template? Documentation has
> > > just been updated... Sorry for this.
> > >
> > > i..e. downgrade packages to 4.12, make sure your DB is rolled back,
> start
> > > 4.12 normally, register 4.11.3 systemVM template as per the doc and
> then
> > > try again to upgrade.
> > >
> > > let me know how it goes.
> > >
> > > Andrija
> > >
> > > On Wed, 2 Oct 2019, 17:27 Fariborz Navidan, <md...@gmail.com>
> > wrote:
> > >
> > > > Any idea?
> > > >
> > > > On Wed, Oct 2, 2019 at 3:59 PM Fariborz Navidan <
> mdvlinquest@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Yes, It was clean 4.12 installation.
> > > > >
> > > > > On Wed, Oct 2, 2019 at 3:52 PM Andrija Panic <
> > andrija.panic@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> "was this a clean 4.12 install, then later upgrade - or was it
> > > > previously
> > > > >> 4.11.x then 4.12 etc?"
> > > > >>
> > > > >> On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <
> > mdvlinquest@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Which question?
> > > > >> >
> > > > >> > I have sent pastebin link. I lost 4.12 backup accidentally. I am
> > > > trying
> > > > >> to
> > > > >> > make 4.13 upgrade successfully. I do run following commands
> before
> > > > >> upgrade
> > > > >> > retry:
> > > > >> >
> > > > >> > [root@fr-kvm1 ~]# cat rollback.sql
> > > > >> > delete from guest_os_hypervisor where guest_os_id >= 277;
> > > > >> > delete from guest_os where id >= 277;
> > > > >> > SET FOREIGN_KEY_CHECKS=0;
> > > > >> > drop table direct_download_certificate;
> > > > >> > drop table vpc_offering_details;
> > > > >> > drop table direct_download_certificate_host_map;
> > > > >> > drop table template_ovf_properties;
> > > > >> > SET FOREIGN_KEY_CHECKS=1;
> > > > >> > alter table data_center drop column sort_key;
> > > > >> > alter table vpc_offerings drop column sort_key;
> > > > >> > alter table disk_offering add column domain_id int(32) not null;
> > > > >> > alter table service_offering_details add constraint
> > > > >> > uk_service_offering_id_name unique (id);
> > > > >> > alter table network_offering_details DROP COLUMN `display`;
> > > > >> > ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP COLUMN
> > > > >> `vm_snapshot_id`;
> > > > >> > ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary` DROP
> COLUMN
> > > > >> > `vm_snapshot_id`;
> > > > >> > [root@fr-kvm1 ~]#
> > > > >> >
> > > > >> > However I don't understand the error I showed you on log in
> > > pastebin.
> > > > >> > Please check and let me know what does it mean.
> > > > >> >
> > > > >> > On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <
> > > andrija.panic@gmail.com
> > > > >
> > > > >> > wrote:
> > > > >> >
> > > > >> > > It was not designed to rollback on its own, you are expected
> to
> > > > >> rollback
> > > > >> > DB
> > > > >> > > yourself (DB restore) and try again if needed.
> > > > >> > >
> > > > >> > > But you have not answered my question.
> > > > >> > >
> > > > >> > > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <
> > > mdvlinquest@gmail.com>
> > > > >> > wrote:
> > > > >> > >
> > > > >> > > > In my experience, if DB upgrade fails, it does not fully
> rolls
> > > > back
> > > > >> > > > remaining 4.13's schema upgraded partially and upon next
> > restart
> > > > it
> > > > >> > does
> > > > >> > > > not check existence of 4.13's table or columns causing SQL
> > > errors
> > > > >> > > > complaining about tables or columns already exists.
> > > > >> > > >
> > > > >> > > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <
> > > > >> andrija.panic@gmail.com>
> > > > >> > > > wrote:
> > > > >> > > >
> > > > >> > > > > was this a clean 4.12 install, then later upgrade - or was
> > it
> > > > >> > > previously
> > > > >> > > > > 4.11.x then 4.12 etc?
> > > > >> > > > >
> > > > >> > > > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <
> > > > >> mdvlinquest@gmail.com
> > > > >> > >
> > > > >> > > > > wrote:
> > > > >> > > > >
> > > > >> > > > > > https://pastebin.com/2EFm0jaW
> > > > >> > > > > >
> > > > >> > > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <
> > > > >> > > andrija.panic@gmail.com>
> > > > >> > > > > > wrote:
> > > > >> > > > > >
> > > > >> > > > > > > (backup, backup, backup...)
> > > > >> > > > > > >
> > > > >> > > > > > > Can you send the full logs (passtebin or similar
> please,
> > > not
> > > > >> in
> > > > >> > > > email)
> > > > >> > > > > ?
> > > > >> > > > > > >
> > > > >> > > > > > > Andrija
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > --
> > > > >> > > > >
> > > > >> > > > > Andrija Panić
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >>
> > > > >> --
> > > > >>
> > > > >> Andrija Panić
> > > > >>
> > > > >
> > > >
> > >
> >
>
>
> --
>
> Andrija Panić
>
Re: Upgrade to 4.13 failed
Posted by Andrija Panic <an...@gmail.com>.
- ake sure mgmt is stopped (you can leave it on 4.13 if not
already downgraded), stop cloudstack-agents on KVM hosts.
- Rollback DB with your SQL statements, that you shared a few emails ago
(unfortunately after this 4.12 is NOT possible to work normally...)
-Download the new systemVM template - it WILL OVERWRITE your old one -
i.e.:
From mgmt, mount the Secondary Storage NFS on /mnt/secondary and
execute:
/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
-m /mnt/secondary -u
http://download.cloudstack.org/systemvm/4.11/systemvmtemplate-4.11.3-kvm.qcow2.bz2
-h kvm -F
This will overwrite existing systemVM template on the
/mnt/secondary/templates/1/3/xxxxxx and will update the DB with new names
etc...
- Upgrade packages to 4.13 now (if not already having 4.13 installed)
- start mgmt - upgrade path will find the new systemVM template and mgmt
server should start fine.
- Destroy your SSVM and CPVM, new ones will be recreated from new systemVM
template.
- Restart all VRs (or restart networks/VPCs with Clean Up option) - new
will be created from the new systemVM template.
I've just tested all this in my lab - let me know if any issues.
On Wed, 2 Oct 2019 at 20:37, Fariborz Navidan <md...@gmail.com> wrote:
> I cannot rollback DB to 4.12. How can I download 4.11.3 and and register in
> DB?
>
> On Wed, Oct 2, 2019 at 9:43 PM Andrija Panic <an...@gmail.com>
> wrote:
>
> > There's a GitHub issue about this (perhaps you or some colleague?)
> >
> > Did you register 4.11.3 (not 4.13) systemVM template? Documentation has
> > just been updated... Sorry for this.
> >
> > i..e. downgrade packages to 4.12, make sure your DB is rolled back, start
> > 4.12 normally, register 4.11.3 systemVM template as per the doc and then
> > try again to upgrade.
> >
> > let me know how it goes.
> >
> > Andrija
> >
> > On Wed, 2 Oct 2019, 17:27 Fariborz Navidan, <md...@gmail.com>
> wrote:
> >
> > > Any idea?
> > >
> > > On Wed, Oct 2, 2019 at 3:59 PM Fariborz Navidan <mdvlinquest@gmail.com
> >
> > > wrote:
> > >
> > > > Yes, It was clean 4.12 installation.
> > > >
> > > > On Wed, Oct 2, 2019 at 3:52 PM Andrija Panic <
> andrija.panic@gmail.com>
> > > > wrote:
> > > >
> > > >> "was this a clean 4.12 install, then later upgrade - or was it
> > > previously
> > > >> 4.11.x then 4.12 etc?"
> > > >>
> > > >> On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <
> mdvlinquest@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Which question?
> > > >> >
> > > >> > I have sent pastebin link. I lost 4.12 backup accidentally. I am
> > > trying
> > > >> to
> > > >> > make 4.13 upgrade successfully. I do run following commands before
> > > >> upgrade
> > > >> > retry:
> > > >> >
> > > >> > [root@fr-kvm1 ~]# cat rollback.sql
> > > >> > delete from guest_os_hypervisor where guest_os_id >= 277;
> > > >> > delete from guest_os where id >= 277;
> > > >> > SET FOREIGN_KEY_CHECKS=0;
> > > >> > drop table direct_download_certificate;
> > > >> > drop table vpc_offering_details;
> > > >> > drop table direct_download_certificate_host_map;
> > > >> > drop table template_ovf_properties;
> > > >> > SET FOREIGN_KEY_CHECKS=1;
> > > >> > alter table data_center drop column sort_key;
> > > >> > alter table vpc_offerings drop column sort_key;
> > > >> > alter table disk_offering add column domain_id int(32) not null;
> > > >> > alter table service_offering_details add constraint
> > > >> > uk_service_offering_id_name unique (id);
> > > >> > alter table network_offering_details DROP COLUMN `display`;
> > > >> > ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP COLUMN
> > > >> `vm_snapshot_id`;
> > > >> > ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary` DROP COLUMN
> > > >> > `vm_snapshot_id`;
> > > >> > [root@fr-kvm1 ~]#
> > > >> >
> > > >> > However I don't understand the error I showed you on log in
> > pastebin.
> > > >> > Please check and let me know what does it mean.
> > > >> >
> > > >> > On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <
> > andrija.panic@gmail.com
> > > >
> > > >> > wrote:
> > > >> >
> > > >> > > It was not designed to rollback on its own, you are expected to
> > > >> rollback
> > > >> > DB
> > > >> > > yourself (DB restore) and try again if needed.
> > > >> > >
> > > >> > > But you have not answered my question.
> > > >> > >
> > > >> > > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <
> > mdvlinquest@gmail.com>
> > > >> > wrote:
> > > >> > >
> > > >> > > > In my experience, if DB upgrade fails, it does not fully rolls
> > > back
> > > >> > > > remaining 4.13's schema upgraded partially and upon next
> restart
> > > it
> > > >> > does
> > > >> > > > not check existence of 4.13's table or columns causing SQL
> > errors
> > > >> > > > complaining about tables or columns already exists.
> > > >> > > >
> > > >> > > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <
> > > >> andrija.panic@gmail.com>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > was this a clean 4.12 install, then later upgrade - or was
> it
> > > >> > > previously
> > > >> > > > > 4.11.x then 4.12 etc?
> > > >> > > > >
> > > >> > > > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <
> > > >> mdvlinquest@gmail.com
> > > >> > >
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > https://pastebin.com/2EFm0jaW
> > > >> > > > > >
> > > >> > > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <
> > > >> > > andrija.panic@gmail.com>
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > > (backup, backup, backup...)
> > > >> > > > > > >
> > > >> > > > > > > Can you send the full logs (passtebin or similar please,
> > not
> > > >> in
> > > >> > > > email)
> > > >> > > > > ?
> > > >> > > > > > >
> > > >> > > > > > > Andrija
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > --
> > > >> > > > >
> > > >> > > > > Andrija Panić
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >>
> > > >> --
> > > >>
> > > >> Andrija Panić
> > > >>
> > > >
> > >
> >
>
--
Andrija Panić
Re: Upgrade to 4.13 failed
Posted by Fariborz Navidan <md...@gmail.com>.
I cannot rollback DB to 4.12. How can I download 4.11.3 and and register in
DB?
On Wed, Oct 2, 2019 at 9:43 PM Andrija Panic <an...@gmail.com>
wrote:
> There's a GitHub issue about this (perhaps you or some colleague?)
>
> Did you register 4.11.3 (not 4.13) systemVM template? Documentation has
> just been updated... Sorry for this.
>
> i..e. downgrade packages to 4.12, make sure your DB is rolled back, start
> 4.12 normally, register 4.11.3 systemVM template as per the doc and then
> try again to upgrade.
>
> let me know how it goes.
>
> Andrija
>
> On Wed, 2 Oct 2019, 17:27 Fariborz Navidan, <md...@gmail.com> wrote:
>
> > Any idea?
> >
> > On Wed, Oct 2, 2019 at 3:59 PM Fariborz Navidan <md...@gmail.com>
> > wrote:
> >
> > > Yes, It was clean 4.12 installation.
> > >
> > > On Wed, Oct 2, 2019 at 3:52 PM Andrija Panic <an...@gmail.com>
> > > wrote:
> > >
> > >> "was this a clean 4.12 install, then later upgrade - or was it
> > previously
> > >> 4.11.x then 4.12 etc?"
> > >>
> > >> On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <md...@gmail.com>
> > >> wrote:
> > >>
> > >> > Which question?
> > >> >
> > >> > I have sent pastebin link. I lost 4.12 backup accidentally. I am
> > trying
> > >> to
> > >> > make 4.13 upgrade successfully. I do run following commands before
> > >> upgrade
> > >> > retry:
> > >> >
> > >> > [root@fr-kvm1 ~]# cat rollback.sql
> > >> > delete from guest_os_hypervisor where guest_os_id >= 277;
> > >> > delete from guest_os where id >= 277;
> > >> > SET FOREIGN_KEY_CHECKS=0;
> > >> > drop table direct_download_certificate;
> > >> > drop table vpc_offering_details;
> > >> > drop table direct_download_certificate_host_map;
> > >> > drop table template_ovf_properties;
> > >> > SET FOREIGN_KEY_CHECKS=1;
> > >> > alter table data_center drop column sort_key;
> > >> > alter table vpc_offerings drop column sort_key;
> > >> > alter table disk_offering add column domain_id int(32) not null;
> > >> > alter table service_offering_details add constraint
> > >> > uk_service_offering_id_name unique (id);
> > >> > alter table network_offering_details DROP COLUMN `display`;
> > >> > ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP COLUMN
> > >> `vm_snapshot_id`;
> > >> > ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary` DROP COLUMN
> > >> > `vm_snapshot_id`;
> > >> > [root@fr-kvm1 ~]#
> > >> >
> > >> > However I don't understand the error I showed you on log in
> pastebin.
> > >> > Please check and let me know what does it mean.
> > >> >
> > >> > On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <
> andrija.panic@gmail.com
> > >
> > >> > wrote:
> > >> >
> > >> > > It was not designed to rollback on its own, you are expected to
> > >> rollback
> > >> > DB
> > >> > > yourself (DB restore) and try again if needed.
> > >> > >
> > >> > > But you have not answered my question.
> > >> > >
> > >> > > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <
> mdvlinquest@gmail.com>
> > >> > wrote:
> > >> > >
> > >> > > > In my experience, if DB upgrade fails, it does not fully rolls
> > back
> > >> > > > remaining 4.13's schema upgraded partially and upon next restart
> > it
> > >> > does
> > >> > > > not check existence of 4.13's table or columns causing SQL
> errors
> > >> > > > complaining about tables or columns already exists.
> > >> > > >
> > >> > > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <
> > >> andrija.panic@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > was this a clean 4.12 install, then later upgrade - or was it
> > >> > > previously
> > >> > > > > 4.11.x then 4.12 etc?
> > >> > > > >
> > >> > > > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <
> > >> mdvlinquest@gmail.com
> > >> > >
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > https://pastebin.com/2EFm0jaW
> > >> > > > > >
> > >> > > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <
> > >> > > andrija.panic@gmail.com>
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > (backup, backup, backup...)
> > >> > > > > > >
> > >> > > > > > > Can you send the full logs (passtebin or similar please,
> not
> > >> in
> > >> > > > email)
> > >> > > > > ?
> > >> > > > > > >
> > >> > > > > > > Andrija
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > >
> > >> > > > > Andrija Panić
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >>
> > >> --
> > >>
> > >> Andrija Panić
> > >>
> > >
> >
>
Re: Upgrade to 4.13 failed
Posted by Andrija Panic <an...@gmail.com>.
There's a GitHub issue about this (perhaps you or some colleague?)
Did you register 4.11.3 (not 4.13) systemVM template? Documentation has
just been updated... Sorry for this.
i..e. downgrade packages to 4.12, make sure your DB is rolled back, start
4.12 normally, register 4.11.3 systemVM template as per the doc and then
try again to upgrade.
let me know how it goes.
Andrija
On Wed, 2 Oct 2019, 17:27 Fariborz Navidan, <md...@gmail.com> wrote:
> Any idea?
>
> On Wed, Oct 2, 2019 at 3:59 PM Fariborz Navidan <md...@gmail.com>
> wrote:
>
> > Yes, It was clean 4.12 installation.
> >
> > On Wed, Oct 2, 2019 at 3:52 PM Andrija Panic <an...@gmail.com>
> > wrote:
> >
> >> "was this a clean 4.12 install, then later upgrade - or was it
> previously
> >> 4.11.x then 4.12 etc?"
> >>
> >> On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <md...@gmail.com>
> >> wrote:
> >>
> >> > Which question?
> >> >
> >> > I have sent pastebin link. I lost 4.12 backup accidentally. I am
> trying
> >> to
> >> > make 4.13 upgrade successfully. I do run following commands before
> >> upgrade
> >> > retry:
> >> >
> >> > [root@fr-kvm1 ~]# cat rollback.sql
> >> > delete from guest_os_hypervisor where guest_os_id >= 277;
> >> > delete from guest_os where id >= 277;
> >> > SET FOREIGN_KEY_CHECKS=0;
> >> > drop table direct_download_certificate;
> >> > drop table vpc_offering_details;
> >> > drop table direct_download_certificate_host_map;
> >> > drop table template_ovf_properties;
> >> > SET FOREIGN_KEY_CHECKS=1;
> >> > alter table data_center drop column sort_key;
> >> > alter table vpc_offerings drop column sort_key;
> >> > alter table disk_offering add column domain_id int(32) not null;
> >> > alter table service_offering_details add constraint
> >> > uk_service_offering_id_name unique (id);
> >> > alter table network_offering_details DROP COLUMN `display`;
> >> > ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP COLUMN
> >> `vm_snapshot_id`;
> >> > ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary` DROP COLUMN
> >> > `vm_snapshot_id`;
> >> > [root@fr-kvm1 ~]#
> >> >
> >> > However I don't understand the error I showed you on log in pastebin.
> >> > Please check and let me know what does it mean.
> >> >
> >> > On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <andrija.panic@gmail.com
> >
> >> > wrote:
> >> >
> >> > > It was not designed to rollback on its own, you are expected to
> >> rollback
> >> > DB
> >> > > yourself (DB restore) and try again if needed.
> >> > >
> >> > > But you have not answered my question.
> >> > >
> >> > > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <md...@gmail.com>
> >> > wrote:
> >> > >
> >> > > > In my experience, if DB upgrade fails, it does not fully rolls
> back
> >> > > > remaining 4.13's schema upgraded partially and upon next restart
> it
> >> > does
> >> > > > not check existence of 4.13's table or columns causing SQL errors
> >> > > > complaining about tables or columns already exists.
> >> > > >
> >> > > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <
> >> andrija.panic@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > was this a clean 4.12 install, then later upgrade - or was it
> >> > > previously
> >> > > > > 4.11.x then 4.12 etc?
> >> > > > >
> >> > > > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <
> >> mdvlinquest@gmail.com
> >> > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > https://pastebin.com/2EFm0jaW
> >> > > > > >
> >> > > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <
> >> > > andrija.panic@gmail.com>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > (backup, backup, backup...)
> >> > > > > > >
> >> > > > > > > Can you send the full logs (passtebin or similar please, not
> >> in
> >> > > > email)
> >> > > > > ?
> >> > > > > > >
> >> > > > > > > Andrija
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > >
> >> > > > > Andrija Panić
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
> >> --
> >>
> >> Andrija Panić
> >>
> >
>
Re: Upgrade to 4.13 failed
Posted by Fariborz Navidan <md...@gmail.com>.
Any idea?
On Wed, Oct 2, 2019 at 3:59 PM Fariborz Navidan <md...@gmail.com>
wrote:
> Yes, It was clean 4.12 installation.
>
> On Wed, Oct 2, 2019 at 3:52 PM Andrija Panic <an...@gmail.com>
> wrote:
>
>> "was this a clean 4.12 install, then later upgrade - or was it previously
>> 4.11.x then 4.12 etc?"
>>
>> On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <md...@gmail.com>
>> wrote:
>>
>> > Which question?
>> >
>> > I have sent pastebin link. I lost 4.12 backup accidentally. I am trying
>> to
>> > make 4.13 upgrade successfully. I do run following commands before
>> upgrade
>> > retry:
>> >
>> > [root@fr-kvm1 ~]# cat rollback.sql
>> > delete from guest_os_hypervisor where guest_os_id >= 277;
>> > delete from guest_os where id >= 277;
>> > SET FOREIGN_KEY_CHECKS=0;
>> > drop table direct_download_certificate;
>> > drop table vpc_offering_details;
>> > drop table direct_download_certificate_host_map;
>> > drop table template_ovf_properties;
>> > SET FOREIGN_KEY_CHECKS=1;
>> > alter table data_center drop column sort_key;
>> > alter table vpc_offerings drop column sort_key;
>> > alter table disk_offering add column domain_id int(32) not null;
>> > alter table service_offering_details add constraint
>> > uk_service_offering_id_name unique (id);
>> > alter table network_offering_details DROP COLUMN `display`;
>> > ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP COLUMN
>> `vm_snapshot_id`;
>> > ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary` DROP COLUMN
>> > `vm_snapshot_id`;
>> > [root@fr-kvm1 ~]#
>> >
>> > However I don't understand the error I showed you on log in pastebin.
>> > Please check and let me know what does it mean.
>> >
>> > On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <an...@gmail.com>
>> > wrote:
>> >
>> > > It was not designed to rollback on its own, you are expected to
>> rollback
>> > DB
>> > > yourself (DB restore) and try again if needed.
>> > >
>> > > But you have not answered my question.
>> > >
>> > > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <md...@gmail.com>
>> > wrote:
>> > >
>> > > > In my experience, if DB upgrade fails, it does not fully rolls back
>> > > > remaining 4.13's schema upgraded partially and upon next restart it
>> > does
>> > > > not check existence of 4.13's table or columns causing SQL errors
>> > > > complaining about tables or columns already exists.
>> > > >
>> > > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <
>> andrija.panic@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > was this a clean 4.12 install, then later upgrade - or was it
>> > > previously
>> > > > > 4.11.x then 4.12 etc?
>> > > > >
>> > > > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <
>> mdvlinquest@gmail.com
>> > >
>> > > > > wrote:
>> > > > >
>> > > > > > https://pastebin.com/2EFm0jaW
>> > > > > >
>> > > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <
>> > > andrija.panic@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > (backup, backup, backup...)
>> > > > > > >
>> > > > > > > Can you send the full logs (passtebin or similar please, not
>> in
>> > > > email)
>> > > > > ?
>> > > > > > >
>> > > > > > > Andrija
>> > > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > >
>> > > > > Andrija Panić
>> > > > >
>> > > >
>> > >
>> >
>>
>>
>> --
>>
>> Andrija Panić
>>
>
Re: Upgrade to 4.13 failed
Posted by Fariborz Navidan <md...@gmail.com>.
Yes, It was clean 4.12 installation.
On Wed, Oct 2, 2019 at 3:52 PM Andrija Panic <an...@gmail.com>
wrote:
> "was this a clean 4.12 install, then later upgrade - or was it previously
> 4.11.x then 4.12 etc?"
>
> On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <md...@gmail.com>
> wrote:
>
> > Which question?
> >
> > I have sent pastebin link. I lost 4.12 backup accidentally. I am trying
> to
> > make 4.13 upgrade successfully. I do run following commands before
> upgrade
> > retry:
> >
> > [root@fr-kvm1 ~]# cat rollback.sql
> > delete from guest_os_hypervisor where guest_os_id >= 277;
> > delete from guest_os where id >= 277;
> > SET FOREIGN_KEY_CHECKS=0;
> > drop table direct_download_certificate;
> > drop table vpc_offering_details;
> > drop table direct_download_certificate_host_map;
> > drop table template_ovf_properties;
> > SET FOREIGN_KEY_CHECKS=1;
> > alter table data_center drop column sort_key;
> > alter table vpc_offerings drop column sort_key;
> > alter table disk_offering add column domain_id int(32) not null;
> > alter table service_offering_details add constraint
> > uk_service_offering_id_name unique (id);
> > alter table network_offering_details DROP COLUMN `display`;
> > ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP COLUMN
> `vm_snapshot_id`;
> > ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary` DROP COLUMN
> > `vm_snapshot_id`;
> > [root@fr-kvm1 ~]#
> >
> > However I don't understand the error I showed you on log in pastebin.
> > Please check and let me know what does it mean.
> >
> > On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <an...@gmail.com>
> > wrote:
> >
> > > It was not designed to rollback on its own, you are expected to
> rollback
> > DB
> > > yourself (DB restore) and try again if needed.
> > >
> > > But you have not answered my question.
> > >
> > > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <md...@gmail.com>
> > wrote:
> > >
> > > > In my experience, if DB upgrade fails, it does not fully rolls back
> > > > remaining 4.13's schema upgraded partially and upon next restart it
> > does
> > > > not check existence of 4.13's table or columns causing SQL errors
> > > > complaining about tables or columns already exists.
> > > >
> > > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <
> andrija.panic@gmail.com>
> > > > wrote:
> > > >
> > > > > was this a clean 4.12 install, then later upgrade - or was it
> > > previously
> > > > > 4.11.x then 4.12 etc?
> > > > >
> > > > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <
> mdvlinquest@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > https://pastebin.com/2EFm0jaW
> > > > > >
> > > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <
> > > andrija.panic@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > (backup, backup, backup...)
> > > > > > >
> > > > > > > Can you send the full logs (passtebin or similar please, not in
> > > > email)
> > > > > ?
> > > > > > >
> > > > > > > Andrija
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Andrija Panić
> > > > >
> > > >
> > >
> >
>
>
> --
>
> Andrija Panić
>
Re: Upgrade to 4.13 failed
Posted by Andrija Panic <an...@gmail.com>.
"was this a clean 4.12 install, then later upgrade - or was it previously
4.11.x then 4.12 etc?"
On Wed, 2 Oct 2019 at 14:20, Fariborz Navidan <md...@gmail.com> wrote:
> Which question?
>
> I have sent pastebin link. I lost 4.12 backup accidentally. I am trying to
> make 4.13 upgrade successfully. I do run following commands before upgrade
> retry:
>
> [root@fr-kvm1 ~]# cat rollback.sql
> delete from guest_os_hypervisor where guest_os_id >= 277;
> delete from guest_os where id >= 277;
> SET FOREIGN_KEY_CHECKS=0;
> drop table direct_download_certificate;
> drop table vpc_offering_details;
> drop table direct_download_certificate_host_map;
> drop table template_ovf_properties;
> SET FOREIGN_KEY_CHECKS=1;
> alter table data_center drop column sort_key;
> alter table vpc_offerings drop column sort_key;
> alter table disk_offering add column domain_id int(32) not null;
> alter table service_offering_details add constraint
> uk_service_offering_id_name unique (id);
> alter table network_offering_details DROP COLUMN `display`;
> ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP COLUMN `vm_snapshot_id`;
> ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary` DROP COLUMN
> `vm_snapshot_id`;
> [root@fr-kvm1 ~]#
>
> However I don't understand the error I showed you on log in pastebin.
> Please check and let me know what does it mean.
>
> On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <an...@gmail.com>
> wrote:
>
> > It was not designed to rollback on its own, you are expected to rollback
> DB
> > yourself (DB restore) and try again if needed.
> >
> > But you have not answered my question.
> >
> > On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <md...@gmail.com>
> wrote:
> >
> > > In my experience, if DB upgrade fails, it does not fully rolls back
> > > remaining 4.13's schema upgraded partially and upon next restart it
> does
> > > not check existence of 4.13's table or columns causing SQL errors
> > > complaining about tables or columns already exists.
> > >
> > > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <an...@gmail.com>
> > > wrote:
> > >
> > > > was this a clean 4.12 install, then later upgrade - or was it
> > previously
> > > > 4.11.x then 4.12 etc?
> > > >
> > > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <mdvlinquest@gmail.com
> >
> > > > wrote:
> > > >
> > > > > https://pastebin.com/2EFm0jaW
> > > > >
> > > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <
> > andrija.panic@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > (backup, backup, backup...)
> > > > > >
> > > > > > Can you send the full logs (passtebin or similar please, not in
> > > email)
> > > > ?
> > > > > >
> > > > > > Andrija
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Andrija Panić
> > > >
> > >
> >
>
--
Andrija Panić
Re: Upgrade to 4.13 failed
Posted by Fariborz Navidan <md...@gmail.com>.
Which question?
I have sent pastebin link. I lost 4.12 backup accidentally. I am trying to
make 4.13 upgrade successfully. I do run following commands before upgrade
retry:
[root@fr-kvm1 ~]# cat rollback.sql
delete from guest_os_hypervisor where guest_os_id >= 277;
delete from guest_os where id >= 277;
SET FOREIGN_KEY_CHECKS=0;
drop table direct_download_certificate;
drop table vpc_offering_details;
drop table direct_download_certificate_host_map;
drop table template_ovf_properties;
SET FOREIGN_KEY_CHECKS=1;
alter table data_center drop column sort_key;
alter table vpc_offerings drop column sort_key;
alter table disk_offering add column domain_id int(32) not null;
alter table service_offering_details add constraint
uk_service_offering_id_name unique (id);
alter table network_offering_details DROP COLUMN `display`;
ALTER TABLE `cloud_usage`.`usage_vmsnapshot` DROP COLUMN `vm_snapshot_id`;
ALTER TABLE `cloud_usage`.`usage_snapshot_on_primary` DROP COLUMN
`vm_snapshot_id`;
[root@fr-kvm1 ~]#
However I don't understand the error I showed you on log in pastebin.
Please check and let me know what does it mean.
On Wed, Oct 2, 2019 at 3:27 PM Andrija Panic <an...@gmail.com>
wrote:
> It was not designed to rollback on its own, you are expected to rollback DB
> yourself (DB restore) and try again if needed.
>
> But you have not answered my question.
>
> On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <md...@gmail.com> wrote:
>
> > In my experience, if DB upgrade fails, it does not fully rolls back
> > remaining 4.13's schema upgraded partially and upon next restart it does
> > not check existence of 4.13's table or columns causing SQL errors
> > complaining about tables or columns already exists.
> >
> > On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <an...@gmail.com>
> > wrote:
> >
> > > was this a clean 4.12 install, then later upgrade - or was it
> previously
> > > 4.11.x then 4.12 etc?
> > >
> > > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <md...@gmail.com>
> > > wrote:
> > >
> > > > https://pastebin.com/2EFm0jaW
> > > >
> > > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <
> andrija.panic@gmail.com>
> > > > wrote:
> > > >
> > > > > (backup, backup, backup...)
> > > > >
> > > > > Can you send the full logs (passtebin or similar please, not in
> > email)
> > > ?
> > > > >
> > > > > Andrija
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Andrija Panić
> > >
> >
>
Re: Upgrade to 4.13 failed
Posted by Andrija Panic <an...@gmail.com>.
It was not designed to rollback on its own, you are expected to rollback DB
yourself (DB restore) and try again if needed.
But you have not answered my question.
On Wed, 2 Oct 2019, 13:50 Fariborz Navidan, <md...@gmail.com> wrote:
> In my experience, if DB upgrade fails, it does not fully rolls back
> remaining 4.13's schema upgraded partially and upon next restart it does
> not check existence of 4.13's table or columns causing SQL errors
> complaining about tables or columns already exists.
>
> On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <an...@gmail.com>
> wrote:
>
> > was this a clean 4.12 install, then later upgrade - or was it previously
> > 4.11.x then 4.12 etc?
> >
> > On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <md...@gmail.com>
> > wrote:
> >
> > > https://pastebin.com/2EFm0jaW
> > >
> > > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <an...@gmail.com>
> > > wrote:
> > >
> > > > (backup, backup, backup...)
> > > >
> > > > Can you send the full logs (passtebin or similar please, not in
> email)
> > ?
> > > >
> > > > Andrija
> > > >
> > >
> >
> >
> > --
> >
> > Andrija Panić
> >
>
Re: Upgrade to 4.13 failed
Posted by Fariborz Navidan <md...@gmail.com>.
In my experience, if DB upgrade fails, it does not fully rolls back
remaining 4.13's schema upgraded partially and upon next restart it does
not check existence of 4.13's table or columns causing SQL errors
complaining about tables or columns already exists.
On Wed, Oct 2, 2019 at 3:14 PM Andrija Panic <an...@gmail.com>
wrote:
> was this a clean 4.12 install, then later upgrade - or was it previously
> 4.11.x then 4.12 etc?
>
> On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <md...@gmail.com>
> wrote:
>
> > https://pastebin.com/2EFm0jaW
> >
> > On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <an...@gmail.com>
> > wrote:
> >
> > > (backup, backup, backup...)
> > >
> > > Can you send the full logs (passtebin or similar please, not in email)
> ?
> > >
> > > Andrija
> > >
> >
>
>
> --
>
> Andrija Panić
>
Re: Upgrade to 4.13 failed
Posted by Andrija Panic <an...@gmail.com>.
was this a clean 4.12 install, then later upgrade - or was it previously
4.11.x then 4.12 etc?
On Wed, 2 Oct 2019 at 12:21, Fariborz Navidan <md...@gmail.com> wrote:
> https://pastebin.com/2EFm0jaW
>
> On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <an...@gmail.com>
> wrote:
>
> > (backup, backup, backup...)
> >
> > Can you send the full logs (passtebin or similar please, not in email) ?
> >
> > Andrija
> >
>
--
Andrija Panić
Re: Upgrade to 4.13 failed
Posted by Fariborz Navidan <md...@gmail.com>.
https://pastebin.com/2EFm0jaW
On Wed, Oct 2, 2019 at 1:19 PM Andrija Panic <an...@gmail.com>
wrote:
> (backup, backup, backup...)
>
> Can you send the full logs (passtebin or similar please, not in email) ?
>
> Andrija
>
Re: Upgrade to 4.13 failed
Posted by Andrija Panic <an...@gmail.com>.
(backup, backup, backup...)
Can you send the full logs (passtebin or similar please, not in email) ?
Andrija