You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Jevgeni Zolotarjov <j....@gmail.com> on 2019/03/21 12:06:15 UTC
cannot start system VMs: disaster after maintenance followup
I reinstalled cloudstack from scratch - everything
But looks like I hit the same wall now
In the last step of installation it cannot create system VMs.
service libvirtd status -l
gives me
------------------------------------
● libvirtd.service - Virtualization daemon
Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled;
vendor preset: enabled)
Active: active (running) since Thu 2019-03-21 11:45:00 GMT; 18min ago
Docs: man:libvirtd(8)
https://libvirt.org
Main PID: 537 (libvirtd)
Tasks: 20 (limit: 32768)
CGroup: /system.slice/libvirtd.service
├─ 537 /usr/sbin/libvirtd -l
├─12206 /usr/sbin/dnsmasq
--conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
--dhcp-script=/usr/libexec/libvirt_leaseshelper
└─12207 /usr/sbin/dnsmasq
--conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
--dhcp-script=/usr/libexec/libvirt_leaseshelper
Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
11:45:01.168+0000: 566: info : libvirt version: 4.5.0, package: 10.el7_6.6
(CentOS BuildSystem <http://bugs.centos.org>, 2019-03-14-10:21:47,
x86-01.bsys.centos.org)
Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
11:45:01.168+0000: 566: info : hostname: mtl1-apphst03.mt.pbt.com.mt
Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
11:45:01.168+0000: 566: error : virFirewallApplyRuleDirect:709 : internal
error: Failed to apply firewall rules /usr/sbin/iptables -w --table nat
--insert POSTROUTING --source 192.168.122.0/24 '!' --destination
192.168.122.0/24 --jump MASQUERADE: iptables v1.4.21: can't initialize
iptables table `nat': Table does not exist (do you need to insmod?)
Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: Perhaps iptables
or your kernel needs to be upgraded.
Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]: read /etc/hosts
- 4 addresses
Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]: read
/var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq-dhcp[12206]: read
/var/lib/libvirt/dnsmasq/default.hostsfile
Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
11:45:01.354+0000: 566: warning : virSecurityManagerNew:189 : Configured
security driver "none" disables default policy to create confined guests
Mar 21 11:49:57 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
11:49:57.354+0000: 542: warning : qemuDomainObjTaint:7521 : Domain id=2
name='s-1-VM' uuid=1a06d3a7-4e3f-4cba-912f-74ae24569bac is tainted:
high-privileges
Mar 21 11:49:59 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
11:49:59.402+0000: 540: warning : qemuDomainObjTaint:7521 : Domain id=3
name='v-2-VM' uuid=af2a8342-cd9b-4b55-ba12-480634a31d65 is tainted:
high-privileges
What can be done about that ?
Re: cannot start system VMs: disaster after maintenance followup
Posted by "Riepl, Gregor (SWISS TXT)" <Gr...@swisstxt.ch>.
> I have a problem with my cloud-management web UI. It just stopped
> accepting connections and gives apache error.
>
> Is theer anyway I could get some help from you?
Is this related to Jevgeni's issue or a completely new one?
You shouldn't simply hit reply on a mail thread, as that will make the
context very confusing...
About the issue:
Can you be a bit more specific about the error you're getting?
"apache error" doesn't say much.
RE: cannot start system VMs: disaster after maintenance followup
Posted by Sam Ceylani <sa...@mistercertified.com>.
Hello,
I have a problem with my cloud-management web UI. It just stopped accepting connections and gives apache error.
Is theer anyway I could get some help from you?
Ricky
-----Original Message-----
From: Andrija Panic <an...@gmail.com>
Sent: Thursday, March 21, 2019 3:23 PM
To: users <us...@cloudstack.apache.org>
Subject: Re: cannot start system VMs: disaster after maintenance followup
Stick to 4.11.2 - 4.12 should be released withing few days officially.
As for qemu-kvm-ev - yes, it's supposed to work - make sure to test new versions obviously.
Did you got your new installation running fine ?
On Thu, 21 Mar 2019 at 19:26, Jevgeni Zolotarjov <j....@gmail.com>
wrote:
> Andrija,
>
> I asked here in the group if its safe to try new version of KVM and
> got reply, that it works. It was back in September. So we installed it
> with yum install centos-release-qemu-ev yum install qemu-kvm-ev
>
> It worked fine ever since.
> But with new maintenance (yum update) apparently some breaking changes
> were introduced.
> So, take care.
>
> Anyway, thanks. for help.
>
> As for your suggestion to use CS4.12. I haven't managed to find
> systemvm images for 4.12. Should I continue to use 4.11.12 systemvm?
>
>
>
>
>
>
> On Thu, Mar 21, 2019 at 7:19 PM Andrija Panic
> <an...@gmail.com>
> wrote:
>
> > Jevgeni, qemu-kvm 1.5.3 is the lastest official one for CentoS
> > 7.6.XXX
> > (latest) which I'm running atm in my lab (just checked for update) -
> > how did you manage to go to 2.0 (custom repo ?)
> >
> > On Thu, 21 Mar 2019 at 18:13, Ivan Kudryavtsev
> > <kudryavtsev_ia@bw-sw.com
> >
> > wrote:
> >
> > > Jevgeniy, simplest and the most obvious way is to flatten their
> > > images
> > with
> > > "qemu-img convert", next import them as templates and recreate VMs
> > > from those templates.
> > >
> > > чт, 21 мар. 2019 г. в 13:05, Jevgeni Zolotarjov <
> j.zolotarjov@gmail.com
> > >:
> > >
> > > > What happened in the end was: qemu-kvm got updated to version
> > > > 2.0
> > during
> > > > the maintenance. We could not manage to make this KVM to work
> > > > with Cloudstack.
> > > > So we rolled back to version 1.5.3.
> > > >
> > > > And now we have clean cloudstack fully operational. We can
> > > > create new
> > VMs
> > > > and it works. I am almost happy.
> > > >
> > > > Now question - how do I get my old VMs to work, considering I
> > > > have
> only
> > > > their volumes?
> > > >
> > > > On Thu, Mar 21, 2019 at 6:24 PM Andrija Panic <
> andrija.panic@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Just replace the URL for systemVM template from 4.11.1 with
> > > > > 4.11.2
> > > (there
> > > > > is a PR for this now).
> > > > >
> > > > > On Thu, 21 Mar 2019 at 16:53, Andrija Panic <
> andrija.panic@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Please use the one, updated specifically for CentOS 7 -
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/cloudstack-documentation/blob/master/source/
> quickinstallationguide/qig.rst
> > > > > >
> > > > > > And please avoid collocating KVM and MGMT on same server
> > (especially
> > > in
> > > > > > any production-like system)
> > > > > >
> > > > > > Please let me know if the guide above gives you problem - we
> > > > > > had
> > > > multiple
> > > > > > users explicitly following it - and successfully installed
> > > > > > (with
> > some
> > > > > minor
> > > > > > modification, which we committed back to that guide).
> > > > > >
> > > > > > Thanks
> > > > > > Andrija
> > > > > >
> > > > > > On Thu, 21 Mar 2019 at 16:34, Jevgeni Zolotarjov <
> > > > j.zolotarjov@gmail.com
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> OS management - centos 7 (1810) OS hypervisor - centos 7
> > > > > >> (1810)
> > > > > >>
> > > > > >> Basic zone - yes
> > > > > >> I am following this quide
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> http://docs.cloudstack.apache.org/en/4.11.2.0/quickinstallationguide/q
> ig.html
> > > > > >>
> > > > > >> Right now from scratch - management ans hypervisor on the
> > > > > >> same
> > > machine
> > > > > >> qemu - version 1.5.3
> > > > > >> libvirt - libvirt version: 4.5.0, package: 10.el7_6.6
> > > > > >>
> > > > > >> Basically - everything out of the box of clean centos
> > > > > >> install
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Thu, Mar 21, 2019 at 5:08 PM Andrija Panic <
> > > > andrija.panic@gmail.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hey Jevgeni,
> > > > > >> >
> > > > > >> > what OS mgmt, what OS hypervisor, what qemu/libvirt
> > > > > >> > versions -
> > > still
> > > > > in
> > > > > >> > Basic Zone, SG ?
> > > > > >> >
> > > > > >> > Andrija
> > > > > >> >
> > > > > >> > On Thu, 21 Mar 2019 at 13:06, Jevgeni Zolotarjov <
> > > > > >> j.zolotarjov@gmail.com>
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> > > I reinstalled cloudstack from scratch - everything
> > > > > >> > >
> > > > > >> > > But looks like I hit the same wall now
> > > > > >> > >
> > > > > >> > > In the last step of installation it cannot create
> > > > > >> > > system
> VMs.
> > > > > >> > >
> > > > > >> > > service libvirtd status -l gives me
> > > > > >> > > ------------------------------------
> > > > > >> > > ● libvirtd.service - Virtualization daemon
> > > > > >> > > Loaded: loaded
> > > > > >> > > (/usr/lib/systemd/system/libvirtd.service;
> > > > > enabled;
> > > > > >> > > vendor preset: enabled)
> > > > > >> > > Active: active (running) since Thu 2019-03-21
> > > > > >> > > 11:45:00
> GMT;
> > > > 18min
> > > > > >> ago
> > > > > >> > > Docs: man:libvirtd(8)
> > > > > >> > > https://libvirt.org Main PID: 537
> > > > > >> > > (libvirtd)
> > > > > >> > > Tasks: 20 (limit: 32768)
> > > > > >> > > CGroup: /system.slice/libvirtd.service
> > > > > >> > > ├─ 537 /usr/sbin/libvirtd -l
> > > > > >> > > ├─12206 /usr/sbin/dnsmasq
> > > > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf
> > --leasefile-ro
> > > > > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > > > >> > > └─12207 /usr/sbin/dnsmasq
> > > > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf
> > --leasefile-ro
> > > > > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > > > >> > >
> > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > > 2019-03-21
> > > > > >> > > 11:45:01.168+0000: 566: info : libvirt version: 4.5.0,
> > package:
> > > > > >> > 10.el7_6.6
> > > > > >> > > (CentOS BuildSystem <http://bugs.centos.org>,
> > > > 2019-03-14-10:21:47,
> > > > > >> > > x86-01.bsys.centos.org) Mar 21 11:45:01
> > > > > >> > > mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > > 2019-03-21
> > > > > >> > > 11:45:01.168+0000: 566: info : hostname:
> > > > > mtl1-apphst03.mt.pbt.com.mt
> > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > > 2019-03-21
> > > > > >> > > 11:45:01.168+0000: 566: error :
> > virFirewallApplyRuleDirect:709 :
> > > > > >> internal
> > > > > >> > > error: Failed to apply firewall rules
> > > > > >> > > /usr/sbin/iptables -w
> > > > --table
> > > > > >> nat
> > > > > >> > > --insert POSTROUTING --source 192.168.122.0/24 '!'
> > > --destination
> > > > > >> > > 192.168.122.0/24 --jump MASQUERADE: iptables v1.4.21:
> > > > > >> > > can't
> > > > > >> initialize
> > > > > >> > > iptables table `nat': Table does not exist (do you need
> > > > > >> > > to
> > > > insmod?)
> > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > Perhaps
> > > > > >> > > iptables
> > > > > >> > > or your kernel needs to be upgraded.
> > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]:
> > > read
> > > > > >> > > /etc/hosts
> > > > > >> > > - 4 addresses
> > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]:
> > > read
> > > > > >> > > /var/lib/libvirt/dnsmasq/default.addnhosts - 0
> > > > > >> > > addresses Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> > > dnsmasq-dhcp[12206]:
> > > > > read
> > > > > >> > > /var/lib/libvirt/dnsmasq/default.hostsfile
> > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > > 2019-03-21
> > > > > >> > > 11:45:01.354+0000: 566: warning :
> > > > > >> > > virSecurityManagerNew:189
> :
> > > > > >> Configured
> > > > > >> > > security driver "none" disables default policy to
> > > > > >> > > create
> > > confined
> > > > > >> guests
> > > > > >> > > Mar 21 11:49:57 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > > 2019-03-21
> > > > > >> > > 11:49:57.354+0000: 542: warning : qemuDomainObjTaint:7521 :
> > > Domain
> > > > > >> id=2
> > > > > >> > > name='s-1-VM' uuid=1a06d3a7-4e3f-4cba-912f-74ae24569bac
> > > > > >> > > is
> > > > tainted:
> > > > > >> > > high-privileges
> > > > > >> > > Mar 21 11:49:59 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > > 2019-03-21
> > > > > >> > > 11:49:59.402+0000: 540: warning : qemuDomainObjTaint:7521 :
> > > Domain
> > > > > >> id=3
> > > > > >> > > name='v-2-VM' uuid=af2a8342-cd9b-4b55-ba12-480634a31d65
> > > > > >> > > is
> > > > tainted:
> > > > > >> > > high-privileges
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > What can be done about that ?
> > > > > >> > >
> > > > > >> >
> > > > > >> >
> > > > > >> > --
> > > > > >> >
> > > > > >> > Andrija Panić
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Andrija Panić
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Andrija Panić
> > > > >
> > > >
> > >
> > >
> > > --
> > > With best regards, Ivan Kudryavtsev Bitworks LLC Cell RU:
> > > +7-923-414-1515 Cell USA: +1-201-257-1512
> > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > >
> >
> >
> > --
> >
> > Andrija Panić
> >
>
--
Andrija Panić
Re: cannot start system VMs: disaster after maintenance followup
Posted by "Riepl, Gregor (SWISS TXT)" <Gr...@swisstxt.ch>.
Hi Jevgeni,
> (1) client -> VM1:80/app -> VM2:8080/app
> (2) client -> VM1:80/data -> VM3:8080/data
>
> This was working fine before the reinstallation.
> We found that it works, if we stop iptables.
>
> But with iptables ON, (1) works, but (2) does not work - it gives
> connection refused.
> How can this be resolved?
With "iptables ON", I assume you are referring to the VM's own
firewall, correct?
Netfilter logging should help to debug this. You can find plenty of
resources on how to create log chains on the web, for example here:
https://www.thegeekstuff.com/2012/08/iptables-log-packets/
If you meant the CloudStack firewall, make sure you have the correct
rules in place. Note that if you're using *external* (i.e. NAT) IPs to
access a service from a different VM, you need an explicit firewall
rule. It's best to use internal hostnames/IPs for VM-to-VM traffic
though.
Does that help or did I totally miss your point?
Re: cannot start system VMs: disaster after maintenance followup
Posted by Jevgeni Zolotarjov <j....@gmail.com>.
This saga is to be continued.
"Security groups" was the correct keyword to resolve my problem.
Now all is in order and all VMs run.
One observation:
This guide here suggests to configure
/etc/libvirt/libvirtd.conf
and
/etc/sysconfig/libvirtd
under
Libvirt Configuration
But these files get overwritten every time cloudstack-agent service is
restarted.
I think, there is inconsistency in this guide for sure.
But Now I face the other problem , which is probably related to correct
configuration of security groups, but maybe it a bug
We have following config
VM1 - running ngninx proxy
VM2 - server hosting webapp on 8080
VM3 - server hosting another webapp on 8080. This webapp is exposing is
connection over websocket - serving data stream
(1) client -> VM1:80/app -> VM2:8080/app
(2) client -> VM1:80/data -> VM3:8080/data
This was working fine before the reinstallation.
We found that it works, if we stop iptables.
But with iptables ON, (1) works, but (2) does not work - it gives
connection refused.
How can this be resolved?
On Fri, Mar 22, 2019 at 11:19 AM Dag Sonstebo <Da...@shapeblue.com>
wrote:
> Jevgeni - you've not provided any network troubleshooting findings - but
> this is all down to security groups so check these are in place and working.
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
>
> On 21/03/2019, 19:47, "Jevgeni Zolotarjov" <j....@gmail.com>
> wrote:
>
> <<<Did you got your new installation running fine ?
> Almost, but not completely.
>
> I am moving VMs one by one. They run and they get IP address from
> Cloudstack and get connected to network.
>
> But I cannot connect to VMs from other PCs in the same LAN. Ping is not
> responding too.
> What can be the problem here?
>
>
> On Thu, 21 Mar 2019, 21:23 Andrija Panic, <an...@gmail.com>
> wrote:
>
> > Stick to 4.11.2 - 4.12 should be released withing few days
> officially.
> >
> > As for qemu-kvm-ev - yes, it's supposed to work - make sure to test
> new
> > versions obviously.
> >
> > Did you got your new installation running fine ?
> >
> > On Thu, 21 Mar 2019 at 19:26, Jevgeni Zolotarjov <
> j.zolotarjov@gmail.com>
> > wrote:
> >
> > > Andrija,
> > >
> > > I asked here in the group if its safe to try new version of KVM
> and got
> > > reply, that it works. It was back in September. So we installed it
> with
> > > yum install centos-release-qemu-ev
> > > yum install qemu-kvm-ev
> > >
> > > It worked fine ever since.
> > > But with new maintenance (yum update) apparently some breaking
> changes
> > were
> > > introduced.
> > > So, take care.
> > >
> > > Anyway, thanks. for help.
> > >
> > > As for your suggestion to use CS4.12. I haven't managed to find
> systemvm
> > > images for 4.12. Should I continue to use 4.11.12 systemvm?
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Thu, Mar 21, 2019 at 7:19 PM Andrija Panic <
> andrija.panic@gmail.com>
> > > wrote:
> > >
> > > > Jevgeni, qemu-kvm 1.5.3 is the lastest official one for CentoS
> 7.6.XXX
> > > > (latest) which I'm running atm in my lab (just checked for
> update) -
> > how
> > > > did you manage to go to 2.0 (custom repo ?)
> > > >
> > > > On Thu, 21 Mar 2019 at 18:13, Ivan Kudryavtsev <
> > kudryavtsev_ia@bw-sw.com
> > > >
> > > > wrote:
> > > >
> > > > > Jevgeniy, simplest and the most obvious way is to flatten their
> > images
> > > > with
> > > > > "qemu-img convert", next import them as templates and recreate
> VMs
> > from
> > > > > those templates.
> > > > >
> > > > > чт, 21 мар. 2019 г. в 13:05, Jevgeni Zolotarjov <
> > > j.zolotarjov@gmail.com
> > > > >:
> > > > >
> > > > > > What happened in the end was: qemu-kvm got updated to
> version 2.0
> > > > during
> > > > > > the maintenance. We could not manage to make this KVM to
> work with
> > > > > > Cloudstack.
> > > > > > So we rolled back to version 1.5.3.
> > > > > >
> > > > > > And now we have clean cloudstack fully operational. We can
> create
> > new
> > > > VMs
> > > > > > and it works. I am almost happy.
> > > > > >
> > > > > > Now question - how do I get my old VMs to work, considering
> I have
> > > only
> > > > > > their volumes?
> > > > > >
> > > > > > On Thu, Mar 21, 2019 at 6:24 PM Andrija Panic <
> > > andrija.panic@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Just replace the URL for systemVM template from 4.11.1 with
> > 4.11.2
> > > > > (there
> > > > > > > is a PR for this now).
> > > > > > >
> > > > > > > On Thu, 21 Mar 2019 at 16:53, Andrija Panic <
> > > andrija.panic@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Please use the one, updated specifically for CentOS 7 -
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/cloudstack-documentation/blob/master/source/quickinstallationguide/qig.rst
> > > > > > > >
> > > > > > > > And please avoid collocating KVM and MGMT on same server
> > > > (especially
> > > > > in
> > > > > > > > any production-like system)
> > > > > > > >
> > > > > > > > Please let me know if the guide above gives you problem
> - we
> > had
> > > > > > multiple
> > > > > > > > users explicitly following it - and successfully
> installed
> > (with
> > > > some
> > > > > > > minor
> > > > > > > > modification, which we committed back to that guide).
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > Andrija
> > > > > > > >
> > > > > > > > On Thu, 21 Mar 2019 at 16:34, Jevgeni Zolotarjov <
> > > > > > j.zolotarjov@gmail.com
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> OS management - centos 7 (1810)
> > > > > > > >> OS hypervisor - centos 7 (1810)
> > > > > > > >>
> > > > > > > >> Basic zone - yes
> > > > > > > >> I am following this quide
> > > > > > > >>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://docs.cloudstack.apache.org/en/4.11.2.0/quickinstallationguide/qig.html
> > > > > > > >>
> > > > > > > >> Right now from scratch - management ans hypervisor on
> the same
> > > > > machine
> > > > > > > >> qemu - version 1.5.3
> > > > > > > >> libvirt - libvirt version: 4.5.0, package: 10.el7_6.6
> > > > > > > >>
> > > > > > > >> Basically - everything out of the box of clean centos
> install
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Thu, Mar 21, 2019 at 5:08 PM Andrija Panic <
> > > > > > andrija.panic@gmail.com>
> > > > > > > >> wrote:
> > > > > > > >>
> > > > > > > >> > Hey Jevgeni,
> > > > > > > >> >
> > > > > > > >> > what OS mgmt, what OS hypervisor, what qemu/libvirt
> > versions -
> > > > > still
> > > > > > > in
> > > > > > > >> > Basic Zone, SG ?
> > > > > > > >> >
> > > > > > > >> > Andrija
> > > > > > > >> >
> > > > > > > >> > On Thu, 21 Mar 2019 at 13:06, Jevgeni Zolotarjov <
> > > > > > > >> j.zolotarjov@gmail.com>
> > > > > > > >> > wrote:
> > > > > > > >> >
> > > > > > > >> > > I reinstalled cloudstack from scratch - everything
> > > > > > > >> > >
> > > > > > > >> > > But looks like I hit the same wall now
> > > > > > > >> > >
> > > > > > > >> > > In the last step of installation it cannot create
> system
> > > VMs.
> > > > > > > >> > >
> > > > > > > >> > > service libvirtd status -l
> > > > > > > >> > > gives me
> > > > > > > >> > > ------------------------------------
> > > > > > > >> > > ● libvirtd.service - Virtualization daemon
> > > > > > > >> > > Loaded: loaded
> > (/usr/lib/systemd/system/libvirtd.service;
> > > > > > > enabled;
> > > > > > > >> > > vendor preset: enabled)
> > > > > > > >> > > Active: active (running) since Thu 2019-03-21
> 11:45:00
> > > GMT;
> > > > > > 18min
> > > > > > > >> ago
> > > > > > > >> > > Docs: man:libvirtd(8)
> > > > > > > >> > > https://libvirt.org
> > > > > > > >> > > Main PID: 537 (libvirtd)
> > > > > > > >> > > Tasks: 20 (limit: 32768)
> > > > > > > >> > > CGroup: /system.slice/libvirtd.service
> > > > > > > >> > > ├─ 537 /usr/sbin/libvirtd -l
> > > > > > > >> > > ├─12206 /usr/sbin/dnsmasq
> > > > > > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf
> > > > --leasefile-ro
> > > > > > > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > > > > > >> > > └─12207 /usr/sbin/dnsmasq
> > > > > > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf
> > > > --leasefile-ro
> > > > > > > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > > > > > >> > >
> > > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> > libvirtd[537]:
> > > > > > > 2019-03-21
> > > > > > > >> > > 11:45:01.168+0000: 566: info : libvirt version:
> 4.5.0,
> > > > package:
> > > > > > > >> > 10.el7_6.6
> > > > > > > >> > > (CentOS BuildSystem <http://bugs.centos.org>,
> > > > > > 2019-03-14-10:21:47,
> > > > > > > >> > > x86-01.bsys.centos.org)
> > > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> > libvirtd[537]:
> > > > > > > 2019-03-21
> > > > > > > >> > > 11:45:01.168+0000: 566: info : hostname:
> > > > > > > mtl1-apphst03.mt.pbt.com.mt
> > > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> > libvirtd[537]:
> > > > > > > 2019-03-21
> > > > > > > >> > > 11:45:01.168+0000: 566: error :
> > > > virFirewallApplyRuleDirect:709 :
> > > > > > > >> internal
> > > > > > > >> > > error: Failed to apply firewall rules
> /usr/sbin/iptables
> > -w
> > > > > > --table
> > > > > > > >> nat
> > > > > > > >> > > --insert POSTROUTING --source 192.168.122.0/24 '!'
> > > > > --destination
> > > > > > > >> > > 192.168.122.0/24 --jump MASQUERADE: iptables
> v1.4.21:
> > can't
> > > > > > > >> initialize
> > > > > > > >> > > iptables table `nat': Table does not exist (do you
> need to
> > > > > > insmod?)
> > > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> > libvirtd[537]:
> > > > > > Perhaps
> > > > > > > >> > > iptables
> > > > > > > >> > > or your kernel needs to be upgraded.
> > > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> > dnsmasq[12206]:
> > > > > read
> > > > > > > >> > > /etc/hosts
> > > > > > > >> > > - 4 addresses
> > > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> > dnsmasq[12206]:
> > > > > read
> > > > > > > >> > > /var/lib/libvirt/dnsmasq/default.addnhosts - 0
> addresses
> > > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> > > > > dnsmasq-dhcp[12206]:
> > > > > > > read
> > > > > > > >> > > /var/lib/libvirt/dnsmasq/default.hostsfile
> > > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> > libvirtd[537]:
> > > > > > > 2019-03-21
> > > > > > > >> > > 11:45:01.354+0000: 566: warning :
> > virSecurityManagerNew:189
> > > :
> > > > > > > >> Configured
> > > > > > > >> > > security driver "none" disables default policy to
> create
> > > > > confined
> > > > > > > >> guests
> > > > > > > >> > > Mar 21 11:49:57 mtl1-apphst03.mt.pbt.com.mt
> > libvirtd[537]:
> > > > > > > 2019-03-21
> > > > > > > >> > > 11:49:57.354+0000: 542: warning :
> qemuDomainObjTaint:7521
> > :
> > > > > Domain
> > > > > > > >> id=2
> > > > > > > >> > > name='s-1-VM'
> uuid=1a06d3a7-4e3f-4cba-912f-74ae24569bac is
> > > > > > tainted:
> > > > > > > >> > > high-privileges
> > > > > > > >> > > Mar 21 11:49:59 mtl1-apphst03.mt.pbt.com.mt
> > libvirtd[537]:
> > > > > > > 2019-03-21
> > > > > > > >> > > 11:49:59.402+0000: 540: warning :
> qemuDomainObjTaint:7521
> > :
> > > > > Domain
> > > > > > > >> id=3
> > > > > > > >> > > name='v-2-VM'
> uuid=af2a8342-cd9b-4b55-ba12-480634a31d65 is
> > > > > > tainted:
> > > > > > > >> > > high-privileges
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > > What can be done about that ?
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> > --
> > > > > > > >> >
> > > > > > > >> > Andrija Panić
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > Andrija Panić
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Andrija Panić
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > With best regards, Ivan Kudryavtsev
> > > > > Bitworks LLC
> > > > > Cell RU: +7-923-414-1515
> > > > > Cell USA: +1-201-257-1512
> > > > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Andrija Panić
> > > >
> > >
> >
> >
> > --
> >
> > Andrija Panić
> >
>
>
>
> Dag.Sonstebo@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London WC2E 9DPUK
> @shapeblue
>
>
>
>
Re: cannot start system VMs: disaster after maintenance followup
Posted by Dag Sonstebo <Da...@shapeblue.com>.
Jevgeni - you've not provided any network troubleshooting findings - but this is all down to security groups so check these are in place and working.
Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue
On 21/03/2019, 19:47, "Jevgeni Zolotarjov" <j....@gmail.com> wrote:
<<<Did you got your new installation running fine ?
Almost, but not completely.
I am moving VMs one by one. They run and they get IP address from
Cloudstack and get connected to network.
But I cannot connect to VMs from other PCs in the same LAN. Ping is not
responding too.
What can be the problem here?
On Thu, 21 Mar 2019, 21:23 Andrija Panic, <an...@gmail.com> wrote:
> Stick to 4.11.2 - 4.12 should be released withing few days officially.
>
> As for qemu-kvm-ev - yes, it's supposed to work - make sure to test new
> versions obviously.
>
> Did you got your new installation running fine ?
>
> On Thu, 21 Mar 2019 at 19:26, Jevgeni Zolotarjov <j....@gmail.com>
> wrote:
>
> > Andrija,
> >
> > I asked here in the group if its safe to try new version of KVM and got
> > reply, that it works. It was back in September. So we installed it with
> > yum install centos-release-qemu-ev
> > yum install qemu-kvm-ev
> >
> > It worked fine ever since.
> > But with new maintenance (yum update) apparently some breaking changes
> were
> > introduced.
> > So, take care.
> >
> > Anyway, thanks. for help.
> >
> > As for your suggestion to use CS4.12. I haven't managed to find systemvm
> > images for 4.12. Should I continue to use 4.11.12 systemvm?
> >
> >
> >
> >
> >
> >
> > On Thu, Mar 21, 2019 at 7:19 PM Andrija Panic <an...@gmail.com>
> > wrote:
> >
> > > Jevgeni, qemu-kvm 1.5.3 is the lastest official one for CentoS 7.6.XXX
> > > (latest) which I'm running atm in my lab (just checked for update) -
> how
> > > did you manage to go to 2.0 (custom repo ?)
> > >
> > > On Thu, 21 Mar 2019 at 18:13, Ivan Kudryavtsev <
> kudryavtsev_ia@bw-sw.com
> > >
> > > wrote:
> > >
> > > > Jevgeniy, simplest and the most obvious way is to flatten their
> images
> > > with
> > > > "qemu-img convert", next import them as templates and recreate VMs
> from
> > > > those templates.
> > > >
> > > > чт, 21 мар. 2019 г. в 13:05, Jevgeni Zolotarjov <
> > j.zolotarjov@gmail.com
> > > >:
> > > >
> > > > > What happened in the end was: qemu-kvm got updated to version 2.0
> > > during
> > > > > the maintenance. We could not manage to make this KVM to work with
> > > > > Cloudstack.
> > > > > So we rolled back to version 1.5.3.
> > > > >
> > > > > And now we have clean cloudstack fully operational. We can create
> new
> > > VMs
> > > > > and it works. I am almost happy.
> > > > >
> > > > > Now question - how do I get my old VMs to work, considering I have
> > only
> > > > > their volumes?
> > > > >
> > > > > On Thu, Mar 21, 2019 at 6:24 PM Andrija Panic <
> > andrija.panic@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Just replace the URL for systemVM template from 4.11.1 with
> 4.11.2
> > > > (there
> > > > > > is a PR for this now).
> > > > > >
> > > > > > On Thu, 21 Mar 2019 at 16:53, Andrija Panic <
> > andrija.panic@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Please use the one, updated specifically for CentOS 7 -
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/cloudstack-documentation/blob/master/source/quickinstallationguide/qig.rst
> > > > > > >
> > > > > > > And please avoid collocating KVM and MGMT on same server
> > > (especially
> > > > in
> > > > > > > any production-like system)
> > > > > > >
> > > > > > > Please let me know if the guide above gives you problem - we
> had
> > > > > multiple
> > > > > > > users explicitly following it - and successfully installed
> (with
> > > some
> > > > > > minor
> > > > > > > modification, which we committed back to that guide).
> > > > > > >
> > > > > > > Thanks
> > > > > > > Andrija
> > > > > > >
> > > > > > > On Thu, 21 Mar 2019 at 16:34, Jevgeni Zolotarjov <
> > > > > j.zolotarjov@gmail.com
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >> OS management - centos 7 (1810)
> > > > > > >> OS hypervisor - centos 7 (1810)
> > > > > > >>
> > > > > > >> Basic zone - yes
> > > > > > >> I am following this quide
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> http://docs.cloudstack.apache.org/en/4.11.2.0/quickinstallationguide/qig.html
> > > > > > >>
> > > > > > >> Right now from scratch - management ans hypervisor on the same
> > > > machine
> > > > > > >> qemu - version 1.5.3
> > > > > > >> libvirt - libvirt version: 4.5.0, package: 10.el7_6.6
> > > > > > >>
> > > > > > >> Basically - everything out of the box of clean centos install
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Thu, Mar 21, 2019 at 5:08 PM Andrija Panic <
> > > > > andrija.panic@gmail.com>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > Hey Jevgeni,
> > > > > > >> >
> > > > > > >> > what OS mgmt, what OS hypervisor, what qemu/libvirt
> versions -
> > > > still
> > > > > > in
> > > > > > >> > Basic Zone, SG ?
> > > > > > >> >
> > > > > > >> > Andrija
> > > > > > >> >
> > > > > > >> > On Thu, 21 Mar 2019 at 13:06, Jevgeni Zolotarjov <
> > > > > > >> j.zolotarjov@gmail.com>
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> > > I reinstalled cloudstack from scratch - everything
> > > > > > >> > >
> > > > > > >> > > But looks like I hit the same wall now
> > > > > > >> > >
> > > > > > >> > > In the last step of installation it cannot create system
> > VMs.
> > > > > > >> > >
> > > > > > >> > > service libvirtd status -l
> > > > > > >> > > gives me
> > > > > > >> > > ------------------------------------
> > > > > > >> > > ● libvirtd.service - Virtualization daemon
> > > > > > >> > > Loaded: loaded
> (/usr/lib/systemd/system/libvirtd.service;
> > > > > > enabled;
> > > > > > >> > > vendor preset: enabled)
> > > > > > >> > > Active: active (running) since Thu 2019-03-21 11:45:00
> > GMT;
> > > > > 18min
> > > > > > >> ago
> > > > > > >> > > Docs: man:libvirtd(8)
> > > > > > >> > > https://libvirt.org
> > > > > > >> > > Main PID: 537 (libvirtd)
> > > > > > >> > > Tasks: 20 (limit: 32768)
> > > > > > >> > > CGroup: /system.slice/libvirtd.service
> > > > > > >> > > ├─ 537 /usr/sbin/libvirtd -l
> > > > > > >> > > ├─12206 /usr/sbin/dnsmasq
> > > > > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf
> > > --leasefile-ro
> > > > > > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > > > > >> > > └─12207 /usr/sbin/dnsmasq
> > > > > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf
> > > --leasefile-ro
> > > > > > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > > > > >> > >
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> libvirtd[537]:
> > > > > > 2019-03-21
> > > > > > >> > > 11:45:01.168+0000: 566: info : libvirt version: 4.5.0,
> > > package:
> > > > > > >> > 10.el7_6.6
> > > > > > >> > > (CentOS BuildSystem <http://bugs.centos.org>,
> > > > > 2019-03-14-10:21:47,
> > > > > > >> > > x86-01.bsys.centos.org)
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> libvirtd[537]:
> > > > > > 2019-03-21
> > > > > > >> > > 11:45:01.168+0000: 566: info : hostname:
> > > > > > mtl1-apphst03.mt.pbt.com.mt
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> libvirtd[537]:
> > > > > > 2019-03-21
> > > > > > >> > > 11:45:01.168+0000: 566: error :
> > > virFirewallApplyRuleDirect:709 :
> > > > > > >> internal
> > > > > > >> > > error: Failed to apply firewall rules /usr/sbin/iptables
> -w
> > > > > --table
> > > > > > >> nat
> > > > > > >> > > --insert POSTROUTING --source 192.168.122.0/24 '!'
> > > > --destination
> > > > > > >> > > 192.168.122.0/24 --jump MASQUERADE: iptables v1.4.21:
> can't
> > > > > > >> initialize
> > > > > > >> > > iptables table `nat': Table does not exist (do you need to
> > > > > insmod?)
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> libvirtd[537]:
> > > > > Perhaps
> > > > > > >> > > iptables
> > > > > > >> > > or your kernel needs to be upgraded.
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> dnsmasq[12206]:
> > > > read
> > > > > > >> > > /etc/hosts
> > > > > > >> > > - 4 addresses
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> dnsmasq[12206]:
> > > > read
> > > > > > >> > > /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> > > > dnsmasq-dhcp[12206]:
> > > > > > read
> > > > > > >> > > /var/lib/libvirt/dnsmasq/default.hostsfile
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> libvirtd[537]:
> > > > > > 2019-03-21
> > > > > > >> > > 11:45:01.354+0000: 566: warning :
> virSecurityManagerNew:189
> > :
> > > > > > >> Configured
> > > > > > >> > > security driver "none" disables default policy to create
> > > > confined
> > > > > > >> guests
> > > > > > >> > > Mar 21 11:49:57 mtl1-apphst03.mt.pbt.com.mt
> libvirtd[537]:
> > > > > > 2019-03-21
> > > > > > >> > > 11:49:57.354+0000: 542: warning : qemuDomainObjTaint:7521
> :
> > > > Domain
> > > > > > >> id=2
> > > > > > >> > > name='s-1-VM' uuid=1a06d3a7-4e3f-4cba-912f-74ae24569bac is
> > > > > tainted:
> > > > > > >> > > high-privileges
> > > > > > >> > > Mar 21 11:49:59 mtl1-apphst03.mt.pbt.com.mt
> libvirtd[537]:
> > > > > > 2019-03-21
> > > > > > >> > > 11:49:59.402+0000: 540: warning : qemuDomainObjTaint:7521
> :
> > > > Domain
> > > > > > >> id=3
> > > > > > >> > > name='v-2-VM' uuid=af2a8342-cd9b-4b55-ba12-480634a31d65 is
> > > > > tainted:
> > > > > > >> > > high-privileges
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > > What can be done about that ?
> > > > > > >> > >
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > --
> > > > > > >> >
> > > > > > >> > Andrija Panić
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Andrija Panić
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Andrija Panić
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > With best regards, Ivan Kudryavtsev
> > > > Bitworks LLC
> > > > Cell RU: +7-923-414-1515
> > > > Cell USA: +1-201-257-1512
> > > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > > >
> > >
> > >
> > > --
> > >
> > > Andrija Panić
> > >
> >
>
>
> --
>
> Andrija Panić
>
Dag.Sonstebo@shapeblue.com
www.shapeblue.com
Amadeus House, Floral Street, London WC2E 9DPUK
@shapeblue
Re: cannot start system VMs: disaster after maintenance followup
Posted by Jevgeni Zolotarjov <j....@gmail.com>.
<<<Did you got your new installation running fine ?
Almost, but not completely.
I am moving VMs one by one. They run and they get IP address from
Cloudstack and get connected to network.
But I cannot connect to VMs from other PCs in the same LAN. Ping is not
responding too.
What can be the problem here?
On Thu, 21 Mar 2019, 21:23 Andrija Panic, <an...@gmail.com> wrote:
> Stick to 4.11.2 - 4.12 should be released withing few days officially.
>
> As for qemu-kvm-ev - yes, it's supposed to work - make sure to test new
> versions obviously.
>
> Did you got your new installation running fine ?
>
> On Thu, 21 Mar 2019 at 19:26, Jevgeni Zolotarjov <j....@gmail.com>
> wrote:
>
> > Andrija,
> >
> > I asked here in the group if its safe to try new version of KVM and got
> > reply, that it works. It was back in September. So we installed it with
> > yum install centos-release-qemu-ev
> > yum install qemu-kvm-ev
> >
> > It worked fine ever since.
> > But with new maintenance (yum update) apparently some breaking changes
> were
> > introduced.
> > So, take care.
> >
> > Anyway, thanks. for help.
> >
> > As for your suggestion to use CS4.12. I haven't managed to find systemvm
> > images for 4.12. Should I continue to use 4.11.12 systemvm?
> >
> >
> >
> >
> >
> >
> > On Thu, Mar 21, 2019 at 7:19 PM Andrija Panic <an...@gmail.com>
> > wrote:
> >
> > > Jevgeni, qemu-kvm 1.5.3 is the lastest official one for CentoS 7.6.XXX
> > > (latest) which I'm running atm in my lab (just checked for update) -
> how
> > > did you manage to go to 2.0 (custom repo ?)
> > >
> > > On Thu, 21 Mar 2019 at 18:13, Ivan Kudryavtsev <
> kudryavtsev_ia@bw-sw.com
> > >
> > > wrote:
> > >
> > > > Jevgeniy, simplest and the most obvious way is to flatten their
> images
> > > with
> > > > "qemu-img convert", next import them as templates and recreate VMs
> from
> > > > those templates.
> > > >
> > > > чт, 21 мар. 2019 г. в 13:05, Jevgeni Zolotarjov <
> > j.zolotarjov@gmail.com
> > > >:
> > > >
> > > > > What happened in the end was: qemu-kvm got updated to version 2.0
> > > during
> > > > > the maintenance. We could not manage to make this KVM to work with
> > > > > Cloudstack.
> > > > > So we rolled back to version 1.5.3.
> > > > >
> > > > > And now we have clean cloudstack fully operational. We can create
> new
> > > VMs
> > > > > and it works. I am almost happy.
> > > > >
> > > > > Now question - how do I get my old VMs to work, considering I have
> > only
> > > > > their volumes?
> > > > >
> > > > > On Thu, Mar 21, 2019 at 6:24 PM Andrija Panic <
> > andrija.panic@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Just replace the URL for systemVM template from 4.11.1 with
> 4.11.2
> > > > (there
> > > > > > is a PR for this now).
> > > > > >
> > > > > > On Thu, 21 Mar 2019 at 16:53, Andrija Panic <
> > andrija.panic@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Please use the one, updated specifically for CentOS 7 -
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/cloudstack-documentation/blob/master/source/quickinstallationguide/qig.rst
> > > > > > >
> > > > > > > And please avoid collocating KVM and MGMT on same server
> > > (especially
> > > > in
> > > > > > > any production-like system)
> > > > > > >
> > > > > > > Please let me know if the guide above gives you problem - we
> had
> > > > > multiple
> > > > > > > users explicitly following it - and successfully installed
> (with
> > > some
> > > > > > minor
> > > > > > > modification, which we committed back to that guide).
> > > > > > >
> > > > > > > Thanks
> > > > > > > Andrija
> > > > > > >
> > > > > > > On Thu, 21 Mar 2019 at 16:34, Jevgeni Zolotarjov <
> > > > > j.zolotarjov@gmail.com
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >> OS management - centos 7 (1810)
> > > > > > >> OS hypervisor - centos 7 (1810)
> > > > > > >>
> > > > > > >> Basic zone - yes
> > > > > > >> I am following this quide
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> http://docs.cloudstack.apache.org/en/4.11.2.0/quickinstallationguide/qig.html
> > > > > > >>
> > > > > > >> Right now from scratch - management ans hypervisor on the same
> > > > machine
> > > > > > >> qemu - version 1.5.3
> > > > > > >> libvirt - libvirt version: 4.5.0, package: 10.el7_6.6
> > > > > > >>
> > > > > > >> Basically - everything out of the box of clean centos install
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Thu, Mar 21, 2019 at 5:08 PM Andrija Panic <
> > > > > andrija.panic@gmail.com>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > Hey Jevgeni,
> > > > > > >> >
> > > > > > >> > what OS mgmt, what OS hypervisor, what qemu/libvirt
> versions -
> > > > still
> > > > > > in
> > > > > > >> > Basic Zone, SG ?
> > > > > > >> >
> > > > > > >> > Andrija
> > > > > > >> >
> > > > > > >> > On Thu, 21 Mar 2019 at 13:06, Jevgeni Zolotarjov <
> > > > > > >> j.zolotarjov@gmail.com>
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> > > I reinstalled cloudstack from scratch - everything
> > > > > > >> > >
> > > > > > >> > > But looks like I hit the same wall now
> > > > > > >> > >
> > > > > > >> > > In the last step of installation it cannot create system
> > VMs.
> > > > > > >> > >
> > > > > > >> > > service libvirtd status -l
> > > > > > >> > > gives me
> > > > > > >> > > ------------------------------------
> > > > > > >> > > ● libvirtd.service - Virtualization daemon
> > > > > > >> > > Loaded: loaded
> (/usr/lib/systemd/system/libvirtd.service;
> > > > > > enabled;
> > > > > > >> > > vendor preset: enabled)
> > > > > > >> > > Active: active (running) since Thu 2019-03-21 11:45:00
> > GMT;
> > > > > 18min
> > > > > > >> ago
> > > > > > >> > > Docs: man:libvirtd(8)
> > > > > > >> > > https://libvirt.org
> > > > > > >> > > Main PID: 537 (libvirtd)
> > > > > > >> > > Tasks: 20 (limit: 32768)
> > > > > > >> > > CGroup: /system.slice/libvirtd.service
> > > > > > >> > > ├─ 537 /usr/sbin/libvirtd -l
> > > > > > >> > > ├─12206 /usr/sbin/dnsmasq
> > > > > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf
> > > --leasefile-ro
> > > > > > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > > > > >> > > └─12207 /usr/sbin/dnsmasq
> > > > > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf
> > > --leasefile-ro
> > > > > > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > > > > >> > >
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> libvirtd[537]:
> > > > > > 2019-03-21
> > > > > > >> > > 11:45:01.168+0000: 566: info : libvirt version: 4.5.0,
> > > package:
> > > > > > >> > 10.el7_6.6
> > > > > > >> > > (CentOS BuildSystem <http://bugs.centos.org>,
> > > > > 2019-03-14-10:21:47,
> > > > > > >> > > x86-01.bsys.centos.org)
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> libvirtd[537]:
> > > > > > 2019-03-21
> > > > > > >> > > 11:45:01.168+0000: 566: info : hostname:
> > > > > > mtl1-apphst03.mt.pbt.com.mt
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> libvirtd[537]:
> > > > > > 2019-03-21
> > > > > > >> > > 11:45:01.168+0000: 566: error :
> > > virFirewallApplyRuleDirect:709 :
> > > > > > >> internal
> > > > > > >> > > error: Failed to apply firewall rules /usr/sbin/iptables
> -w
> > > > > --table
> > > > > > >> nat
> > > > > > >> > > --insert POSTROUTING --source 192.168.122.0/24 '!'
> > > > --destination
> > > > > > >> > > 192.168.122.0/24 --jump MASQUERADE: iptables v1.4.21:
> can't
> > > > > > >> initialize
> > > > > > >> > > iptables table `nat': Table does not exist (do you need to
> > > > > insmod?)
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> libvirtd[537]:
> > > > > Perhaps
> > > > > > >> > > iptables
> > > > > > >> > > or your kernel needs to be upgraded.
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> dnsmasq[12206]:
> > > > read
> > > > > > >> > > /etc/hosts
> > > > > > >> > > - 4 addresses
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> dnsmasq[12206]:
> > > > read
> > > > > > >> > > /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> > > > dnsmasq-dhcp[12206]:
> > > > > > read
> > > > > > >> > > /var/lib/libvirt/dnsmasq/default.hostsfile
> > > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> libvirtd[537]:
> > > > > > 2019-03-21
> > > > > > >> > > 11:45:01.354+0000: 566: warning :
> virSecurityManagerNew:189
> > :
> > > > > > >> Configured
> > > > > > >> > > security driver "none" disables default policy to create
> > > > confined
> > > > > > >> guests
> > > > > > >> > > Mar 21 11:49:57 mtl1-apphst03.mt.pbt.com.mt
> libvirtd[537]:
> > > > > > 2019-03-21
> > > > > > >> > > 11:49:57.354+0000: 542: warning : qemuDomainObjTaint:7521
> :
> > > > Domain
> > > > > > >> id=2
> > > > > > >> > > name='s-1-VM' uuid=1a06d3a7-4e3f-4cba-912f-74ae24569bac is
> > > > > tainted:
> > > > > > >> > > high-privileges
> > > > > > >> > > Mar 21 11:49:59 mtl1-apphst03.mt.pbt.com.mt
> libvirtd[537]:
> > > > > > 2019-03-21
> > > > > > >> > > 11:49:59.402+0000: 540: warning : qemuDomainObjTaint:7521
> :
> > > > Domain
> > > > > > >> id=3
> > > > > > >> > > name='v-2-VM' uuid=af2a8342-cd9b-4b55-ba12-480634a31d65 is
> > > > > tainted:
> > > > > > >> > > high-privileges
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > > What can be done about that ?
> > > > > > >> > >
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > --
> > > > > > >> >
> > > > > > >> > Andrija Panić
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Andrija Panić
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Andrija Panić
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > With best regards, Ivan Kudryavtsev
> > > > Bitworks LLC
> > > > Cell RU: +7-923-414-1515
> > > > Cell USA: +1-201-257-1512
> > > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > > >
> > >
> > >
> > > --
> > >
> > > Andrija Panić
> > >
> >
>
>
> --
>
> Andrija Panić
>
Re: cannot start system VMs: disaster after maintenance followup
Posted by Andrija Panic <an...@gmail.com>.
Stick to 4.11.2 - 4.12 should be released withing few days officially.
As for qemu-kvm-ev - yes, it's supposed to work - make sure to test new
versions obviously.
Did you got your new installation running fine ?
On Thu, 21 Mar 2019 at 19:26, Jevgeni Zolotarjov <j....@gmail.com>
wrote:
> Andrija,
>
> I asked here in the group if its safe to try new version of KVM and got
> reply, that it works. It was back in September. So we installed it with
> yum install centos-release-qemu-ev
> yum install qemu-kvm-ev
>
> It worked fine ever since.
> But with new maintenance (yum update) apparently some breaking changes were
> introduced.
> So, take care.
>
> Anyway, thanks. for help.
>
> As for your suggestion to use CS4.12. I haven't managed to find systemvm
> images for 4.12. Should I continue to use 4.11.12 systemvm?
>
>
>
>
>
>
> On Thu, Mar 21, 2019 at 7:19 PM Andrija Panic <an...@gmail.com>
> wrote:
>
> > Jevgeni, qemu-kvm 1.5.3 is the lastest official one for CentoS 7.6.XXX
> > (latest) which I'm running atm in my lab (just checked for update) - how
> > did you manage to go to 2.0 (custom repo ?)
> >
> > On Thu, 21 Mar 2019 at 18:13, Ivan Kudryavtsev <kudryavtsev_ia@bw-sw.com
> >
> > wrote:
> >
> > > Jevgeniy, simplest and the most obvious way is to flatten their images
> > with
> > > "qemu-img convert", next import them as templates and recreate VMs from
> > > those templates.
> > >
> > > чт, 21 мар. 2019 г. в 13:05, Jevgeni Zolotarjov <
> j.zolotarjov@gmail.com
> > >:
> > >
> > > > What happened in the end was: qemu-kvm got updated to version 2.0
> > during
> > > > the maintenance. We could not manage to make this KVM to work with
> > > > Cloudstack.
> > > > So we rolled back to version 1.5.3.
> > > >
> > > > And now we have clean cloudstack fully operational. We can create new
> > VMs
> > > > and it works. I am almost happy.
> > > >
> > > > Now question - how do I get my old VMs to work, considering I have
> only
> > > > their volumes?
> > > >
> > > > On Thu, Mar 21, 2019 at 6:24 PM Andrija Panic <
> andrija.panic@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Just replace the URL for systemVM template from 4.11.1 with 4.11.2
> > > (there
> > > > > is a PR for this now).
> > > > >
> > > > > On Thu, 21 Mar 2019 at 16:53, Andrija Panic <
> andrija.panic@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Please use the one, updated specifically for CentOS 7 -
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/cloudstack-documentation/blob/master/source/quickinstallationguide/qig.rst
> > > > > >
> > > > > > And please avoid collocating KVM and MGMT on same server
> > (especially
> > > in
> > > > > > any production-like system)
> > > > > >
> > > > > > Please let me know if the guide above gives you problem - we had
> > > > multiple
> > > > > > users explicitly following it - and successfully installed (with
> > some
> > > > > minor
> > > > > > modification, which we committed back to that guide).
> > > > > >
> > > > > > Thanks
> > > > > > Andrija
> > > > > >
> > > > > > On Thu, 21 Mar 2019 at 16:34, Jevgeni Zolotarjov <
> > > > j.zolotarjov@gmail.com
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> OS management - centos 7 (1810)
> > > > > >> OS hypervisor - centos 7 (1810)
> > > > > >>
> > > > > >> Basic zone - yes
> > > > > >> I am following this quide
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> http://docs.cloudstack.apache.org/en/4.11.2.0/quickinstallationguide/qig.html
> > > > > >>
> > > > > >> Right now from scratch - management ans hypervisor on the same
> > > machine
> > > > > >> qemu - version 1.5.3
> > > > > >> libvirt - libvirt version: 4.5.0, package: 10.el7_6.6
> > > > > >>
> > > > > >> Basically - everything out of the box of clean centos install
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Thu, Mar 21, 2019 at 5:08 PM Andrija Panic <
> > > > andrija.panic@gmail.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hey Jevgeni,
> > > > > >> >
> > > > > >> > what OS mgmt, what OS hypervisor, what qemu/libvirt versions -
> > > still
> > > > > in
> > > > > >> > Basic Zone, SG ?
> > > > > >> >
> > > > > >> > Andrija
> > > > > >> >
> > > > > >> > On Thu, 21 Mar 2019 at 13:06, Jevgeni Zolotarjov <
> > > > > >> j.zolotarjov@gmail.com>
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> > > I reinstalled cloudstack from scratch - everything
> > > > > >> > >
> > > > > >> > > But looks like I hit the same wall now
> > > > > >> > >
> > > > > >> > > In the last step of installation it cannot create system
> VMs.
> > > > > >> > >
> > > > > >> > > service libvirtd status -l
> > > > > >> > > gives me
> > > > > >> > > ------------------------------------
> > > > > >> > > ● libvirtd.service - Virtualization daemon
> > > > > >> > > Loaded: loaded (/usr/lib/systemd/system/libvirtd.service;
> > > > > enabled;
> > > > > >> > > vendor preset: enabled)
> > > > > >> > > Active: active (running) since Thu 2019-03-21 11:45:00
> GMT;
> > > > 18min
> > > > > >> ago
> > > > > >> > > Docs: man:libvirtd(8)
> > > > > >> > > https://libvirt.org
> > > > > >> > > Main PID: 537 (libvirtd)
> > > > > >> > > Tasks: 20 (limit: 32768)
> > > > > >> > > CGroup: /system.slice/libvirtd.service
> > > > > >> > > ├─ 537 /usr/sbin/libvirtd -l
> > > > > >> > > ├─12206 /usr/sbin/dnsmasq
> > > > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf
> > --leasefile-ro
> > > > > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > > > >> > > └─12207 /usr/sbin/dnsmasq
> > > > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf
> > --leasefile-ro
> > > > > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > > > >> > >
> > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > > 2019-03-21
> > > > > >> > > 11:45:01.168+0000: 566: info : libvirt version: 4.5.0,
> > package:
> > > > > >> > 10.el7_6.6
> > > > > >> > > (CentOS BuildSystem <http://bugs.centos.org>,
> > > > 2019-03-14-10:21:47,
> > > > > >> > > x86-01.bsys.centos.org)
> > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > > 2019-03-21
> > > > > >> > > 11:45:01.168+0000: 566: info : hostname:
> > > > > mtl1-apphst03.mt.pbt.com.mt
> > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > > 2019-03-21
> > > > > >> > > 11:45:01.168+0000: 566: error :
> > virFirewallApplyRuleDirect:709 :
> > > > > >> internal
> > > > > >> > > error: Failed to apply firewall rules /usr/sbin/iptables -w
> > > > --table
> > > > > >> nat
> > > > > >> > > --insert POSTROUTING --source 192.168.122.0/24 '!'
> > > --destination
> > > > > >> > > 192.168.122.0/24 --jump MASQUERADE: iptables v1.4.21: can't
> > > > > >> initialize
> > > > > >> > > iptables table `nat': Table does not exist (do you need to
> > > > insmod?)
> > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > Perhaps
> > > > > >> > > iptables
> > > > > >> > > or your kernel needs to be upgraded.
> > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]:
> > > read
> > > > > >> > > /etc/hosts
> > > > > >> > > - 4 addresses
> > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]:
> > > read
> > > > > >> > > /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
> > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> > > dnsmasq-dhcp[12206]:
> > > > > read
> > > > > >> > > /var/lib/libvirt/dnsmasq/default.hostsfile
> > > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > > 2019-03-21
> > > > > >> > > 11:45:01.354+0000: 566: warning : virSecurityManagerNew:189
> :
> > > > > >> Configured
> > > > > >> > > security driver "none" disables default policy to create
> > > confined
> > > > > >> guests
> > > > > >> > > Mar 21 11:49:57 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > > 2019-03-21
> > > > > >> > > 11:49:57.354+0000: 542: warning : qemuDomainObjTaint:7521 :
> > > Domain
> > > > > >> id=2
> > > > > >> > > name='s-1-VM' uuid=1a06d3a7-4e3f-4cba-912f-74ae24569bac is
> > > > tainted:
> > > > > >> > > high-privileges
> > > > > >> > > Mar 21 11:49:59 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > > 2019-03-21
> > > > > >> > > 11:49:59.402+0000: 540: warning : qemuDomainObjTaint:7521 :
> > > Domain
> > > > > >> id=3
> > > > > >> > > name='v-2-VM' uuid=af2a8342-cd9b-4b55-ba12-480634a31d65 is
> > > > tainted:
> > > > > >> > > high-privileges
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > What can be done about that ?
> > > > > >> > >
> > > > > >> >
> > > > > >> >
> > > > > >> > --
> > > > > >> >
> > > > > >> > Andrija Panić
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Andrija Panić
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Andrija Panić
> > > > >
> > > >
> > >
> > >
> > > --
> > > With best regards, Ivan Kudryavtsev
> > > Bitworks LLC
> > > Cell RU: +7-923-414-1515
> > > Cell USA: +1-201-257-1512
> > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > >
> >
> >
> > --
> >
> > Andrija Panić
> >
>
--
Andrija Panić
Re: cannot start system VMs: disaster after maintenance followup
Posted by Jevgeni Zolotarjov <j....@gmail.com>.
Andrija,
I asked here in the group if its safe to try new version of KVM and got
reply, that it works. It was back in September. So we installed it with
yum install centos-release-qemu-ev
yum install qemu-kvm-ev
It worked fine ever since.
But with new maintenance (yum update) apparently some breaking changes were
introduced.
So, take care.
Anyway, thanks. for help.
As for your suggestion to use CS4.12. I haven't managed to find systemvm
images for 4.12. Should I continue to use 4.11.12 systemvm?
On Thu, Mar 21, 2019 at 7:19 PM Andrija Panic <an...@gmail.com>
wrote:
> Jevgeni, qemu-kvm 1.5.3 is the lastest official one for CentoS 7.6.XXX
> (latest) which I'm running atm in my lab (just checked for update) - how
> did you manage to go to 2.0 (custom repo ?)
>
> On Thu, 21 Mar 2019 at 18:13, Ivan Kudryavtsev <ku...@bw-sw.com>
> wrote:
>
> > Jevgeniy, simplest and the most obvious way is to flatten their images
> with
> > "qemu-img convert", next import them as templates and recreate VMs from
> > those templates.
> >
> > чт, 21 мар. 2019 г. в 13:05, Jevgeni Zolotarjov <j.zolotarjov@gmail.com
> >:
> >
> > > What happened in the end was: qemu-kvm got updated to version 2.0
> during
> > > the maintenance. We could not manage to make this KVM to work with
> > > Cloudstack.
> > > So we rolled back to version 1.5.3.
> > >
> > > And now we have clean cloudstack fully operational. We can create new
> VMs
> > > and it works. I am almost happy.
> > >
> > > Now question - how do I get my old VMs to work, considering I have only
> > > their volumes?
> > >
> > > On Thu, Mar 21, 2019 at 6:24 PM Andrija Panic <andrija.panic@gmail.com
> >
> > > wrote:
> > >
> > > > Just replace the URL for systemVM template from 4.11.1 with 4.11.2
> > (there
> > > > is a PR for this now).
> > > >
> > > > On Thu, 21 Mar 2019 at 16:53, Andrija Panic <andrija.panic@gmail.com
> >
> > > > wrote:
> > > >
> > > > > Please use the one, updated specifically for CentOS 7 -
> > > > >
> > > >
> > >
> >
> https://github.com/apache/cloudstack-documentation/blob/master/source/quickinstallationguide/qig.rst
> > > > >
> > > > > And please avoid collocating KVM and MGMT on same server
> (especially
> > in
> > > > > any production-like system)
> > > > >
> > > > > Please let me know if the guide above gives you problem - we had
> > > multiple
> > > > > users explicitly following it - and successfully installed (with
> some
> > > > minor
> > > > > modification, which we committed back to that guide).
> > > > >
> > > > > Thanks
> > > > > Andrija
> > > > >
> > > > > On Thu, 21 Mar 2019 at 16:34, Jevgeni Zolotarjov <
> > > j.zolotarjov@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > >> OS management - centos 7 (1810)
> > > > >> OS hypervisor - centos 7 (1810)
> > > > >>
> > > > >> Basic zone - yes
> > > > >> I am following this quide
> > > > >>
> > > > >>
> > > >
> > >
> >
> http://docs.cloudstack.apache.org/en/4.11.2.0/quickinstallationguide/qig.html
> > > > >>
> > > > >> Right now from scratch - management ans hypervisor on the same
> > machine
> > > > >> qemu - version 1.5.3
> > > > >> libvirt - libvirt version: 4.5.0, package: 10.el7_6.6
> > > > >>
> > > > >> Basically - everything out of the box of clean centos install
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Thu, Mar 21, 2019 at 5:08 PM Andrija Panic <
> > > andrija.panic@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Hey Jevgeni,
> > > > >> >
> > > > >> > what OS mgmt, what OS hypervisor, what qemu/libvirt versions -
> > still
> > > > in
> > > > >> > Basic Zone, SG ?
> > > > >> >
> > > > >> > Andrija
> > > > >> >
> > > > >> > On Thu, 21 Mar 2019 at 13:06, Jevgeni Zolotarjov <
> > > > >> j.zolotarjov@gmail.com>
> > > > >> > wrote:
> > > > >> >
> > > > >> > > I reinstalled cloudstack from scratch - everything
> > > > >> > >
> > > > >> > > But looks like I hit the same wall now
> > > > >> > >
> > > > >> > > In the last step of installation it cannot create system VMs.
> > > > >> > >
> > > > >> > > service libvirtd status -l
> > > > >> > > gives me
> > > > >> > > ------------------------------------
> > > > >> > > ● libvirtd.service - Virtualization daemon
> > > > >> > > Loaded: loaded (/usr/lib/systemd/system/libvirtd.service;
> > > > enabled;
> > > > >> > > vendor preset: enabled)
> > > > >> > > Active: active (running) since Thu 2019-03-21 11:45:00 GMT;
> > > 18min
> > > > >> ago
> > > > >> > > Docs: man:libvirtd(8)
> > > > >> > > https://libvirt.org
> > > > >> > > Main PID: 537 (libvirtd)
> > > > >> > > Tasks: 20 (limit: 32768)
> > > > >> > > CGroup: /system.slice/libvirtd.service
> > > > >> > > ├─ 537 /usr/sbin/libvirtd -l
> > > > >> > > ├─12206 /usr/sbin/dnsmasq
> > > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf
> --leasefile-ro
> > > > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > > >> > > └─12207 /usr/sbin/dnsmasq
> > > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf
> --leasefile-ro
> > > > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > > >> > >
> > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > 2019-03-21
> > > > >> > > 11:45:01.168+0000: 566: info : libvirt version: 4.5.0,
> package:
> > > > >> > 10.el7_6.6
> > > > >> > > (CentOS BuildSystem <http://bugs.centos.org>,
> > > 2019-03-14-10:21:47,
> > > > >> > > x86-01.bsys.centos.org)
> > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > 2019-03-21
> > > > >> > > 11:45:01.168+0000: 566: info : hostname:
> > > > mtl1-apphst03.mt.pbt.com.mt
> > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > 2019-03-21
> > > > >> > > 11:45:01.168+0000: 566: error :
> virFirewallApplyRuleDirect:709 :
> > > > >> internal
> > > > >> > > error: Failed to apply firewall rules /usr/sbin/iptables -w
> > > --table
> > > > >> nat
> > > > >> > > --insert POSTROUTING --source 192.168.122.0/24 '!'
> > --destination
> > > > >> > > 192.168.122.0/24 --jump MASQUERADE: iptables v1.4.21: can't
> > > > >> initialize
> > > > >> > > iptables table `nat': Table does not exist (do you need to
> > > insmod?)
> > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > Perhaps
> > > > >> > > iptables
> > > > >> > > or your kernel needs to be upgraded.
> > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]:
> > read
> > > > >> > > /etc/hosts
> > > > >> > > - 4 addresses
> > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]:
> > read
> > > > >> > > /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
> > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> > dnsmasq-dhcp[12206]:
> > > > read
> > > > >> > > /var/lib/libvirt/dnsmasq/default.hostsfile
> > > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > 2019-03-21
> > > > >> > > 11:45:01.354+0000: 566: warning : virSecurityManagerNew:189 :
> > > > >> Configured
> > > > >> > > security driver "none" disables default policy to create
> > confined
> > > > >> guests
> > > > >> > > Mar 21 11:49:57 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > 2019-03-21
> > > > >> > > 11:49:57.354+0000: 542: warning : qemuDomainObjTaint:7521 :
> > Domain
> > > > >> id=2
> > > > >> > > name='s-1-VM' uuid=1a06d3a7-4e3f-4cba-912f-74ae24569bac is
> > > tainted:
> > > > >> > > high-privileges
> > > > >> > > Mar 21 11:49:59 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > > 2019-03-21
> > > > >> > > 11:49:59.402+0000: 540: warning : qemuDomainObjTaint:7521 :
> > Domain
> > > > >> id=3
> > > > >> > > name='v-2-VM' uuid=af2a8342-cd9b-4b55-ba12-480634a31d65 is
> > > tainted:
> > > > >> > > high-privileges
> > > > >> > >
> > > > >> > >
> > > > >> > > What can be done about that ?
> > > > >> > >
> > > > >> >
> > > > >> >
> > > > >> > --
> > > > >> >
> > > > >> > Andrija Panić
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Andrija Panić
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Andrija Panić
> > > >
> > >
> >
> >
> > --
> > With best regards, Ivan Kudryavtsev
> > Bitworks LLC
> > Cell RU: +7-923-414-1515
> > Cell USA: +1-201-257-1512
> > WWW: http://bitworks.software/ <http://bw-sw.com/>
> >
>
>
> --
>
> Andrija Panić
>
RE: cannot start system VMs: disaster after maintenance followup
Posted by Piotr Pisz <pi...@piszki.pl>.
Andrija,
Qemu-ev repo add 2.x to CentOS 7
yum install centos-release-qemu-ev
Regards,
Piotr
-----Original Message-----
From: Andrija Panic <an...@gmail.com>
Sent: Thursday, March 21, 2019 6:19 PM
To: users <us...@cloudstack.apache.org>
Subject: Re: cannot start system VMs: disaster after maintenance followup
Jevgeni, qemu-kvm 1.5.3 is the lastest official one for CentoS 7.6.XXX
(latest) which I'm running atm in my lab (just checked for update) - how did you manage to go to 2.0 (custom repo ?)
On Thu, 21 Mar 2019 at 18:13, Ivan Kudryavtsev <ku...@bw-sw.com>
wrote:
> Jevgeniy, simplest and the most obvious way is to flatten their images
> with "qemu-img convert", next import them as templates and recreate
> VMs from those templates.
>
> чт, 21 мар. 2019 г. в 13:05, Jevgeni Zolotarjov <j....@gmail.com>:
>
> > What happened in the end was: qemu-kvm got updated to version 2.0
> > during the maintenance. We could not manage to make this KVM to
> > work with Cloudstack.
> > So we rolled back to version 1.5.3.
> >
> > And now we have clean cloudstack fully operational. We can create
> > new VMs and it works. I am almost happy.
> >
> > Now question - how do I get my old VMs to work, considering I have
> > only their volumes?
> >
> > On Thu, Mar 21, 2019 at 6:24 PM Andrija Panic
> > <an...@gmail.com>
> > wrote:
> >
> > > Just replace the URL for systemVM template from 4.11.1 with 4.11.2
> (there
> > > is a PR for this now).
> > >
> > > On Thu, 21 Mar 2019 at 16:53, Andrija Panic
> > > <an...@gmail.com>
> > > wrote:
> > >
> > > > Please use the one, updated specifically for CentOS 7 -
> > > >
> > >
> >
> https://github.com/apache/cloudstack-documentation/blob/master/source/
> quickinstallationguide/qig.rst
> > > >
> > > > And please avoid collocating KVM and MGMT on same server
> > > > (especially
> in
> > > > any production-like system)
> > > >
> > > > Please let me know if the guide above gives you problem - we had
> > multiple
> > > > users explicitly following it - and successfully installed (with
> > > > some
> > > minor
> > > > modification, which we committed back to that guide).
> > > >
> > > > Thanks
> > > > Andrija
> > > >
> > > > On Thu, 21 Mar 2019 at 16:34, Jevgeni Zolotarjov <
> > j.zolotarjov@gmail.com
> > > >
> > > > wrote:
> > > >
> > > >> OS management - centos 7 (1810) OS hypervisor - centos 7 (1810)
> > > >>
> > > >> Basic zone - yes
> > > >> I am following this quide
> > > >>
> > > >>
> > >
> >
> http://docs.cloudstack.apache.org/en/4.11.2.0/quickinstallationguide/q
> ig.html
> > > >>
> > > >> Right now from scratch - management ans hypervisor on the same
> machine
> > > >> qemu - version 1.5.3
> > > >> libvirt - libvirt version: 4.5.0, package: 10.el7_6.6
> > > >>
> > > >> Basically - everything out of the box of clean centos install
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Thu, Mar 21, 2019 at 5:08 PM Andrija Panic <
> > andrija.panic@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Hey Jevgeni,
> > > >> >
> > > >> > what OS mgmt, what OS hypervisor, what qemu/libvirt versions
> > > >> > -
> still
> > > in
> > > >> > Basic Zone, SG ?
> > > >> >
> > > >> > Andrija
> > > >> >
> > > >> > On Thu, 21 Mar 2019 at 13:06, Jevgeni Zolotarjov <
> > > >> j.zolotarjov@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> > > I reinstalled cloudstack from scratch - everything
> > > >> > >
> > > >> > > But looks like I hit the same wall now
> > > >> > >
> > > >> > > In the last step of installation it cannot create system VMs.
> > > >> > >
> > > >> > > service libvirtd status -l
> > > >> > > gives me
> > > >> > > ------------------------------------
> > > >> > > ● libvirtd.service - Virtualization daemon
> > > >> > > Loaded: loaded
> > > >> > > (/usr/lib/systemd/system/libvirtd.service;
> > > enabled;
> > > >> > > vendor preset: enabled)
> > > >> > > Active: active (running) since Thu 2019-03-21 11:45:00
> > > >> > > GMT;
> > 18min
> > > >> ago
> > > >> > > Docs: man:libvirtd(8)
> > > >> > > https://libvirt.org Main PID: 537 (libvirtd)
> > > >> > > Tasks: 20 (limit: 32768)
> > > >> > > CGroup: /system.slice/libvirtd.service
> > > >> > > ├─ 537 /usr/sbin/libvirtd -l
> > > >> > > ├─12206 /usr/sbin/dnsmasq
> > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf
> > > >> > > --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > >> > > └─12207 /usr/sbin/dnsmasq
> > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf
> > > >> > > --leasefile-ro
> > > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > >> > >
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > 2019-03-21
> > > >> > > 11:45:01.168+0000: 566: info : libvirt version: 4.5.0, package:
> > > >> > 10.el7_6.6
> > > >> > > (CentOS BuildSystem <http://bugs.centos.org>,
> > 2019-03-14-10:21:47,
> > > >> > > x86-01.bsys.centos.org)
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > 2019-03-21
> > > >> > > 11:45:01.168+0000: 566: info : hostname:
> > > mtl1-apphst03.mt.pbt.com.mt
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > 2019-03-21
> > > >> > > 11:45:01.168+0000: 566: error :
> > > >> > > virFirewallApplyRuleDirect:709
> > > >> > > :
> > > >> internal
> > > >> > > error: Failed to apply firewall rules /usr/sbin/iptables -w
> > --table
> > > >> nat
> > > >> > > --insert POSTROUTING --source 192.168.122.0/24 '!'
> --destination
> > > >> > > 192.168.122.0/24 --jump MASQUERADE: iptables v1.4.21: can't
> > > >> initialize
> > > >> > > iptables table `nat': Table does not exist (do you need to
> > insmod?)
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > Perhaps
> > > >> > > iptables
> > > >> > > or your kernel needs to be upgraded.
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]:
> read
> > > >> > > /etc/hosts
> > > >> > > - 4 addresses
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]:
> read
> > > >> > > /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> dnsmasq-dhcp[12206]:
> > > read
> > > >> > > /var/lib/libvirt/dnsmasq/default.hostsfile
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > 2019-03-21
> > > >> > > 11:45:01.354+0000: 566: warning : virSecurityManagerNew:189 :
> > > >> Configured
> > > >> > > security driver "none" disables default policy to create
> confined
> > > >> guests
> > > >> > > Mar 21 11:49:57 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > 2019-03-21
> > > >> > > 11:49:57.354+0000: 542: warning : qemuDomainObjTaint:7521 :
> Domain
> > > >> id=2
> > > >> > > name='s-1-VM' uuid=1a06d3a7-4e3f-4cba-912f-74ae24569bac is
> > tainted:
> > > >> > > high-privileges
> > > >> > > Mar 21 11:49:59 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > 2019-03-21
> > > >> > > 11:49:59.402+0000: 540: warning : qemuDomainObjTaint:7521 :
> Domain
> > > >> id=3
> > > >> > > name='v-2-VM' uuid=af2a8342-cd9b-4b55-ba12-480634a31d65 is
> > tainted:
> > > >> > > high-privileges
> > > >> > >
> > > >> > >
> > > >> > > What can be done about that ?
> > > >> > >
> > > >> >
> > > >> >
> > > >> > --
> > > >> >
> > > >> > Andrija Panić
> > > >> >
> > > >>
> > > >
> > > >
> > > > --
> > > >
> > > > Andrija Panić
> > > >
> > >
> > >
> > > --
> > >
> > > Andrija Panić
> > >
> >
>
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks LLC
> Cell RU: +7-923-414-1515
> Cell USA: +1-201-257-1512
> WWW: http://bitworks.software/ <http://bw-sw.com/>
>
--
Andrija Panić
Re: cannot start system VMs: disaster after maintenance followup
Posted by Andrija Panic <an...@gmail.com>.
Jevgeni, qemu-kvm 1.5.3 is the lastest official one for CentoS 7.6.XXX
(latest) which I'm running atm in my lab (just checked for update) - how
did you manage to go to 2.0 (custom repo ?)
On Thu, 21 Mar 2019 at 18:13, Ivan Kudryavtsev <ku...@bw-sw.com>
wrote:
> Jevgeniy, simplest and the most obvious way is to flatten their images with
> "qemu-img convert", next import them as templates and recreate VMs from
> those templates.
>
> чт, 21 мар. 2019 г. в 13:05, Jevgeni Zolotarjov <j....@gmail.com>:
>
> > What happened in the end was: qemu-kvm got updated to version 2.0 during
> > the maintenance. We could not manage to make this KVM to work with
> > Cloudstack.
> > So we rolled back to version 1.5.3.
> >
> > And now we have clean cloudstack fully operational. We can create new VMs
> > and it works. I am almost happy.
> >
> > Now question - how do I get my old VMs to work, considering I have only
> > their volumes?
> >
> > On Thu, Mar 21, 2019 at 6:24 PM Andrija Panic <an...@gmail.com>
> > wrote:
> >
> > > Just replace the URL for systemVM template from 4.11.1 with 4.11.2
> (there
> > > is a PR for this now).
> > >
> > > On Thu, 21 Mar 2019 at 16:53, Andrija Panic <an...@gmail.com>
> > > wrote:
> > >
> > > > Please use the one, updated specifically for CentOS 7 -
> > > >
> > >
> >
> https://github.com/apache/cloudstack-documentation/blob/master/source/quickinstallationguide/qig.rst
> > > >
> > > > And please avoid collocating KVM and MGMT on same server (especially
> in
> > > > any production-like system)
> > > >
> > > > Please let me know if the guide above gives you problem - we had
> > multiple
> > > > users explicitly following it - and successfully installed (with some
> > > minor
> > > > modification, which we committed back to that guide).
> > > >
> > > > Thanks
> > > > Andrija
> > > >
> > > > On Thu, 21 Mar 2019 at 16:34, Jevgeni Zolotarjov <
> > j.zolotarjov@gmail.com
> > > >
> > > > wrote:
> > > >
> > > >> OS management - centos 7 (1810)
> > > >> OS hypervisor - centos 7 (1810)
> > > >>
> > > >> Basic zone - yes
> > > >> I am following this quide
> > > >>
> > > >>
> > >
> >
> http://docs.cloudstack.apache.org/en/4.11.2.0/quickinstallationguide/qig.html
> > > >>
> > > >> Right now from scratch - management ans hypervisor on the same
> machine
> > > >> qemu - version 1.5.3
> > > >> libvirt - libvirt version: 4.5.0, package: 10.el7_6.6
> > > >>
> > > >> Basically - everything out of the box of clean centos install
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Thu, Mar 21, 2019 at 5:08 PM Andrija Panic <
> > andrija.panic@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Hey Jevgeni,
> > > >> >
> > > >> > what OS mgmt, what OS hypervisor, what qemu/libvirt versions -
> still
> > > in
> > > >> > Basic Zone, SG ?
> > > >> >
> > > >> > Andrija
> > > >> >
> > > >> > On Thu, 21 Mar 2019 at 13:06, Jevgeni Zolotarjov <
> > > >> j.zolotarjov@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> > > I reinstalled cloudstack from scratch - everything
> > > >> > >
> > > >> > > But looks like I hit the same wall now
> > > >> > >
> > > >> > > In the last step of installation it cannot create system VMs.
> > > >> > >
> > > >> > > service libvirtd status -l
> > > >> > > gives me
> > > >> > > ------------------------------------
> > > >> > > ● libvirtd.service - Virtualization daemon
> > > >> > > Loaded: loaded (/usr/lib/systemd/system/libvirtd.service;
> > > enabled;
> > > >> > > vendor preset: enabled)
> > > >> > > Active: active (running) since Thu 2019-03-21 11:45:00 GMT;
> > 18min
> > > >> ago
> > > >> > > Docs: man:libvirtd(8)
> > > >> > > https://libvirt.org
> > > >> > > Main PID: 537 (libvirtd)
> > > >> > > Tasks: 20 (limit: 32768)
> > > >> > > CGroup: /system.slice/libvirtd.service
> > > >> > > ├─ 537 /usr/sbin/libvirtd -l
> > > >> > > ├─12206 /usr/sbin/dnsmasq
> > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> > > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > >> > > └─12207 /usr/sbin/dnsmasq
> > > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> > > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > >> > >
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > 2019-03-21
> > > >> > > 11:45:01.168+0000: 566: info : libvirt version: 4.5.0, package:
> > > >> > 10.el7_6.6
> > > >> > > (CentOS BuildSystem <http://bugs.centos.org>,
> > 2019-03-14-10:21:47,
> > > >> > > x86-01.bsys.centos.org)
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > 2019-03-21
> > > >> > > 11:45:01.168+0000: 566: info : hostname:
> > > mtl1-apphst03.mt.pbt.com.mt
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > 2019-03-21
> > > >> > > 11:45:01.168+0000: 566: error : virFirewallApplyRuleDirect:709 :
> > > >> internal
> > > >> > > error: Failed to apply firewall rules /usr/sbin/iptables -w
> > --table
> > > >> nat
> > > >> > > --insert POSTROUTING --source 192.168.122.0/24 '!'
> --destination
> > > >> > > 192.168.122.0/24 --jump MASQUERADE: iptables v1.4.21: can't
> > > >> initialize
> > > >> > > iptables table `nat': Table does not exist (do you need to
> > insmod?)
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > Perhaps
> > > >> > > iptables
> > > >> > > or your kernel needs to be upgraded.
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]:
> read
> > > >> > > /etc/hosts
> > > >> > > - 4 addresses
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]:
> read
> > > >> > > /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt
> dnsmasq-dhcp[12206]:
> > > read
> > > >> > > /var/lib/libvirt/dnsmasq/default.hostsfile
> > > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > 2019-03-21
> > > >> > > 11:45:01.354+0000: 566: warning : virSecurityManagerNew:189 :
> > > >> Configured
> > > >> > > security driver "none" disables default policy to create
> confined
> > > >> guests
> > > >> > > Mar 21 11:49:57 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > 2019-03-21
> > > >> > > 11:49:57.354+0000: 542: warning : qemuDomainObjTaint:7521 :
> Domain
> > > >> id=2
> > > >> > > name='s-1-VM' uuid=1a06d3a7-4e3f-4cba-912f-74ae24569bac is
> > tainted:
> > > >> > > high-privileges
> > > >> > > Mar 21 11:49:59 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > > 2019-03-21
> > > >> > > 11:49:59.402+0000: 540: warning : qemuDomainObjTaint:7521 :
> Domain
> > > >> id=3
> > > >> > > name='v-2-VM' uuid=af2a8342-cd9b-4b55-ba12-480634a31d65 is
> > tainted:
> > > >> > > high-privileges
> > > >> > >
> > > >> > >
> > > >> > > What can be done about that ?
> > > >> > >
> > > >> >
> > > >> >
> > > >> > --
> > > >> >
> > > >> > Andrija Panić
> > > >> >
> > > >>
> > > >
> > > >
> > > > --
> > > >
> > > > Andrija Panić
> > > >
> > >
> > >
> > > --
> > >
> > > Andrija Panić
> > >
> >
>
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks LLC
> Cell RU: +7-923-414-1515
> Cell USA: +1-201-257-1512
> WWW: http://bitworks.software/ <http://bw-sw.com/>
>
--
Andrija Panić
Re: cannot start system VMs: disaster after maintenance followup
Posted by Ivan Kudryavtsev <ku...@bw-sw.com>.
Jevgeniy, simplest and the most obvious way is to flatten their images with
"qemu-img convert", next import them as templates and recreate VMs from
those templates.
чт, 21 мар. 2019 г. в 13:05, Jevgeni Zolotarjov <j....@gmail.com>:
> What happened in the end was: qemu-kvm got updated to version 2.0 during
> the maintenance. We could not manage to make this KVM to work with
> Cloudstack.
> So we rolled back to version 1.5.3.
>
> And now we have clean cloudstack fully operational. We can create new VMs
> and it works. I am almost happy.
>
> Now question - how do I get my old VMs to work, considering I have only
> their volumes?
>
> On Thu, Mar 21, 2019 at 6:24 PM Andrija Panic <an...@gmail.com>
> wrote:
>
> > Just replace the URL for systemVM template from 4.11.1 with 4.11.2 (there
> > is a PR for this now).
> >
> > On Thu, 21 Mar 2019 at 16:53, Andrija Panic <an...@gmail.com>
> > wrote:
> >
> > > Please use the one, updated specifically for CentOS 7 -
> > >
> >
> https://github.com/apache/cloudstack-documentation/blob/master/source/quickinstallationguide/qig.rst
> > >
> > > And please avoid collocating KVM and MGMT on same server (especially in
> > > any production-like system)
> > >
> > > Please let me know if the guide above gives you problem - we had
> multiple
> > > users explicitly following it - and successfully installed (with some
> > minor
> > > modification, which we committed back to that guide).
> > >
> > > Thanks
> > > Andrija
> > >
> > > On Thu, 21 Mar 2019 at 16:34, Jevgeni Zolotarjov <
> j.zolotarjov@gmail.com
> > >
> > > wrote:
> > >
> > >> OS management - centos 7 (1810)
> > >> OS hypervisor - centos 7 (1810)
> > >>
> > >> Basic zone - yes
> > >> I am following this quide
> > >>
> > >>
> >
> http://docs.cloudstack.apache.org/en/4.11.2.0/quickinstallationguide/qig.html
> > >>
> > >> Right now from scratch - management ans hypervisor on the same machine
> > >> qemu - version 1.5.3
> > >> libvirt - libvirt version: 4.5.0, package: 10.el7_6.6
> > >>
> > >> Basically - everything out of the box of clean centos install
> > >>
> > >>
> > >>
> > >>
> > >> On Thu, Mar 21, 2019 at 5:08 PM Andrija Panic <
> andrija.panic@gmail.com>
> > >> wrote:
> > >>
> > >> > Hey Jevgeni,
> > >> >
> > >> > what OS mgmt, what OS hypervisor, what qemu/libvirt versions - still
> > in
> > >> > Basic Zone, SG ?
> > >> >
> > >> > Andrija
> > >> >
> > >> > On Thu, 21 Mar 2019 at 13:06, Jevgeni Zolotarjov <
> > >> j.zolotarjov@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > I reinstalled cloudstack from scratch - everything
> > >> > >
> > >> > > But looks like I hit the same wall now
> > >> > >
> > >> > > In the last step of installation it cannot create system VMs.
> > >> > >
> > >> > > service libvirtd status -l
> > >> > > gives me
> > >> > > ------------------------------------
> > >> > > ● libvirtd.service - Virtualization daemon
> > >> > > Loaded: loaded (/usr/lib/systemd/system/libvirtd.service;
> > enabled;
> > >> > > vendor preset: enabled)
> > >> > > Active: active (running) since Thu 2019-03-21 11:45:00 GMT;
> 18min
> > >> ago
> > >> > > Docs: man:libvirtd(8)
> > >> > > https://libvirt.org
> > >> > > Main PID: 537 (libvirtd)
> > >> > > Tasks: 20 (limit: 32768)
> > >> > > CGroup: /system.slice/libvirtd.service
> > >> > > ├─ 537 /usr/sbin/libvirtd -l
> > >> > > ├─12206 /usr/sbin/dnsmasq
> > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > >> > > └─12207 /usr/sbin/dnsmasq
> > >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> > >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > >> > >
> > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > 2019-03-21
> > >> > > 11:45:01.168+0000: 566: info : libvirt version: 4.5.0, package:
> > >> > 10.el7_6.6
> > >> > > (CentOS BuildSystem <http://bugs.centos.org>,
> 2019-03-14-10:21:47,
> > >> > > x86-01.bsys.centos.org)
> > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > 2019-03-21
> > >> > > 11:45:01.168+0000: 566: info : hostname:
> > mtl1-apphst03.mt.pbt.com.mt
> > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > 2019-03-21
> > >> > > 11:45:01.168+0000: 566: error : virFirewallApplyRuleDirect:709 :
> > >> internal
> > >> > > error: Failed to apply firewall rules /usr/sbin/iptables -w
> --table
> > >> nat
> > >> > > --insert POSTROUTING --source 192.168.122.0/24 '!' --destination
> > >> > > 192.168.122.0/24 --jump MASQUERADE: iptables v1.4.21: can't
> > >> initialize
> > >> > > iptables table `nat': Table does not exist (do you need to
> insmod?)
> > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> Perhaps
> > >> > > iptables
> > >> > > or your kernel needs to be upgraded.
> > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]: read
> > >> > > /etc/hosts
> > >> > > - 4 addresses
> > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]: read
> > >> > > /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
> > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq-dhcp[12206]:
> > read
> > >> > > /var/lib/libvirt/dnsmasq/default.hostsfile
> > >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > 2019-03-21
> > >> > > 11:45:01.354+0000: 566: warning : virSecurityManagerNew:189 :
> > >> Configured
> > >> > > security driver "none" disables default policy to create confined
> > >> guests
> > >> > > Mar 21 11:49:57 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > 2019-03-21
> > >> > > 11:49:57.354+0000: 542: warning : qemuDomainObjTaint:7521 : Domain
> > >> id=2
> > >> > > name='s-1-VM' uuid=1a06d3a7-4e3f-4cba-912f-74ae24569bac is
> tainted:
> > >> > > high-privileges
> > >> > > Mar 21 11:49:59 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> > 2019-03-21
> > >> > > 11:49:59.402+0000: 540: warning : qemuDomainObjTaint:7521 : Domain
> > >> id=3
> > >> > > name='v-2-VM' uuid=af2a8342-cd9b-4b55-ba12-480634a31d65 is
> tainted:
> > >> > > high-privileges
> > >> > >
> > >> > >
> > >> > > What can be done about that ?
> > >> > >
> > >> >
> > >> >
> > >> > --
> > >> >
> > >> > Andrija Panić
> > >> >
> > >>
> > >
> > >
> > > --
> > >
> > > Andrija Panić
> > >
> >
> >
> > --
> >
> > Andrija Panić
> >
>
--
With best regards, Ivan Kudryavtsev
Bitworks LLC
Cell RU: +7-923-414-1515
Cell USA: +1-201-257-1512
WWW: http://bitworks.software/ <http://bw-sw.com/>
Re: cannot start system VMs: disaster after maintenance followup
Posted by Jevgeni Zolotarjov <j....@gmail.com>.
What happened in the end was: qemu-kvm got updated to version 2.0 during
the maintenance. We could not manage to make this KVM to work with
Cloudstack.
So we rolled back to version 1.5.3.
And now we have clean cloudstack fully operational. We can create new VMs
and it works. I am almost happy.
Now question - how do I get my old VMs to work, considering I have only
their volumes?
On Thu, Mar 21, 2019 at 6:24 PM Andrija Panic <an...@gmail.com>
wrote:
> Just replace the URL for systemVM template from 4.11.1 with 4.11.2 (there
> is a PR for this now).
>
> On Thu, 21 Mar 2019 at 16:53, Andrija Panic <an...@gmail.com>
> wrote:
>
> > Please use the one, updated specifically for CentOS 7 -
> >
> https://github.com/apache/cloudstack-documentation/blob/master/source/quickinstallationguide/qig.rst
> >
> > And please avoid collocating KVM and MGMT on same server (especially in
> > any production-like system)
> >
> > Please let me know if the guide above gives you problem - we had multiple
> > users explicitly following it - and successfully installed (with some
> minor
> > modification, which we committed back to that guide).
> >
> > Thanks
> > Andrija
> >
> > On Thu, 21 Mar 2019 at 16:34, Jevgeni Zolotarjov <j.zolotarjov@gmail.com
> >
> > wrote:
> >
> >> OS management - centos 7 (1810)
> >> OS hypervisor - centos 7 (1810)
> >>
> >> Basic zone - yes
> >> I am following this quide
> >>
> >>
> http://docs.cloudstack.apache.org/en/4.11.2.0/quickinstallationguide/qig.html
> >>
> >> Right now from scratch - management ans hypervisor on the same machine
> >> qemu - version 1.5.3
> >> libvirt - libvirt version: 4.5.0, package: 10.el7_6.6
> >>
> >> Basically - everything out of the box of clean centos install
> >>
> >>
> >>
> >>
> >> On Thu, Mar 21, 2019 at 5:08 PM Andrija Panic <an...@gmail.com>
> >> wrote:
> >>
> >> > Hey Jevgeni,
> >> >
> >> > what OS mgmt, what OS hypervisor, what qemu/libvirt versions - still
> in
> >> > Basic Zone, SG ?
> >> >
> >> > Andrija
> >> >
> >> > On Thu, 21 Mar 2019 at 13:06, Jevgeni Zolotarjov <
> >> j.zolotarjov@gmail.com>
> >> > wrote:
> >> >
> >> > > I reinstalled cloudstack from scratch - everything
> >> > >
> >> > > But looks like I hit the same wall now
> >> > >
> >> > > In the last step of installation it cannot create system VMs.
> >> > >
> >> > > service libvirtd status -l
> >> > > gives me
> >> > > ------------------------------------
> >> > > ● libvirtd.service - Virtualization daemon
> >> > > Loaded: loaded (/usr/lib/systemd/system/libvirtd.service;
> enabled;
> >> > > vendor preset: enabled)
> >> > > Active: active (running) since Thu 2019-03-21 11:45:00 GMT; 18min
> >> ago
> >> > > Docs: man:libvirtd(8)
> >> > > https://libvirt.org
> >> > > Main PID: 537 (libvirtd)
> >> > > Tasks: 20 (limit: 32768)
> >> > > CGroup: /system.slice/libvirtd.service
> >> > > ├─ 537 /usr/sbin/libvirtd -l
> >> > > ├─12206 /usr/sbin/dnsmasq
> >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> >> > > └─12207 /usr/sbin/dnsmasq
> >> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> >> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> >> > >
> >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> 2019-03-21
> >> > > 11:45:01.168+0000: 566: info : libvirt version: 4.5.0, package:
> >> > 10.el7_6.6
> >> > > (CentOS BuildSystem <http://bugs.centos.org>, 2019-03-14-10:21:47,
> >> > > x86-01.bsys.centos.org)
> >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> 2019-03-21
> >> > > 11:45:01.168+0000: 566: info : hostname:
> mtl1-apphst03.mt.pbt.com.mt
> >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> 2019-03-21
> >> > > 11:45:01.168+0000: 566: error : virFirewallApplyRuleDirect:709 :
> >> internal
> >> > > error: Failed to apply firewall rules /usr/sbin/iptables -w --table
> >> nat
> >> > > --insert POSTROUTING --source 192.168.122.0/24 '!' --destination
> >> > > 192.168.122.0/24 --jump MASQUERADE: iptables v1.4.21: can't
> >> initialize
> >> > > iptables table `nat': Table does not exist (do you need to insmod?)
> >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: Perhaps
> >> > > iptables
> >> > > or your kernel needs to be upgraded.
> >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]: read
> >> > > /etc/hosts
> >> > > - 4 addresses
> >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]: read
> >> > > /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
> >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq-dhcp[12206]:
> read
> >> > > /var/lib/libvirt/dnsmasq/default.hostsfile
> >> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> 2019-03-21
> >> > > 11:45:01.354+0000: 566: warning : virSecurityManagerNew:189 :
> >> Configured
> >> > > security driver "none" disables default policy to create confined
> >> guests
> >> > > Mar 21 11:49:57 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> 2019-03-21
> >> > > 11:49:57.354+0000: 542: warning : qemuDomainObjTaint:7521 : Domain
> >> id=2
> >> > > name='s-1-VM' uuid=1a06d3a7-4e3f-4cba-912f-74ae24569bac is tainted:
> >> > > high-privileges
> >> > > Mar 21 11:49:59 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]:
> 2019-03-21
> >> > > 11:49:59.402+0000: 540: warning : qemuDomainObjTaint:7521 : Domain
> >> id=3
> >> > > name='v-2-VM' uuid=af2a8342-cd9b-4b55-ba12-480634a31d65 is tainted:
> >> > > high-privileges
> >> > >
> >> > >
> >> > > What can be done about that ?
> >> > >
> >> >
> >> >
> >> > --
> >> >
> >> > Andrija Panić
> >> >
> >>
> >
> >
> > --
> >
> > Andrija Panić
> >
>
>
> --
>
> Andrija Panić
>
Re: cannot start system VMs: disaster after maintenance followup
Posted by Andrija Panic <an...@gmail.com>.
Just replace the URL for systemVM template from 4.11.1 with 4.11.2 (there
is a PR for this now).
On Thu, 21 Mar 2019 at 16:53, Andrija Panic <an...@gmail.com> wrote:
> Please use the one, updated specifically for CentOS 7 -
> https://github.com/apache/cloudstack-documentation/blob/master/source/quickinstallationguide/qig.rst
>
> And please avoid collocating KVM and MGMT on same server (especially in
> any production-like system)
>
> Please let me know if the guide above gives you problem - we had multiple
> users explicitly following it - and successfully installed (with some minor
> modification, which we committed back to that guide).
>
> Thanks
> Andrija
>
> On Thu, 21 Mar 2019 at 16:34, Jevgeni Zolotarjov <j....@gmail.com>
> wrote:
>
>> OS management - centos 7 (1810)
>> OS hypervisor - centos 7 (1810)
>>
>> Basic zone - yes
>> I am following this quide
>>
>> http://docs.cloudstack.apache.org/en/4.11.2.0/quickinstallationguide/qig.html
>>
>> Right now from scratch - management ans hypervisor on the same machine
>> qemu - version 1.5.3
>> libvirt - libvirt version: 4.5.0, package: 10.el7_6.6
>>
>> Basically - everything out of the box of clean centos install
>>
>>
>>
>>
>> On Thu, Mar 21, 2019 at 5:08 PM Andrija Panic <an...@gmail.com>
>> wrote:
>>
>> > Hey Jevgeni,
>> >
>> > what OS mgmt, what OS hypervisor, what qemu/libvirt versions - still in
>> > Basic Zone, SG ?
>> >
>> > Andrija
>> >
>> > On Thu, 21 Mar 2019 at 13:06, Jevgeni Zolotarjov <
>> j.zolotarjov@gmail.com>
>> > wrote:
>> >
>> > > I reinstalled cloudstack from scratch - everything
>> > >
>> > > But looks like I hit the same wall now
>> > >
>> > > In the last step of installation it cannot create system VMs.
>> > >
>> > > service libvirtd status -l
>> > > gives me
>> > > ------------------------------------
>> > > ● libvirtd.service - Virtualization daemon
>> > > Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled;
>> > > vendor preset: enabled)
>> > > Active: active (running) since Thu 2019-03-21 11:45:00 GMT; 18min
>> ago
>> > > Docs: man:libvirtd(8)
>> > > https://libvirt.org
>> > > Main PID: 537 (libvirtd)
>> > > Tasks: 20 (limit: 32768)
>> > > CGroup: /system.slice/libvirtd.service
>> > > ├─ 537 /usr/sbin/libvirtd -l
>> > > ├─12206 /usr/sbin/dnsmasq
>> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
>> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
>> > > └─12207 /usr/sbin/dnsmasq
>> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
>> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
>> > >
>> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
>> > > 11:45:01.168+0000: 566: info : libvirt version: 4.5.0, package:
>> > 10.el7_6.6
>> > > (CentOS BuildSystem <http://bugs.centos.org>, 2019-03-14-10:21:47,
>> > > x86-01.bsys.centos.org)
>> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
>> > > 11:45:01.168+0000: 566: info : hostname: mtl1-apphst03.mt.pbt.com.mt
>> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
>> > > 11:45:01.168+0000: 566: error : virFirewallApplyRuleDirect:709 :
>> internal
>> > > error: Failed to apply firewall rules /usr/sbin/iptables -w --table
>> nat
>> > > --insert POSTROUTING --source 192.168.122.0/24 '!' --destination
>> > > 192.168.122.0/24 --jump MASQUERADE: iptables v1.4.21: can't
>> initialize
>> > > iptables table `nat': Table does not exist (do you need to insmod?)
>> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: Perhaps
>> > > iptables
>> > > or your kernel needs to be upgraded.
>> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]: read
>> > > /etc/hosts
>> > > - 4 addresses
>> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]: read
>> > > /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
>> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq-dhcp[12206]: read
>> > > /var/lib/libvirt/dnsmasq/default.hostsfile
>> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
>> > > 11:45:01.354+0000: 566: warning : virSecurityManagerNew:189 :
>> Configured
>> > > security driver "none" disables default policy to create confined
>> guests
>> > > Mar 21 11:49:57 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
>> > > 11:49:57.354+0000: 542: warning : qemuDomainObjTaint:7521 : Domain
>> id=2
>> > > name='s-1-VM' uuid=1a06d3a7-4e3f-4cba-912f-74ae24569bac is tainted:
>> > > high-privileges
>> > > Mar 21 11:49:59 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
>> > > 11:49:59.402+0000: 540: warning : qemuDomainObjTaint:7521 : Domain
>> id=3
>> > > name='v-2-VM' uuid=af2a8342-cd9b-4b55-ba12-480634a31d65 is tainted:
>> > > high-privileges
>> > >
>> > >
>> > > What can be done about that ?
>> > >
>> >
>> >
>> > --
>> >
>> > Andrija Panić
>> >
>>
>
>
> --
>
> Andrija Panić
>
--
Andrija Panić
Re: cannot start system VMs: disaster after maintenance followup
Posted by Andrija Panic <an...@gmail.com>.
Please use the one, updated specifically for CentOS 7 -
https://github.com/apache/cloudstack-documentation/blob/master/source/quickinstallationguide/qig.rst
And please avoid collocating KVM and MGMT on same server (especially in any
production-like system)
Please let me know if the guide above gives you problem - we had multiple
users explicitly following it - and successfully installed (with some minor
modification, which we committed back to that guide).
Thanks
Andrija
On Thu, 21 Mar 2019 at 16:34, Jevgeni Zolotarjov <j....@gmail.com>
wrote:
> OS management - centos 7 (1810)
> OS hypervisor - centos 7 (1810)
>
> Basic zone - yes
> I am following this quide
>
> http://docs.cloudstack.apache.org/en/4.11.2.0/quickinstallationguide/qig.html
>
> Right now from scratch - management ans hypervisor on the same machine
> qemu - version 1.5.3
> libvirt - libvirt version: 4.5.0, package: 10.el7_6.6
>
> Basically - everything out of the box of clean centos install
>
>
>
>
> On Thu, Mar 21, 2019 at 5:08 PM Andrija Panic <an...@gmail.com>
> wrote:
>
> > Hey Jevgeni,
> >
> > what OS mgmt, what OS hypervisor, what qemu/libvirt versions - still in
> > Basic Zone, SG ?
> >
> > Andrija
> >
> > On Thu, 21 Mar 2019 at 13:06, Jevgeni Zolotarjov <j.zolotarjov@gmail.com
> >
> > wrote:
> >
> > > I reinstalled cloudstack from scratch - everything
> > >
> > > But looks like I hit the same wall now
> > >
> > > In the last step of installation it cannot create system VMs.
> > >
> > > service libvirtd status -l
> > > gives me
> > > ------------------------------------
> > > ● libvirtd.service - Virtualization daemon
> > > Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled;
> > > vendor preset: enabled)
> > > Active: active (running) since Thu 2019-03-21 11:45:00 GMT; 18min
> ago
> > > Docs: man:libvirtd(8)
> > > https://libvirt.org
> > > Main PID: 537 (libvirtd)
> > > Tasks: 20 (limit: 32768)
> > > CGroup: /system.slice/libvirtd.service
> > > ├─ 537 /usr/sbin/libvirtd -l
> > > ├─12206 /usr/sbin/dnsmasq
> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > > └─12207 /usr/sbin/dnsmasq
> > > --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> > > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > >
> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> > > 11:45:01.168+0000: 566: info : libvirt version: 4.5.0, package:
> > 10.el7_6.6
> > > (CentOS BuildSystem <http://bugs.centos.org>, 2019-03-14-10:21:47,
> > > x86-01.bsys.centos.org)
> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> > > 11:45:01.168+0000: 566: info : hostname: mtl1-apphst03.mt.pbt.com.mt
> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> > > 11:45:01.168+0000: 566: error : virFirewallApplyRuleDirect:709 :
> internal
> > > error: Failed to apply firewall rules /usr/sbin/iptables -w --table nat
> > > --insert POSTROUTING --source 192.168.122.0/24 '!' --destination
> > > 192.168.122.0/24 --jump MASQUERADE: iptables v1.4.21: can't initialize
> > > iptables table `nat': Table does not exist (do you need to insmod?)
> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: Perhaps
> > > iptables
> > > or your kernel needs to be upgraded.
> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]: read
> > > /etc/hosts
> > > - 4 addresses
> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]: read
> > > /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq-dhcp[12206]: read
> > > /var/lib/libvirt/dnsmasq/default.hostsfile
> > > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> > > 11:45:01.354+0000: 566: warning : virSecurityManagerNew:189 :
> Configured
> > > security driver "none" disables default policy to create confined
> guests
> > > Mar 21 11:49:57 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> > > 11:49:57.354+0000: 542: warning : qemuDomainObjTaint:7521 : Domain id=2
> > > name='s-1-VM' uuid=1a06d3a7-4e3f-4cba-912f-74ae24569bac is tainted:
> > > high-privileges
> > > Mar 21 11:49:59 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> > > 11:49:59.402+0000: 540: warning : qemuDomainObjTaint:7521 : Domain id=3
> > > name='v-2-VM' uuid=af2a8342-cd9b-4b55-ba12-480634a31d65 is tainted:
> > > high-privileges
> > >
> > >
> > > What can be done about that ?
> > >
> >
> >
> > --
> >
> > Andrija Panić
> >
>
--
Andrija Panić
Re: cannot start system VMs: disaster after maintenance followup
Posted by Jevgeni Zolotarjov <j....@gmail.com>.
OS management - centos 7 (1810)
OS hypervisor - centos 7 (1810)
Basic zone - yes
I am following this quide
http://docs.cloudstack.apache.org/en/4.11.2.0/quickinstallationguide/qig.html
Right now from scratch - management ans hypervisor on the same machine
qemu - version 1.5.3
libvirt - libvirt version: 4.5.0, package: 10.el7_6.6
Basically - everything out of the box of clean centos install
On Thu, Mar 21, 2019 at 5:08 PM Andrija Panic <an...@gmail.com>
wrote:
> Hey Jevgeni,
>
> what OS mgmt, what OS hypervisor, what qemu/libvirt versions - still in
> Basic Zone, SG ?
>
> Andrija
>
> On Thu, 21 Mar 2019 at 13:06, Jevgeni Zolotarjov <j....@gmail.com>
> wrote:
>
> > I reinstalled cloudstack from scratch - everything
> >
> > But looks like I hit the same wall now
> >
> > In the last step of installation it cannot create system VMs.
> >
> > service libvirtd status -l
> > gives me
> > ------------------------------------
> > ● libvirtd.service - Virtualization daemon
> > Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled;
> > vendor preset: enabled)
> > Active: active (running) since Thu 2019-03-21 11:45:00 GMT; 18min ago
> > Docs: man:libvirtd(8)
> > https://libvirt.org
> > Main PID: 537 (libvirtd)
> > Tasks: 20 (limit: 32768)
> > CGroup: /system.slice/libvirtd.service
> > ├─ 537 /usr/sbin/libvirtd -l
> > ├─12206 /usr/sbin/dnsmasq
> > --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> > └─12207 /usr/sbin/dnsmasq
> > --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> > --dhcp-script=/usr/libexec/libvirt_leaseshelper
> >
> > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> > 11:45:01.168+0000: 566: info : libvirt version: 4.5.0, package:
> 10.el7_6.6
> > (CentOS BuildSystem <http://bugs.centos.org>, 2019-03-14-10:21:47,
> > x86-01.bsys.centos.org)
> > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> > 11:45:01.168+0000: 566: info : hostname: mtl1-apphst03.mt.pbt.com.mt
> > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> > 11:45:01.168+0000: 566: error : virFirewallApplyRuleDirect:709 : internal
> > error: Failed to apply firewall rules /usr/sbin/iptables -w --table nat
> > --insert POSTROUTING --source 192.168.122.0/24 '!' --destination
> > 192.168.122.0/24 --jump MASQUERADE: iptables v1.4.21: can't initialize
> > iptables table `nat': Table does not exist (do you need to insmod?)
> > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: Perhaps
> > iptables
> > or your kernel needs to be upgraded.
> > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]: read
> > /etc/hosts
> > - 4 addresses
> > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]: read
> > /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
> > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq-dhcp[12206]: read
> > /var/lib/libvirt/dnsmasq/default.hostsfile
> > Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> > 11:45:01.354+0000: 566: warning : virSecurityManagerNew:189 : Configured
> > security driver "none" disables default policy to create confined guests
> > Mar 21 11:49:57 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> > 11:49:57.354+0000: 542: warning : qemuDomainObjTaint:7521 : Domain id=2
> > name='s-1-VM' uuid=1a06d3a7-4e3f-4cba-912f-74ae24569bac is tainted:
> > high-privileges
> > Mar 21 11:49:59 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> > 11:49:59.402+0000: 540: warning : qemuDomainObjTaint:7521 : Domain id=3
> > name='v-2-VM' uuid=af2a8342-cd9b-4b55-ba12-480634a31d65 is tainted:
> > high-privileges
> >
> >
> > What can be done about that ?
> >
>
>
> --
>
> Andrija Panić
>
Re: cannot start system VMs: disaster after maintenance followup
Posted by Andrija Panic <an...@gmail.com>.
Hey Jevgeni,
what OS mgmt, what OS hypervisor, what qemu/libvirt versions - still in
Basic Zone, SG ?
Andrija
On Thu, 21 Mar 2019 at 13:06, Jevgeni Zolotarjov <j....@gmail.com>
wrote:
> I reinstalled cloudstack from scratch - everything
>
> But looks like I hit the same wall now
>
> In the last step of installation it cannot create system VMs.
>
> service libvirtd status -l
> gives me
> ------------------------------------
> ● libvirtd.service - Virtualization daemon
> Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled;
> vendor preset: enabled)
> Active: active (running) since Thu 2019-03-21 11:45:00 GMT; 18min ago
> Docs: man:libvirtd(8)
> https://libvirt.org
> Main PID: 537 (libvirtd)
> Tasks: 20 (limit: 32768)
> CGroup: /system.slice/libvirtd.service
> ├─ 537 /usr/sbin/libvirtd -l
> ├─12206 /usr/sbin/dnsmasq
> --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> --dhcp-script=/usr/libexec/libvirt_leaseshelper
> └─12207 /usr/sbin/dnsmasq
> --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro
> --dhcp-script=/usr/libexec/libvirt_leaseshelper
>
> Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> 11:45:01.168+0000: 566: info : libvirt version: 4.5.0, package: 10.el7_6.6
> (CentOS BuildSystem <http://bugs.centos.org>, 2019-03-14-10:21:47,
> x86-01.bsys.centos.org)
> Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> 11:45:01.168+0000: 566: info : hostname: mtl1-apphst03.mt.pbt.com.mt
> Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> 11:45:01.168+0000: 566: error : virFirewallApplyRuleDirect:709 : internal
> error: Failed to apply firewall rules /usr/sbin/iptables -w --table nat
> --insert POSTROUTING --source 192.168.122.0/24 '!' --destination
> 192.168.122.0/24 --jump MASQUERADE: iptables v1.4.21: can't initialize
> iptables table `nat': Table does not exist (do you need to insmod?)
> Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: Perhaps
> iptables
> or your kernel needs to be upgraded.
> Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]: read
> /etc/hosts
> - 4 addresses
> Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq[12206]: read
> /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
> Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt dnsmasq-dhcp[12206]: read
> /var/lib/libvirt/dnsmasq/default.hostsfile
> Mar 21 11:45:01 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> 11:45:01.354+0000: 566: warning : virSecurityManagerNew:189 : Configured
> security driver "none" disables default policy to create confined guests
> Mar 21 11:49:57 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> 11:49:57.354+0000: 542: warning : qemuDomainObjTaint:7521 : Domain id=2
> name='s-1-VM' uuid=1a06d3a7-4e3f-4cba-912f-74ae24569bac is tainted:
> high-privileges
> Mar 21 11:49:59 mtl1-apphst03.mt.pbt.com.mt libvirtd[537]: 2019-03-21
> 11:49:59.402+0000: 540: warning : qemuDomainObjTaint:7521 : Domain id=3
> name='v-2-VM' uuid=af2a8342-cd9b-4b55-ba12-480634a31d65 is tainted:
> high-privileges
>
>
> What can be done about that ?
>
--
Andrija Panić