You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by cs user <ac...@gmail.com> on 2015/09/02 12:16:21 UTC
Fwd: Instance reboot issue - CS 4.5.1 - Xen 6.5
Forwarding to users channel in case anyone else has seen this...
Hi All,
We have seen this in 2 separate environments, both running the same
versions of Cloudstack and Xenserver. When we reboot an instance, we lose
access to it.
Looking at the iptables config on the xen host, we can see that the vif is
incremented for the bridged entries, but not updated for the rules.
For example, this is how the iptables look before a reboot:
[root@xen001 cloud]# iptables -L|grep 25075
i-2-25075-def all -- anywhere anywhere PHYSDEV
match --physdev-in vif108.0 --physdev-is-bridged
i-2-25075-def all -- anywhere anywhere PHYSDEV
match --physdev-out vif108.0 --physdev-is-bridged
Chain i-2-25075-VM (1 references)
Chain i-2-25075-VM-eg (1 references)
Chain i-2-25075-def (2 references)
RETURN udp -- anywhere anywhere PHYSDEV match
--physdev-in vif108.0 --physdev-is-bridged match-set i-2-25075-VM src udp
dpt:domain
DROP all -- anywhere anywhere PHYSDEV match
--physdev-in vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM src
DROP all -- anywhere anywhere PHYSDEV match
--physdev-out vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM dst
i-2-25075-VM-eg all -- anywhere anywhere PHYSDEV
match --physdev-in vif108.0 --physdev-is-bridged match-set i-2-25075-VM src
i-2-25075-VM all -- anywhere anywhere PHYSDEV
match --physdev-out vif108.0 --physdev-is-bridged
After a reboot, we can see the following:
[root@xen001 cloud]# iptables -L|grep 25075
i-2-25075-def all -- anywhere anywhere PHYSDEV
match --physdev-in vif109.0 --physdev-is-bridged
i-2-25075-def all -- anywhere anywhere PHYSDEV
match --physdev-out vif109.0 --physdev-is-bridged
Chain i-2-25075-VM (1 references)
Chain i-2-25075-VM-eg (1 references)
Chain i-2-25075-def (2 references)
RETURN udp -- anywhere anywhere PHYSDEV match
--physdev-in vif108.0 --physdev-is-bridged match-set i-2-25075-VM src udp
dpt:domain
DROP all -- anywhere anywhere PHYSDEV match
--physdev-in vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM src
DROP all -- anywhere anywhere PHYSDEV match
--physdev-out vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM dst
i-2-25075-VM-eg all -- anywhere anywhere PHYSDEV
match --physdev-in vif108.0 --physdev-is-bridged match-set i-2-25075-VM src
i-2-25075-VM all -- anywhere anywhere PHYSDEV
match --physdev-out vif108.0 --physdev-is-bridged
You can see that the bridged entries have been incremented to vif109, where
as the rules still reference vif108.
Stopping the instance appears to clear out the rules, and then everything
works fine again once the instance is started.
Is this a known issue? Is anyone able to replicate this?
Cheers!
Re: Instance reboot issue - CS 4.5.1 - Xen 6.5
Posted by cs user <ac...@gmail.com>.
Great, thanks !
On Thu, Sep 3, 2015 at 7:10 AM, Rajani Karuturi <ra...@apache.org> wrote:
> its in 4.5.2 and will also be in 4.6.0
>
> ~Rajani
>
> On Wed, Sep 2, 2015 at 5:33 PM, cs user <ac...@gmail.com> wrote:
>
> > Hi Folks,
> >
> > Looks like this is fixed in
> https://github.com/apache/cloudstack/pull/479
> > ?
> >
> > Which cloudstack release contains this fix?
> >
> > Many thanks!
> >
> > On Wed, Sep 2, 2015 at 11:16 AM, cs user <ac...@gmail.com> wrote:
> >
> > > Forwarding to users channel in case anyone else has seen this...
> > >
> > >
> > >
> > > Hi All,
> > >
> > > We have seen this in 2 separate environments, both running the same
> > > versions of Cloudstack and Xenserver. When we reboot an instance, we
> lose
> > > access to it.
> > >
> > > Looking at the iptables config on the xen host, we can see that the vif
> > is
> > > incremented for the bridged entries, but not updated for the rules.
> > >
> > > For example, this is how the iptables look before a reboot:
> > >
> > > [root@xen001 cloud]# iptables -L|grep 25075
> > > i-2-25075-def all -- anywhere anywhere
> PHYSDEV
> > > match --physdev-in vif108.0 --physdev-is-bridged
> > > i-2-25075-def all -- anywhere anywhere
> PHYSDEV
> > > match --physdev-out vif108.0 --physdev-is-bridged
> > > Chain i-2-25075-VM (1 references)
> > > Chain i-2-25075-VM-eg (1 references)
> > > Chain i-2-25075-def (2 references)
> > > RETURN udp -- anywhere anywhere PHYSDEV
> > > match --physdev-in vif108.0 --physdev-is-bridged match-set i-2-25075-VM
> > src
> > > udp dpt:domain
> > > DROP all -- anywhere anywhere PHYSDEV
> > > match --physdev-in vif108.0 --physdev-is-bridged ! match-set
> i-2-25075-VM
> > > src
> > > DROP all -- anywhere anywhere PHYSDEV
> > > match --physdev-out vif108.0 --physdev-is-bridged ! match-set
> > i-2-25075-VM
> > > dst
> > > i-2-25075-VM-eg all -- anywhere anywhere
> > > PHYSDEV match --physdev-in vif108.0 --physdev-is-bridged match-set
> > > i-2-25075-VM src
> > > i-2-25075-VM all -- anywhere anywhere
> PHYSDEV
> > > match --physdev-out vif108.0 --physdev-is-bridged
> > >
> > > After a reboot, we can see the following:
> > >
> > > [root@xen001 cloud]# iptables -L|grep 25075
> > > i-2-25075-def all -- anywhere anywhere
> PHYSDEV
> > > match --physdev-in vif109.0 --physdev-is-bridged
> > > i-2-25075-def all -- anywhere anywhere
> PHYSDEV
> > > match --physdev-out vif109.0 --physdev-is-bridged
> > > Chain i-2-25075-VM (1 references)
> > > Chain i-2-25075-VM-eg (1 references)
> > > Chain i-2-25075-def (2 references)
> > > RETURN udp -- anywhere anywhere PHYSDEV
> > > match --physdev-in vif108.0 --physdev-is-bridged match-set i-2-25075-VM
> > src
> > > udp dpt:domain
> > > DROP all -- anywhere anywhere PHYSDEV
> > > match --physdev-in vif108.0 --physdev-is-bridged ! match-set
> i-2-25075-VM
> > > src
> > > DROP all -- anywhere anywhere PHYSDEV
> > > match --physdev-out vif108.0 --physdev-is-bridged ! match-set
> > i-2-25075-VM
> > > dst
> > > i-2-25075-VM-eg all -- anywhere anywhere
> > > PHYSDEV match --physdev-in vif108.0 --physdev-is-bridged match-set
> > > i-2-25075-VM src
> > > i-2-25075-VM all -- anywhere anywhere
> PHYSDEV
> > > match --physdev-out vif108.0 --physdev-is-bridged
> > >
> > > You can see that the bridged entries have been incremented to vif109,
> > > where as the rules still reference vif108.
> > >
> > > Stopping the instance appears to clear out the rules, and then
> everything
> > > works fine again once the instance is started.
> > >
> > > Is this a known issue? Is anyone able to replicate this?
> > >
> > > Cheers!
> > >
> > >
> >
>
Re: Instance reboot issue - CS 4.5.1 - Xen 6.5
Posted by Rajani Karuturi <ra...@apache.org>.
its in 4.5.2 and will also be in 4.6.0
~Rajani
On Wed, Sep 2, 2015 at 5:33 PM, cs user <ac...@gmail.com> wrote:
> Hi Folks,
>
> Looks like this is fixed in https://github.com/apache/cloudstack/pull/479
> ?
>
> Which cloudstack release contains this fix?
>
> Many thanks!
>
> On Wed, Sep 2, 2015 at 11:16 AM, cs user <ac...@gmail.com> wrote:
>
> > Forwarding to users channel in case anyone else has seen this...
> >
> >
> >
> > Hi All,
> >
> > We have seen this in 2 separate environments, both running the same
> > versions of Cloudstack and Xenserver. When we reboot an instance, we lose
> > access to it.
> >
> > Looking at the iptables config on the xen host, we can see that the vif
> is
> > incremented for the bridged entries, but not updated for the rules.
> >
> > For example, this is how the iptables look before a reboot:
> >
> > [root@xen001 cloud]# iptables -L|grep 25075
> > i-2-25075-def all -- anywhere anywhere PHYSDEV
> > match --physdev-in vif108.0 --physdev-is-bridged
> > i-2-25075-def all -- anywhere anywhere PHYSDEV
> > match --physdev-out vif108.0 --physdev-is-bridged
> > Chain i-2-25075-VM (1 references)
> > Chain i-2-25075-VM-eg (1 references)
> > Chain i-2-25075-def (2 references)
> > RETURN udp -- anywhere anywhere PHYSDEV
> > match --physdev-in vif108.0 --physdev-is-bridged match-set i-2-25075-VM
> src
> > udp dpt:domain
> > DROP all -- anywhere anywhere PHYSDEV
> > match --physdev-in vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM
> > src
> > DROP all -- anywhere anywhere PHYSDEV
> > match --physdev-out vif108.0 --physdev-is-bridged ! match-set
> i-2-25075-VM
> > dst
> > i-2-25075-VM-eg all -- anywhere anywhere
> > PHYSDEV match --physdev-in vif108.0 --physdev-is-bridged match-set
> > i-2-25075-VM src
> > i-2-25075-VM all -- anywhere anywhere PHYSDEV
> > match --physdev-out vif108.0 --physdev-is-bridged
> >
> > After a reboot, we can see the following:
> >
> > [root@xen001 cloud]# iptables -L|grep 25075
> > i-2-25075-def all -- anywhere anywhere PHYSDEV
> > match --physdev-in vif109.0 --physdev-is-bridged
> > i-2-25075-def all -- anywhere anywhere PHYSDEV
> > match --physdev-out vif109.0 --physdev-is-bridged
> > Chain i-2-25075-VM (1 references)
> > Chain i-2-25075-VM-eg (1 references)
> > Chain i-2-25075-def (2 references)
> > RETURN udp -- anywhere anywhere PHYSDEV
> > match --physdev-in vif108.0 --physdev-is-bridged match-set i-2-25075-VM
> src
> > udp dpt:domain
> > DROP all -- anywhere anywhere PHYSDEV
> > match --physdev-in vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM
> > src
> > DROP all -- anywhere anywhere PHYSDEV
> > match --physdev-out vif108.0 --physdev-is-bridged ! match-set
> i-2-25075-VM
> > dst
> > i-2-25075-VM-eg all -- anywhere anywhere
> > PHYSDEV match --physdev-in vif108.0 --physdev-is-bridged match-set
> > i-2-25075-VM src
> > i-2-25075-VM all -- anywhere anywhere PHYSDEV
> > match --physdev-out vif108.0 --physdev-is-bridged
> >
> > You can see that the bridged entries have been incremented to vif109,
> > where as the rules still reference vif108.
> >
> > Stopping the instance appears to clear out the rules, and then everything
> > works fine again once the instance is started.
> >
> > Is this a known issue? Is anyone able to replicate this?
> >
> > Cheers!
> >
> >
>
Re: Instance reboot issue - CS 4.5.1 - Xen 6.5
Posted by Rajani Karuturi <ra...@apache.org>.
its in 4.5.2 and will also be in 4.6.0
~Rajani
On Wed, Sep 2, 2015 at 5:33 PM, cs user <ac...@gmail.com> wrote:
> Hi Folks,
>
> Looks like this is fixed in https://github.com/apache/cloudstack/pull/479
> ?
>
> Which cloudstack release contains this fix?
>
> Many thanks!
>
> On Wed, Sep 2, 2015 at 11:16 AM, cs user <ac...@gmail.com> wrote:
>
> > Forwarding to users channel in case anyone else has seen this...
> >
> >
> >
> > Hi All,
> >
> > We have seen this in 2 separate environments, both running the same
> > versions of Cloudstack and Xenserver. When we reboot an instance, we lose
> > access to it.
> >
> > Looking at the iptables config on the xen host, we can see that the vif
> is
> > incremented for the bridged entries, but not updated for the rules.
> >
> > For example, this is how the iptables look before a reboot:
> >
> > [root@xen001 cloud]# iptables -L|grep 25075
> > i-2-25075-def all -- anywhere anywhere PHYSDEV
> > match --physdev-in vif108.0 --physdev-is-bridged
> > i-2-25075-def all -- anywhere anywhere PHYSDEV
> > match --physdev-out vif108.0 --physdev-is-bridged
> > Chain i-2-25075-VM (1 references)
> > Chain i-2-25075-VM-eg (1 references)
> > Chain i-2-25075-def (2 references)
> > RETURN udp -- anywhere anywhere PHYSDEV
> > match --physdev-in vif108.0 --physdev-is-bridged match-set i-2-25075-VM
> src
> > udp dpt:domain
> > DROP all -- anywhere anywhere PHYSDEV
> > match --physdev-in vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM
> > src
> > DROP all -- anywhere anywhere PHYSDEV
> > match --physdev-out vif108.0 --physdev-is-bridged ! match-set
> i-2-25075-VM
> > dst
> > i-2-25075-VM-eg all -- anywhere anywhere
> > PHYSDEV match --physdev-in vif108.0 --physdev-is-bridged match-set
> > i-2-25075-VM src
> > i-2-25075-VM all -- anywhere anywhere PHYSDEV
> > match --physdev-out vif108.0 --physdev-is-bridged
> >
> > After a reboot, we can see the following:
> >
> > [root@xen001 cloud]# iptables -L|grep 25075
> > i-2-25075-def all -- anywhere anywhere PHYSDEV
> > match --physdev-in vif109.0 --physdev-is-bridged
> > i-2-25075-def all -- anywhere anywhere PHYSDEV
> > match --physdev-out vif109.0 --physdev-is-bridged
> > Chain i-2-25075-VM (1 references)
> > Chain i-2-25075-VM-eg (1 references)
> > Chain i-2-25075-def (2 references)
> > RETURN udp -- anywhere anywhere PHYSDEV
> > match --physdev-in vif108.0 --physdev-is-bridged match-set i-2-25075-VM
> src
> > udp dpt:domain
> > DROP all -- anywhere anywhere PHYSDEV
> > match --physdev-in vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM
> > src
> > DROP all -- anywhere anywhere PHYSDEV
> > match --physdev-out vif108.0 --physdev-is-bridged ! match-set
> i-2-25075-VM
> > dst
> > i-2-25075-VM-eg all -- anywhere anywhere
> > PHYSDEV match --physdev-in vif108.0 --physdev-is-bridged match-set
> > i-2-25075-VM src
> > i-2-25075-VM all -- anywhere anywhere PHYSDEV
> > match --physdev-out vif108.0 --physdev-is-bridged
> >
> > You can see that the bridged entries have been incremented to vif109,
> > where as the rules still reference vif108.
> >
> > Stopping the instance appears to clear out the rules, and then everything
> > works fine again once the instance is started.
> >
> > Is this a known issue? Is anyone able to replicate this?
> >
> > Cheers!
> >
> >
>
Re: Instance reboot issue - CS 4.5.1 - Xen 6.5
Posted by cs user <ac...@gmail.com>.
Hi Folks,
Looks like this is fixed in https://github.com/apache/cloudstack/pull/479 ?
Which cloudstack release contains this fix?
Many thanks!
On Wed, Sep 2, 2015 at 11:16 AM, cs user <ac...@gmail.com> wrote:
> Forwarding to users channel in case anyone else has seen this...
>
>
>
> Hi All,
>
> We have seen this in 2 separate environments, both running the same
> versions of Cloudstack and Xenserver. When we reboot an instance, we lose
> access to it.
>
> Looking at the iptables config on the xen host, we can see that the vif is
> incremented for the bridged entries, but not updated for the rules.
>
> For example, this is how the iptables look before a reboot:
>
> [root@xen001 cloud]# iptables -L|grep 25075
> i-2-25075-def all -- anywhere anywhere PHYSDEV
> match --physdev-in vif108.0 --physdev-is-bridged
> i-2-25075-def all -- anywhere anywhere PHYSDEV
> match --physdev-out vif108.0 --physdev-is-bridged
> Chain i-2-25075-VM (1 references)
> Chain i-2-25075-VM-eg (1 references)
> Chain i-2-25075-def (2 references)
> RETURN udp -- anywhere anywhere PHYSDEV
> match --physdev-in vif108.0 --physdev-is-bridged match-set i-2-25075-VM src
> udp dpt:domain
> DROP all -- anywhere anywhere PHYSDEV
> match --physdev-in vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM
> src
> DROP all -- anywhere anywhere PHYSDEV
> match --physdev-out vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM
> dst
> i-2-25075-VM-eg all -- anywhere anywhere
> PHYSDEV match --physdev-in vif108.0 --physdev-is-bridged match-set
> i-2-25075-VM src
> i-2-25075-VM all -- anywhere anywhere PHYSDEV
> match --physdev-out vif108.0 --physdev-is-bridged
>
> After a reboot, we can see the following:
>
> [root@xen001 cloud]# iptables -L|grep 25075
> i-2-25075-def all -- anywhere anywhere PHYSDEV
> match --physdev-in vif109.0 --physdev-is-bridged
> i-2-25075-def all -- anywhere anywhere PHYSDEV
> match --physdev-out vif109.0 --physdev-is-bridged
> Chain i-2-25075-VM (1 references)
> Chain i-2-25075-VM-eg (1 references)
> Chain i-2-25075-def (2 references)
> RETURN udp -- anywhere anywhere PHYSDEV
> match --physdev-in vif108.0 --physdev-is-bridged match-set i-2-25075-VM src
> udp dpt:domain
> DROP all -- anywhere anywhere PHYSDEV
> match --physdev-in vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM
> src
> DROP all -- anywhere anywhere PHYSDEV
> match --physdev-out vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM
> dst
> i-2-25075-VM-eg all -- anywhere anywhere
> PHYSDEV match --physdev-in vif108.0 --physdev-is-bridged match-set
> i-2-25075-VM src
> i-2-25075-VM all -- anywhere anywhere PHYSDEV
> match --physdev-out vif108.0 --physdev-is-bridged
>
> You can see that the bridged entries have been incremented to vif109,
> where as the rules still reference vif108.
>
> Stopping the instance appears to clear out the rules, and then everything
> works fine again once the instance is started.
>
> Is this a known issue? Is anyone able to replicate this?
>
> Cheers!
>
>
Re: Instance reboot issue - CS 4.5.1 - Xen 6.5
Posted by cs user <ac...@gmail.com>.
Hi Folks,
Looks like this is fixed in https://github.com/apache/cloudstack/pull/479 ?
Which cloudstack release contains this fix?
Many thanks!
On Wed, Sep 2, 2015 at 11:16 AM, cs user <ac...@gmail.com> wrote:
> Forwarding to users channel in case anyone else has seen this...
>
>
>
> Hi All,
>
> We have seen this in 2 separate environments, both running the same
> versions of Cloudstack and Xenserver. When we reboot an instance, we lose
> access to it.
>
> Looking at the iptables config on the xen host, we can see that the vif is
> incremented for the bridged entries, but not updated for the rules.
>
> For example, this is how the iptables look before a reboot:
>
> [root@xen001 cloud]# iptables -L|grep 25075
> i-2-25075-def all -- anywhere anywhere PHYSDEV
> match --physdev-in vif108.0 --physdev-is-bridged
> i-2-25075-def all -- anywhere anywhere PHYSDEV
> match --physdev-out vif108.0 --physdev-is-bridged
> Chain i-2-25075-VM (1 references)
> Chain i-2-25075-VM-eg (1 references)
> Chain i-2-25075-def (2 references)
> RETURN udp -- anywhere anywhere PHYSDEV
> match --physdev-in vif108.0 --physdev-is-bridged match-set i-2-25075-VM src
> udp dpt:domain
> DROP all -- anywhere anywhere PHYSDEV
> match --physdev-in vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM
> src
> DROP all -- anywhere anywhere PHYSDEV
> match --physdev-out vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM
> dst
> i-2-25075-VM-eg all -- anywhere anywhere
> PHYSDEV match --physdev-in vif108.0 --physdev-is-bridged match-set
> i-2-25075-VM src
> i-2-25075-VM all -- anywhere anywhere PHYSDEV
> match --physdev-out vif108.0 --physdev-is-bridged
>
> After a reboot, we can see the following:
>
> [root@xen001 cloud]# iptables -L|grep 25075
> i-2-25075-def all -- anywhere anywhere PHYSDEV
> match --physdev-in vif109.0 --physdev-is-bridged
> i-2-25075-def all -- anywhere anywhere PHYSDEV
> match --physdev-out vif109.0 --physdev-is-bridged
> Chain i-2-25075-VM (1 references)
> Chain i-2-25075-VM-eg (1 references)
> Chain i-2-25075-def (2 references)
> RETURN udp -- anywhere anywhere PHYSDEV
> match --physdev-in vif108.0 --physdev-is-bridged match-set i-2-25075-VM src
> udp dpt:domain
> DROP all -- anywhere anywhere PHYSDEV
> match --physdev-in vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM
> src
> DROP all -- anywhere anywhere PHYSDEV
> match --physdev-out vif108.0 --physdev-is-bridged ! match-set i-2-25075-VM
> dst
> i-2-25075-VM-eg all -- anywhere anywhere
> PHYSDEV match --physdev-in vif108.0 --physdev-is-bridged match-set
> i-2-25075-VM src
> i-2-25075-VM all -- anywhere anywhere PHYSDEV
> match --physdev-out vif108.0 --physdev-is-bridged
>
> You can see that the bridged entries have been incremented to vif109,
> where as the rules still reference vif108.
>
> Stopping the instance appears to clear out the rules, and then everything
> works fine again once the instance is started.
>
> Is this a known issue? Is anyone able to replicate this?
>
> Cheers!
>
>