You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Girish Shilamkar <gi...@clogeny.com> on 2013/07/04 11:59:54 UTC

Verifying Load Balancer logic

Hello,

Some of the BVT test cases verify if load balancing rule set to round robin is working or not. 
It is seen that often LBing is not perfect round robin. If first request goes to one vm the next does not always go to second,
when there are just two vms added to LB rule.

I vaguely recall discussing this with Prasanna where he suggested removing these test cases. Prasanna, please correct me if I am wrong.

Regards,
Girish

RE: Verifying Load Balancer logic

Posted by Sudha Ponnaganti <su...@citrix.com>.
+1 - let us not remove them. If they are not suitable for BVT, include them in regression suite. 

-----Original Message-----
From: Ahmad Emneina [mailto:aemneina@gmail.com] 
Sent: Thursday, July 04, 2013 3:09 AM
To: dev@cloudstack.apache.org
Subject: Re: Verifying Load Balancer logic

why would we remove these cases and not find the root cause, if there is an issue, and fixing that.


On Thu, Jul 4, 2013 at 2:59 AM, Girish Shilamkar <gi...@clogeny.com> wrote:

> Hello,
>
> Some of the BVT test cases verify if load balancing rule set to round 
> robin is working or not.
> It is seen that often LBing is not perfect round robin. If first 
> request goes to one vm the next does not always go to second, when 
> there are just two vms added to LB rule.
>
> I vaguely recall discussing this with Prasanna where he suggested 
> removing these test cases. Prasanna, please correct me if I am wrong.
>
> Regards,
> Girish

RE: Verifying Load Balancer logic

Posted by Ram Ganesh <Ra...@citrix.com>.
> -----Original Message-----
> From: Prasanna Santhanam [mailto:prasanna.santhanam@citrix.com]
> Sent: 04 July 2013 17:10
> To: dev@cloudstack.apache.org
> Cc: aemneina@gmail.com
> Subject: Re: Verifying Load Balancer logic
> 
> On Thu, Jul 04, 2013 at 10:21:55AM +0000, Ram Ganesh wrote:
> > This could happen if there is a stickiness policy configured on the LB rule.
> 
> This is the basic round-robin on a VR, we don't use any stickiness policy on
> this.

Could be a bug in HAproxy. 

> 
> >
> > > -----Original Message-----
> > > From: Ahmad Emneina [mailto:aemneina@gmail.com]
> > > Sent: 04 July 2013 15:39
> > > To: dev@cloudstack.apache.org
> > > Subject: Re: Verifying Load Balancer logic
> > >
> > > why would we remove these cases and not find the root cause, if
> > > there is an issue, and fixing that.
> > >
> Not the test cases, just the haproxy lb verification where we ssh and detect
> that the hostname we get back is different each time, ascertaining that we
> are going round robin.
> 
> > >
> > > On Thu, Jul 4, 2013 at 2:59 AM, Girish Shilamkar <gi...@clogeny.com>
> wrote:
> > >
> > > > Hello,
> > > >
> > > > Some of the BVT test cases verify if load balancing rule set to
> > > > round robin is working or not.
> > > > It is seen that often LBing is not perfect round robin. If first
> > > > request goes to one vm the next does not always go to second, when
> > > > there are just two vms added to LB rule.
> 
> This test is not consistent. I separated the suite and added some more logs
> for the multiple ssh attempts we do in the test. The test passed on a KVM
> setup and fails on XCP,Xen.
> 
> http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-smoke-
> matrix/suite=test_loadbalance/lastCompletedBuild/testReport/
> 
> It's possible that the ssh connection closes when you make a second attempt
> and that's why haproxy (having no active connections) redirects to the same
> host repeatedly.  But I'm only guessing. If there's anyone familiar with
> haproxy behaviour, they could throw some light here.
> 
> --
> Prasanna.,

Re: Verifying Load Balancer logic

Posted by Prasanna Santhanam <pr...@citrix.com>.
On Thu, Jul 04, 2013 at 10:21:55AM +0000, Ram Ganesh wrote:
> This could happen if there is a stickiness policy configured on the LB rule.

This is the basic round-robin on a VR, we don't use any stickiness
policy on this.

> 
> > -----Original Message-----
> > From: Ahmad Emneina [mailto:aemneina@gmail.com]
> > Sent: 04 July 2013 15:39
> > To: dev@cloudstack.apache.org
> > Subject: Re: Verifying Load Balancer logic
> > 
> > why would we remove these cases and not find the root cause, if there is an
> > issue, and fixing that.
> > 
Not the test cases, just the haproxy lb verification where we ssh and
detect that the hostname we get back is different each time,
ascertaining that we are going round robin.

> > 
> > On Thu, Jul 4, 2013 at 2:59 AM, Girish Shilamkar <gi...@clogeny.com> wrote:
> > 
> > > Hello,
> > >
> > > Some of the BVT test cases verify if load balancing rule set to round
> > > robin is working or not.
> > > It is seen that often LBing is not perfect round robin. If first
> > > request goes to one vm the next does not always go to second, when
> > > there are just two vms added to LB rule.

This test is not consistent. I separated the suite and added some more
logs for the multiple ssh attempts we do in the test. The test passed
on a KVM setup and fails on XCP,Xen.

http://jenkins.buildacloud.org/view/cloudstack-qa/job/test-smoke-matrix/suite=test_loadbalance/lastCompletedBuild/testReport/

It's possible that the ssh connection closes when you make a second
attempt and that's why haproxy (having no active connections)
redirects to the same host repeatedly.  But I'm only guessing. If
there's anyone familiar with haproxy behaviour, they could throw some
light here.

-- 
Prasanna.,

RE: Verifying Load Balancer logic

Posted by Ram Ganesh <Ra...@citrix.com>.
This could happen if there is a stickiness policy configured on the LB rule.

> -----Original Message-----
> From: Ahmad Emneina [mailto:aemneina@gmail.com]
> Sent: 04 July 2013 15:39
> To: dev@cloudstack.apache.org
> Subject: Re: Verifying Load Balancer logic
> 
> why would we remove these cases and not find the root cause, if there is an
> issue, and fixing that.
> 
> 
> On Thu, Jul 4, 2013 at 2:59 AM, Girish Shilamkar <gi...@clogeny.com> wrote:
> 
> > Hello,
> >
> > Some of the BVT test cases verify if load balancing rule set to round
> > robin is working or not.
> > It is seen that often LBing is not perfect round robin. If first
> > request goes to one vm the next does not always go to second, when
> > there are just two vms added to LB rule.
> >
> > I vaguely recall discussing this with Prasanna where he suggested
> > removing these test cases. Prasanna, please correct me if I am wrong.
> >
> > Regards,
> > Girish

Re: Verifying Load Balancer logic

Posted by Ahmad Emneina <ae...@gmail.com>.
why would we remove these cases and not find the root cause, if there is an
issue, and fixing that.


On Thu, Jul 4, 2013 at 2:59 AM, Girish Shilamkar <gi...@clogeny.com> wrote:

> Hello,
>
> Some of the BVT test cases verify if load balancing rule set to round
> robin is working or not.
> It is seen that often LBing is not perfect round robin. If first request
> goes to one vm the next does not always go to second,
> when there are just two vms added to LB rule.
>
> I vaguely recall discussing this with Prasanna where he suggested removing
> these test cases. Prasanna, please correct me if I am wrong.
>
> Regards,
> Girish