Loadbalancing producers

[Mark <st...@gmail.com>]

Is it common/necessary to load balance the connections your producers use? I'm looking at the Ruby gem and it doesn't look like this is handled via the gem. 

Re: Loadbalancing producers

[Benjamin Black <b...@b3k.us>]

Producers discover the broker that owns each partition and send messages
directly. There isn't anything to load balance at layer 4.
On Aug 29, 2013 8:54 AM, "Mark" <st...@gmail.com> wrote:

> Can you explain why not?
>
> On Aug 29, 2013, at 8:43 AM, Benjamin Black <b...@b3k.us> wrote:
>
> > The LB in front of the brokers doesn't make sense.
> > On Aug 29, 2013 8:42 AM, "Mark" <st...@gmail.com> wrote:
> >
> >> We have a few dozen front-end web apps running rails. Each one of these
> >> rails instances has an embedded producer which in turn connects to a 3
> >> brokers who are behind a load-balancer. Does that sound about right?
> >>
> >> Is there anything special to account for since these TCP connections are
> >> long lived?
> >>
> >> On Aug 29, 2013, at 7:20 AM, Jun Rao <ju...@gmail.com> wrote:
> >>
> >>> I think a typical setup is that you have a load balancer in front of a
> >>> bunch of frontend services, each of which has an embedded producer.
> >>>
> >>> Thanks,
> >>>
> >>> Jun
> >>>
> >>>
> >>> On Wed, Aug 28, 2013 at 9:01 PM, Mark <st...@gmail.com>
> wrote:
> >>>
> >>>> Is it common/necessary to load balance the connections your producers
> >> use?
> >>>> I'm looking at the Ruby gem and it doesn't look like this is handled
> via
> >>>> the gem.
> >>
> >>
>
>

Re: Loadbalancing producers

[chetan conikee <co...@gmail.com>]

As Benjamin indicated, you wouldn't get much out of an LB in front of
brokers, especially if the producers are configured in async mode (which
you would want to in order to increase throughput).



On Thu, Aug 29, 2013 at 8:54 AM, Mark <st...@gmail.com> wrote:

> Can you explain why not?
>
> On Aug 29, 2013, at 8:43 AM, Benjamin Black <b...@b3k.us> wrote:
>
> > The LB in front of the brokers doesn't make sense.
> > On Aug 29, 2013 8:42 AM, "Mark" <st...@gmail.com> wrote:
> >
> >> We have a few dozen front-end web apps running rails. Each one of these
> >> rails instances has an embedded producer which in turn connects to a 3
> >> brokers who are behind a load-balancer. Does that sound about right?
> >>
> >> Is there anything special to account for since these TCP connections are
> >> long lived?
> >>
> >> On Aug 29, 2013, at 7:20 AM, Jun Rao <ju...@gmail.com> wrote:
> >>
> >>> I think a typical setup is that you have a load balancer in front of a
> >>> bunch of frontend services, each of which has an embedded producer.
> >>>
> >>> Thanks,
> >>>
> >>> Jun
> >>>
> >>>
> >>> On Wed, Aug 28, 2013 at 9:01 PM, Mark <st...@gmail.com>
> wrote:
> >>>
> >>>> Is it common/necessary to load balance the connections your producers
> >> use?
> >>>> I'm looking at the Ruby gem and it doesn't look like this is handled
> via
> >>>> the gem.
> >>
> >>
>
>

Re: Loadbalancing producers

[Mark <st...@gmail.com>]

Can you explain why not?

On Aug 29, 2013, at 8:43 AM, Benjamin Black <b...@b3k.us> wrote:

> The LB in front of the brokers doesn't make sense.
> On Aug 29, 2013 8:42 AM, "Mark" <st...@gmail.com> wrote:
> 
>> We have a few dozen front-end web apps running rails. Each one of these
>> rails instances has an embedded producer which in turn connects to a 3
>> brokers who are behind a load-balancer. Does that sound about right?
>> 
>> Is there anything special to account for since these TCP connections are
>> long lived?
>> 
>> On Aug 29, 2013, at 7:20 AM, Jun Rao <ju...@gmail.com> wrote:
>> 
>>> I think a typical setup is that you have a load balancer in front of a
>>> bunch of frontend services, each of which has an embedded producer.
>>> 
>>> Thanks,
>>> 
>>> Jun
>>> 
>>> 
>>> On Wed, Aug 28, 2013 at 9:01 PM, Mark <st...@gmail.com> wrote:
>>> 
>>>> Is it common/necessary to load balance the connections your producers
>> use?
>>>> I'm looking at the Ruby gem and it doesn't look like this is handled via
>>>> the gem.
>> 
>> 

Re: Loadbalancing producers

[Benjamin Black <b...@b3k.us>]

The LB in front of the brokers doesn't make sense.
On Aug 29, 2013 8:42 AM, "Mark" <st...@gmail.com> wrote:

> We have a few dozen front-end web apps running rails. Each one of these
> rails instances has an embedded producer which in turn connects to a 3
> brokers who are behind a load-balancer. Does that sound about right?
>
> Is there anything special to account for since these TCP connections are
> long lived?
>
> On Aug 29, 2013, at 7:20 AM, Jun Rao <ju...@gmail.com> wrote:
>
> > I think a typical setup is that you have a load balancer in front of a
> > bunch of frontend services, each of which has an embedded producer.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Wed, Aug 28, 2013 at 9:01 PM, Mark <st...@gmail.com> wrote:
> >
> >> Is it common/necessary to load balance the connections your producers
> use?
> >> I'm looking at the Ruby gem and it doesn't look like this is handled via
> >> the gem.
>
>

Re: Loadbalancing producers

[Mark <st...@gmail.com>]

We have a few dozen front-end web apps running rails. Each one of these rails instances has an embedded producer which in turn connects to a 3 brokers who are behind a load-balancer. Does that sound about right?

Is there anything special to account for since these TCP connections are long lived?

On Aug 29, 2013, at 7:20 AM, Jun Rao <ju...@gmail.com> wrote:

> I think a typical setup is that you have a load balancer in front of a
> bunch of frontend services, each of which has an embedded producer.
> 
> Thanks,
> 
> Jun
> 
> 
> On Wed, Aug 28, 2013 at 9:01 PM, Mark <st...@gmail.com> wrote:
> 
>> Is it common/necessary to load balance the connections your producers use?
>> I'm looking at the Ruby gem and it doesn't look like this is handled via
>> the gem.

Re: Loadbalancing producers

[Jun Rao <ju...@gmail.com>]

I think a typical setup is that you have a load balancer in front of a
bunch of frontend services, each of which has an embedded producer.

Thanks,

Jun


On Wed, Aug 28, 2013 at 9:01 PM, Mark <st...@gmail.com> wrote:

> Is it common/necessary to load balance the connections your producers use?
> I'm looking at the Ruby gem and it doesn't look like this is handled via
> the gem.