You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by "Rodenburg, Jeff" <je...@teamaol.com> on 2012/06/20 20:48:38 UTC

Topic/partition creation

I'm exploring the v0.7 code to understand topics and partitions and how those are created/managed. I'm trying to understand how cheap/expensive it is to create/shutdown/manage those items.  A few questions:


  *   "Topics" appear to be metadata on partitions, which are the actual objects in play. Is this accurate?
  *   What is the relationship between ZK and broker regarding topics and partitions?

I'm walking the code sequences now, but some clarification would be beneficial.

-j

Re: Topic/partition creation

Posted by Jun Rao <ju...@gmail.com>.

Yes, the call is blocking, but is only called once per broker startup.

Thanks,

Jun

On Wed, Jun 20, 2012 at 3:06 PM, Rodenburg, Jeff <jeff.rodenburg@teamaol.com
> wrote:

> Thanks Jun.
>
> When the broker registers the topic/partition info with Zookeeper, is that
> a blocking call within the broker?
>
> On Jun 20, 2012, at 1:32 PM, Jun Rao wrote:
>
> > A topic can have 1 or more partitions, each of which corresponds to a
> > directory on local storage.
> >
> > ZK registers for each topic, how many partitions are there and which
> > brokers are those partitions stored.
> >
> > Thanks,
> >
> > Jun
> >
> > On Wed, Jun 20, 2012 at 11:48 AM, Rodenburg, Jeff <
> > jeff.rodenburg@teamaol.com> wrote:
> >
> >> I'm exploring the v0.7 code to understand topics and partitions and how
> >> those are created/managed. I'm trying to understand how cheap/expensive
> it
> >> is to create/shutdown/manage those items.  A few questions:
> >>
> >>
> >> *   "Topics" appear to be metadata on partitions, which are the actual
> >> objects in play. Is this accurate?
> >> *   What is the relationship between ZK and broker regarding topics and
> >> partitions?
> >>
> >> I'm walking the code sequences now, but some clarification would be
> >> beneficial.
> >>
> >> -j
> >>
>
>

Re: Topic/partition creation

Posted by "Rodenburg, Jeff" <je...@teamaol.com>.

Thanks Jun.

When the broker registers the topic/partition info with Zookeeper, is that a blocking call within the broker?

On Jun 20, 2012, at 1:32 PM, Jun Rao wrote:

> A topic can have 1 or more partitions, each of which corresponds to a
> directory on local storage.
> 
> ZK registers for each topic, how many partitions are there and which
> brokers are those partitions stored.
> 
> Thanks,
> 
> Jun
> 
> On Wed, Jun 20, 2012 at 11:48 AM, Rodenburg, Jeff <
> jeff.rodenburg@teamaol.com> wrote:
> 
>> I'm exploring the v0.7 code to understand topics and partitions and how
>> those are created/managed. I'm trying to understand how cheap/expensive it
>> is to create/shutdown/manage those items.  A few questions:
>> 
>> 
>> *   "Topics" appear to be metadata on partitions, which are the actual
>> objects in play. Is this accurate?
>> *   What is the relationship between ZK and broker regarding topics and
>> partitions?
>> 
>> I'm walking the code sequences now, but some clarification would be
>> beneficial.
>> 
>> -j
>>

Re: Topic/partition creation

Posted by Jun Rao <ju...@gmail.com>.

A topic can have 1 or more partitions, each of which corresponds to a
directory on local storage.

ZK registers for each topic, how many partitions are there and which
brokers are those partitions stored.

Thanks,

Jun

On Wed, Jun 20, 2012 at 11:48 AM, Rodenburg, Jeff <
jeff.rodenburg@teamaol.com> wrote:

> I'm exploring the v0.7 code to understand topics and partitions and how
> those are created/managed. I'm trying to understand how cheap/expensive it
> is to create/shutdown/manage those items.  A few questions:
>
>
>  *   "Topics" appear to be metadata on partitions, which are the actual
> objects in play. Is this accurate?
>  *   What is the relationship between ZK and broker regarding topics and
> partitions?
>
> I'm walking the code sequences now, but some clarification would be
> beneficial.
>
> -j
>