You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Inder Pall <in...@gmail.com> on 2011/10/27 10:22:55 UTC
embedded consumer within a broker
i want to try out this use-case what's the config....couldn't find much on
http://incubator.apache.org/kafka/configuration.html
Also, would be nice if the archives could be searched through
http://search-hadoop.com/
--
-- Inder
Re: embedded consumer within a broker
Posted by Inder Pall <in...@gmail.com>.
Neha,
i'll try this out. Regarding search-hadoop.com( i got to know through one of
cloudera's tech talk)...i think it's maintained by a company
http://www.sematext.com/.
They do have a contact us and about - http://search-hadoop.com/about.html.
- inder
On Thu, Oct 27, 2011 at 7:46 PM, Neha Narkhede <ne...@gmail.com>wrote:
> Inder,
>
> As part of the 0.7 release, we will update the documentation. To use
> this you want to pass the producer and consumer properties to the
> kafka server -
>
> ./bin/kafka-server-start.sh config/server.properties
> config/consumer.properties config/producer.properties
>
> In the config/consumer.properties, you can specify nothing if you want
> all the topics to be replicated. If you want only few topics to be
> replicated, specify them as a comma separated list in
> mirror.topics.whitelist. If you want few topics to not be replicated,
> specify those in mirror.topics.blacklist.
>
> >> are they supposed to be used exclusively?
> No
>
> >> Also, would be nice if the archives could be searched through
> http://search-hadoop.com/
>
> That is a good suggestion. Do you know how we can make the code
> searchable through that website ? Seems like currently only Apache
> TLPs are available there.
>
> Thanks,
> Neha
>
> On Thu, Oct 27, 2011 at 2:24 AM, Inder Pall <in...@gmail.com> wrote:
> > got the following from code -
> >
> >
> > val MirrorTopicsWhitelistProp = "mirror.topics.whitelist"
> > val MirrorTopicsBlacklistProp = "mirror.topics.blacklist"
> > val MirrorConsumerNumThreadsProp = "mirror.consumer.numthreads"
> >
> > what's the difference between whitelist and blacklist, are they supposed
> to
> > be used exclusively?
> > Will try with this.
> >
> > - inder
> >
> > On Thu, Oct 27, 2011 at 1:52 PM, Inder Pall <in...@gmail.com>
> wrote:
> >
> >>
> >> i want to try out this use-case what's the config....couldn't find much
> on
> >> http://incubator.apache.org/kafka/configuration.html
> >> Also, would be nice if the archives could be searched through
> >> http://search-hadoop.com/
> >>
> >> --
> >> -- Inder
> >>
> >
> >
> >
> > --
> > -- Inder
> >
>
--
-- Inder
Re: embedded consumer within a broker
Posted by Jun Rao <ju...@gmail.com>.
>> are they supposed to be used exclusively?
The whitelist and the blacklist are exclusive at this moment. You can only
use one of them. The whitelist takes precedence if both are specified.
Jun
On Thu, Oct 27, 2011 at 7:16 AM, Neha Narkhede <ne...@gmail.com>wrote:
> Inder,
>
> As part of the 0.7 release, we will update the documentation. To use
> this you want to pass the producer and consumer properties to the
> kafka server -
>
> ./bin/kafka-server-start.sh config/server.properties
> config/consumer.properties config/producer.properties
>
> In the config/consumer.properties, you can specify nothing if you want
> all the topics to be replicated. If you want only few topics to be
> replicated, specify them as a comma separated list in
> mirror.topics.whitelist. If you want few topics to not be replicated,
> specify those in mirror.topics.blacklist.
>
> >> are they supposed to be used exclusively?
> No
>
> >> Also, would be nice if the archives could be searched through
> http://search-hadoop.com/
>
> That is a good suggestion. Do you know how we can make the code
> searchable through that website ? Seems like currently only Apache
> TLPs are available there.
>
> Thanks,
> Neha
>
> On Thu, Oct 27, 2011 at 2:24 AM, Inder Pall <in...@gmail.com> wrote:
> > got the following from code -
> >
> >
> > val MirrorTopicsWhitelistProp = "mirror.topics.whitelist"
> > val MirrorTopicsBlacklistProp = "mirror.topics.blacklist"
> > val MirrorConsumerNumThreadsProp = "mirror.consumer.numthreads"
> >
> > what's the difference between whitelist and blacklist, are they supposed
> to
> > be used exclusively?
> > Will try with this.
> >
> > - inder
> >
> > On Thu, Oct 27, 2011 at 1:52 PM, Inder Pall <in...@gmail.com>
> wrote:
> >
> >>
> >> i want to try out this use-case what's the config....couldn't find much
> on
> >> http://incubator.apache.org/kafka/configuration.html
> >> Also, would be nice if the archives could be searched through
> >> http://search-hadoop.com/
> >>
> >> --
> >> -- Inder
> >>
> >
> >
> >
> > --
> > -- Inder
> >
>
Re: embedded consumer within a broker
Posted by Neha Narkhede <ne...@gmail.com>.
Inder,
As part of the 0.7 release, we will update the documentation. To use
this you want to pass the producer and consumer properties to the
kafka server -
./bin/kafka-server-start.sh config/server.properties
config/consumer.properties config/producer.properties
In the config/consumer.properties, you can specify nothing if you want
all the topics to be replicated. If you want only few topics to be
replicated, specify them as a comma separated list in
mirror.topics.whitelist. If you want few topics to not be replicated,
specify those in mirror.topics.blacklist.
>> are they supposed to be used exclusively?
No
>> Also, would be nice if the archives could be searched through http://search-hadoop.com/
That is a good suggestion. Do you know how we can make the code
searchable through that website ? Seems like currently only Apache
TLPs are available there.
Thanks,
Neha
On Thu, Oct 27, 2011 at 2:24 AM, Inder Pall <in...@gmail.com> wrote:
> got the following from code -
>
>
> val MirrorTopicsWhitelistProp = "mirror.topics.whitelist"
> val MirrorTopicsBlacklistProp = "mirror.topics.blacklist"
> val MirrorConsumerNumThreadsProp = "mirror.consumer.numthreads"
>
> what's the difference between whitelist and blacklist, are they supposed to
> be used exclusively?
> Will try with this.
>
> - inder
>
> On Thu, Oct 27, 2011 at 1:52 PM, Inder Pall <in...@gmail.com> wrote:
>
>>
>> i want to try out this use-case what's the config....couldn't find much on
>> http://incubator.apache.org/kafka/configuration.html
>> Also, would be nice if the archives could be searched through
>> http://search-hadoop.com/
>>
>> --
>> -- Inder
>>
>
>
>
> --
> -- Inder
>
Re: embedded consumer within a broker
Posted by Inder Pall <in...@gmail.com>.
got the following from code -
val MirrorTopicsWhitelistProp = "mirror.topics.whitelist"
val MirrorTopicsBlacklistProp = "mirror.topics.blacklist"
val MirrorConsumerNumThreadsProp = "mirror.consumer.numthreads"
what's the difference between whitelist and blacklist, are they supposed to
be used exclusively?
Will try with this.
- inder
On Thu, Oct 27, 2011 at 1:52 PM, Inder Pall <in...@gmail.com> wrote:
>
> i want to try out this use-case what's the config....couldn't find much on
> http://incubator.apache.org/kafka/configuration.html
> Also, would be nice if the archives could be searched through
> http://search-hadoop.com/
>
> --
> -- Inder
>
--
-- Inder