You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by l vic <lv...@gmail.com> on 2018/11/20 15:09:51 UTC
setting topic name dynamically in PutKafka?
Hi,
I have to set topic name property of PutKafka processors from an attribute
of the incoming flowfile...Is there any possible way of doing this?
Thank you,
-V
Re: setting topic name dynamically in PutKafka?
Posted by Bryan Bende <bb...@gmail.com>.
PutKafka doesn't support dynamic topic names, but it is a fairly old
processor and should only be used with a 0.8.0 Kafka broker.
I think the PublishKafka variants (0.9, 0.10, 0.11, 1.0, 2.0) should
all support dynamic topic names.
On Tue, Nov 20, 2018 at 10:12 AM l vic <lv...@gmail.com> wrote:
>
> Hi,
> I have to set topic name property of PutKafka processors from an attribute of the incoming flowfile...Is there any possible way of doing this?
> Thank you,
> -V
Re: setting topic name dynamically in PutKafka?
Posted by l vic <lv...@gmail.com>.
It does work, at least in the prototype
On Tue, Nov 20, 2018 at 2:08 PM Bryan Bende <bb...@gmail.com> wrote:
> That is correct for the ConsumeKafka processor, but this question was
> about the producer side :)
>
> I believe you can make a producer ahead of time, and then specify the
> topic when making a call to actually send a message, at least that
> what it looks like the code does in all the PublishKafka variants...
>
>
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-0-9-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/PublishKafka.java#L325
>
> On Tue, Nov 20, 2018 at 1:53 PM Boris Tyukin <bo...@boristyukin.com>
> wrote:
> >
> > I think they did in on purpose because it takes time (3-5 seconds) for a
> consumer group to connect to Kafka (joining consumer group) and whoever
> created processor did not want to do it for each flowfile.
> >
> > On Tue, Nov 20, 2018 at 10:09 AM l vic <lv...@gmail.com> wrote:
> >>
> >> Hi,
> >> I have to set topic name property of PutKafka processors from an
> attribute of the incoming flowfile...Is there any possible way of doing
> this?
> >> Thank you,
> >> -V
>
Re: setting topic name dynamically in PutKafka?
Posted by Bryan Bende <bb...@gmail.com>.
That is correct for the ConsumeKafka processor, but this question was
about the producer side :)
I believe you can make a producer ahead of time, and then specify the
topic when making a call to actually send a message, at least that
what it looks like the code does in all the PublishKafka variants...
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-0-9-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/PublishKafka.java#L325
On Tue, Nov 20, 2018 at 1:53 PM Boris Tyukin <bo...@boristyukin.com> wrote:
>
> I think they did in on purpose because it takes time (3-5 seconds) for a consumer group to connect to Kafka (joining consumer group) and whoever created processor did not want to do it for each flowfile.
>
> On Tue, Nov 20, 2018 at 10:09 AM l vic <lv...@gmail.com> wrote:
>>
>> Hi,
>> I have to set topic name property of PutKafka processors from an attribute of the incoming flowfile...Is there any possible way of doing this?
>> Thank you,
>> -V
Re: setting topic name dynamically in PutKafka?
Posted by Boris Tyukin <bo...@boristyukin.com>.
I think they did in on purpose because it takes time (3-5 seconds) for a
consumer group to connect to Kafka (joining consumer group) and whoever
created processor did not want to do it for each flowfile.
On Tue, Nov 20, 2018 at 10:09 AM l vic <lv...@gmail.com> wrote:
> Hi,
> I have to set topic name property of PutKafka processors from an attribute
> of the incoming flowfile...Is there any possible way of doing this?
> Thank you,
> -V
>