PutKafka taks number is huge

["BOUKRAA, Kacem" <bk...@esi.dz>]

Hello Nifi users community,

​So i'm using a ListenHTTP processor followed by PutKafka,
I've created the Kafka topic. launched the producer, but the number of task
is very high, is that normal?

My config:


​Thanks
​​

-- 

[image: Image]
*Kacem BOUKRAA*

Re: PutKafka taks number is huge

["BOUKRAA, Kacem" <bk...@esi.dz>]

Jeff thanks,
It seems like the time interval was the reason for this extra load as you
mentioned .
Thanks for your help. You're no longer a begginer x)

Sent from my Nexus7
On Feb 15, 2016 11:36 AM, "Jeff - Data Bean Australia" <
databean.au@gmail.com> wrote:

> Hi Kacem,
>
> I am also new to NiFi. Just try to put some thoughts and see whether it
> makes sense to you.
>
> I can't say it is normal, but I may understand why the task number was so
> high:
>
>
>    - PutKafka is annotated as @TriggerWhenEmpty. With this, PutKafka will
>    be triggered as often as the SchedulingAgent want to. The scheduling agent
>    in your case was Timer Driven Scheduling Agent I think. And by default, the
>    value of Run schedule is 0 sec, which means Timer Driven Scheduling Agent
>    will schedule PutKafka without waiting.
>    - When PutKafka tries to poll data/FlowFile out of a ProcessSession, I
>    guess it doesn't block when FlowFile is not ready.
>
>
> So I guess this is the reason the number of tasks for PutKafka is so high.
> And I suspect it consumed a lot of CPU time also. Maybe you can put a small
> value, such as 1 ms to Run Schedule to slow it down if it makes sense to
> you.
>
> Thanks,
> Jeff
>
> On Mon, Feb 15, 2016 at 10:07 PM, BOUKRAA, Kacem <bk...@esi.dz>
> wrote:
>
>> Hello Nifi users community,
>>
>> ​So i'm using a ListenHTTP processor followed by PutKafka,
>> I've created the Kafka topic. launched the producer, but the number of
>> task is very high, is that normal?
>>
>> My config:
>>
>>
>> ​Thanks
>> ​​
>>
>> --
>>
>> [image: Image]
>> *Kacem BOUKRAA*
>>
>>
>>
>
>
> --
> Data Bean - A Big Data Solution Provider in Australia.
>

Re: PutKafka taks number is huge

[Jeff - Data Bean Australia <da...@gmail.com>]

Hi Kacem,

I am also new to NiFi. Just try to put some thoughts and see whether it
makes sense to you.

I can't say it is normal, but I may understand why the task number was so
high:


   - PutKafka is annotated as @TriggerWhenEmpty. With this, PutKafka will
   be triggered as often as the SchedulingAgent want to. The scheduling agent
   in your case was Timer Driven Scheduling Agent I think. And by default, the
   value of Run schedule is 0 sec, which means Timer Driven Scheduling Agent
   will schedule PutKafka without waiting.
   - When PutKafka tries to poll data/FlowFile out of a ProcessSession, I
   guess it doesn't block when FlowFile is not ready.


So I guess this is the reason the number of tasks for PutKafka is so high.
And I suspect it consumed a lot of CPU time also. Maybe you can put a small
value, such as 1 ms to Run Schedule to slow it down if it makes sense to
you.

Thanks,
Jeff

On Mon, Feb 15, 2016 at 10:07 PM, BOUKRAA, Kacem <bk...@esi.dz> wrote:

> Hello Nifi users community,
>
> ​So i'm using a ListenHTTP processor followed by PutKafka,
> I've created the Kafka topic. launched the producer, but the number of
> task is very high, is that normal?
>
> My config:
>
>
> ​Thanks
> ​​
>
> --
>
> [image: Image]
> *Kacem BOUKRAA*
>
>
>


-- 
Data Bean - A Big Data Solution Provider in Australia.