You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by "wanglei2@geekplus.com.cn" <wa...@geekplus.com.cn> on 2019/08/03 14:18:05 UTC
CaptureChangeMySQL processor takes more than 100% cpu usage after cathing up the lastet binlog position
First I set the binlog position to a earlier value, the processor takes about 20% cpu usage and consumes binlog much faster than what is generated.
But after it catched up with the latest binlog postion, the cpu usage go up to about 200%.
I have no idea why this happens.
Any insight on this?
wanglei2@geekplus.com.cn
发件人： wanglei2@geekplus.com.cn
发送时间： 2019-08-02 15:25
收件人： users
主题： Does CaptureChangeMySQL processor is CPU intensive?
I just run a single CaptureChangeMySQL processor， the cpu usage is more than 100%
Is it normal?
If so, why it is so cpu intensive?
wanglei2@geekplus.com.cn
CaptureChangeMySQL processor takes more than 100% cpu usage after cathing up the lastet binlog position
Posted by "wanglei2@geekplus.com.cn" <wa...@geekplus.com.cn>.
First I set the binlog position to a earlier value, the processor takes about 20% cpu usage and consumes binlog much faster than what is generated.
But after it catched up with the latest binlog postion, the cpu usage go up to about 200%.
I have no idea why this happens.
Any insight on this?
wanglei2@geekplus.com.cn
发件人： wanglei2@geekplus.com.cn
发送时间： 2019-08-02 15:25
收件人： users
主题： Does CaptureChangeMySQL processor is CPU intensive?
I just run a single CaptureChangeMySQL processor， the cpu usage is more than 100%
Is it normal?
If so, why it is so cpu intensive?
wanglei2@geekplus.com.cn
Re: Re: CaptureChangeMySQL processor takes more than 100% cpu usage after cathing up the lastet binlog position
Posted by "wanglei2@geekplus.com.cn" <wa...@geekplus.com.cn>.
Thanks very much, changing the scheduling to 10ms works.
wanglei2@geekplus.com.cn
From: Bryan Bende
Date: 2019-08-03 23:26
To: users
Subject: Re: CaptureChangeMySQL processor takes more than 100% cpu usage after cathing up the lastet binlog position
I only looked at this quickly, but I think the queue.poll in the onTrigger method should use a small blocking time of about 10ms so that when no events are available it won’t crush the CPU. Right now you could change the scheduling of the processor from 0 seconds to 10ms but then that slows it down when data is available too.
On Sat, Aug 3, 2019 at 10:18 AM wanglei2@geekplus.com.cn <wa...@geekplus.com.cn> wrote:
First I set the binlog position to a earlier value, the processor takes about 20% cpu usage and consumes binlog much faster than what is generated.
But after it catched up with the latest binlog postion, the cpu usage go up to about 200%.
I have no idea why this happens.
Any insight on this?
wanglei2@geekplus.com.cn
发件人： wanglei2@geekplus.com.cn
发送时间： 2019-08-02 15:25
收件人： users
主题： Does CaptureChangeMySQL processor is CPU intensive?
I just run a single CaptureChangeMySQL processor， the cpu usage is more than 100%
Is it normal?
If so, why it is so cpu intensive?
wanglei2@geekplus.com.cn
--
Sent from Gmail Mobile
Re: CaptureChangeMySQL processor takes more than 100% cpu usage after
cathing up the lastet binlog position
Posted by Bryan Bende <bb...@gmail.com>.
I only looked at this quickly, but I think the queue.poll in the onTrigger
method should use a small blocking time of about 10ms so that when no
events are available it won’t crush the CPU. Right now you could change the
scheduling of the processor from 0 seconds to 10ms but then that slows it
down when data is available too.
On Sat, Aug 3, 2019 at 10:18 AM wanglei2@geekplus.com.cn <
wanglei2@geekplus.com.cn> wrote:
>
> First I set the binlog position to a earlier value, the processor takes
> about 20% cpu usage and consumes binlog much faster than what is
> generated.
> But after it catched up with the latest binlog postion, the cpu usage go
> up to about 200%.
> I have no idea why this happens.
>
> Any insight on this?
> ------------------------------
> wanglei2@geekplus.com.cn
>
>
> *发件人：* wanglei2@geekplus.com.cn
> *发送时间：* 2019-08-02 15:25
> *收件人：* users <us...@nifi.apache.org>
> *主题：* Does CaptureChangeMySQL processor is CPU intensive?
>
> I just run a single CaptureChangeMySQL processor， the cpu usage is more
> than 100%
>
> Is it normal?
> If so, why it is so cpu intensive?
>
>
>
> ------------------------------
> wanglei2@geekplus.com.cn
>
> --
Sent from Gmail Mobile
CaptureChangeMySQL processor takes more than 100% cpu usage after cathing up the lastet binlog position
Posted by "wanglei2@geekplus.com.cn" <wa...@geekplus.com.cn>.
First I set the binlog position to a earlier value, the processor takes about 20% cpu usage and consumes binlog much faster than what is generated.
But after it catched up with the latest binlog postion, the cpu usage go up to about 200%.
I have no idea why this happens.
Any insight on this?
wanglei2@geekplus.com.cn
发件人： wanglei2@geekplus.com.cn
发送时间： 2019-08-02 15:25
收件人： users
主题： Does CaptureChangeMySQL processor is CPU intensive?
I just run a single CaptureChangeMySQL processor， the cpu usage is more than 100%
Is it normal?
If so, why it is so cpu intensive?
wanglei2@geekplus.com.cn