You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Joseph Witt (JIRA)" <ji...@apache.org> on 2017/12/11 14:01:00 UTC
[jira] [Commented] (NIFI-4682) when i start consumerKafka processor
more than eleven,all of the processors get into an indefinite stuck state
[ https://issues.apache.org/jira/browse/NIFI-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285948#comment-16285948 ]
Joseph Witt commented on NIFI-4682:
-----------------------------------
[~clearlove] in looking at the provided stack trace all of the kafka clients are sitting in their polling calls against the kafka broker. It is not clear that they're actually stuck though. Are you looking at the lag indicators showing that these clients are not keeping up or have lag? What command are you running to verify that?
The warning you show when you stop these processors is just an indicator that we told the kafka client to break out of the wait/poll cycle. This is expected when a stop command is issued to the processor if it was in the middle of doing that. https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-0-10-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumeKafka_0_10.java
There may be an issue with the client where it is actually stuck but we'll have to keep looking for more detail to confirm that. The more you can do to provide actual configuration details the better. Someone who will help dig into this will need actual details on the Kafka broker and configuration, the niFi server configuration, application logs, and ideally a flow template. I appreciate you sharing the images but those often do not reveal enough data and are not reproducible. Please share a flow template that replicates the issue if you can.
> when i start consumerKafka processor more than eleven,all of the processors get into an indefinite stuck state
> --------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-4682
> URL: https://issues.apache.org/jira/browse/NIFI-4682
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.3.0
> Environment: Linux
> Reporter: clearlove
> Priority: Blocker
> Attachments: 2.png, nifi-bootstrap.log
>
>
> hello,
> When i start one consumeKafka_0_10 processor ,all of the processors are all right.
> but when i start consumeKafka_0_10 processor more than eleven,all of the processors get into an indefinite stuck state.
> !1.png|thumbnail!
> like in the picture ,look like everything is all right, but all processor(even processor is not consumeKafka_0_10 ) are stop work.All of the flowfile don't move.
> And when i stop them , some of consumeKafka_0_10 processor are warning
> {{Was interrupted while trying to communicate with Kafka with lease {}. Will roll back session and discard any partially received data.}}
> in the nifi-app.log
> {noformat}
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=099a83f6-7ddc-3df8-fe07-5b901799ade3] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.SplitJson
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=fdd4850d-a619-3f17-6287-198cac3bb52a] to run
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling SplitJson[id=f3f85270-6bb8-3f65-4347-6700426d6a2e] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=6aacc9a9-c7e8-32b0-ed3c-3f89e72c30cf] to run
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,130 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=02aaae90-5ada-343e-6c15-3cfff7cfded0] to run
> 2017-12-05 10:12:02,130 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=278dc7a5-6d2e-336a-45c3-56cb7039dc7c] to run
> 2017-12-05 10:12:02,131 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=60fd28d8-b5f0-3a5b-e31a-e3ba7b73d7cb] to run
> 2017-12-05 10:12:02,131 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=efa69212-d0c9-3c83-0985-b22115b5e861] to run
> 2017-12-05 10:12:02,724 INFO [Flow Service Tasks Thread-1] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@a987d5e // Another save pending = false
> 2017-12-05 10:12:07,130 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=2fedc410-9641-3bda-cc40-59cf739a5400] to run
> 2017-12-05 10:12:07,130 WARN [Timer-Driven Process Thread-8] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=a2b469df-a2ef-355f-cc68-7b5dea6abb8b] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@2b5a2eed. Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,130 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=d925dcdc-edef-3184-2185-47a0ebf3ea3b] to run
> 2017-12-05 10:12:07,131 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=8b6fef2d-3db8-3558-89a3-abce7b0e8797] to run
> 2017-12-05 10:12:07,131 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=e1af6798-96ba-31ec-c135-ff2f10d1306b] to run
> 2017-12-05 10:12:07,131 WARN [Timer-Driven Process Thread-7] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=d4b01bbd-62a3-3584-8391-371d52b83bce] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@7060d90d. Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,132 INFO [StandardProcessScheduler Thread-8] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=b88105fe-cb1e-3c4e-f91d-22f5542cecd7] to run
> 2017-12-05 10:12:07,132 WARN [Timer-Driven Process Thread-2] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=4c067c00-fbc2-3571-5b7a-37b4957afb87] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@158837ab. Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=182347e6-14ef-38c2-6873-5602f0405296] to run
> 2017-12-05 10:12:07,133 WARN [Timer-Driven Process Thread-3] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=4035760a-8b9e-3e34-51e9-e292c58e6ed7] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@4967b312. Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=f7285790-6159-3726-662e-5717235a31dc] to run
> 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-1] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=fae14360-943d-3a81-f3b1-675f5505e265] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=86b794b6-5875-3eac-9fdb-447743459b7d] to run
> 2017-12-05 10:12:07,133 WARN [Timer-Driven Process Thread-9] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=716f4415-e4ec-36da-8441-375bdddc690b] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@35eec0c6. Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,134 WARN [Timer-Driven Process Thread-6] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=d2e3d4a1-3257-38a0-9e7b-2e0c5c7f7395] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@261dc487. Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=2fb57630-0d77-3947-954c-dc3b1fd6174e] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=4cdd0444-8047-3447-9f6c-f8f536b8228a] to run
> 2017-12-05 10:12:07,134 WARN [Timer-Driven Process Thread-1] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@4f649784. Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=d4372710-f550-3d31-10ae-8ae7e9300b44] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=df40050e-ebf1-368f-df61-43b5eb7bbdb6] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=8c979a96-03de-3beb-ec3a-1c91027162c9] to run
> {noformat}
> no Kerberos involved
> All configuration is default besides 'Kafka Brokers','Topic Name(s)','Group ID'
> The configuration file has been uploaded in the attachment 2.png
> the nifi-bootstrap.log is result of runing 'bin/nifi.sh dump'
> one consumeKafka processor -----------one topic-----------one partition --------------one replication-factor
> Every consumeKafka processor has a different topic.
> And i had tried out nifi 1.4.0,but it can not resolved the issues.
> thanks
> Sorry,my english is so bad.Please forgive me.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)