You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Joseph Witt (JIRA)" <ji...@apache.org> on 2017/12/06 07:30:00 UTC
[jira] [Commented] (NIFI-4670) when i start consumerKafka processor
more than eleven,all of the processors get into an indefinite stuck state
[ https://issues.apache.org/jira/browse/NIFI-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16279785#comment-16279785 ]
Joseph Witt commented on NIFI-4670:
-----------------------------------
hello - we'll need you to share a lot more details about the configuration. Is Kerberos involved? What are the settings on the kafka processors?
When the threads are stuck please run 'bin/nifi.sh dump' so we can see the thread dumps by you sharing the logs/nifi-bootstrap.log
thanks
> when i start consumerKafka processor more than eleven,all of the processors get into an indefinite stuck state
> --------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-4670
> URL: https://issues.apache.org/jira/browse/NIFI-4670
> Project: Apache NiFi
> Issue Type: Task
> Affects Versions: 1.3.0
> Environment: linux
> Reporter: clearlove
> Priority: Critical
> Labels: beginner, test
> Attachments: 1.png
>
>
> When i start one consumeKafka_0_10 processor ,all of the processors are all right.
> but when i start consumeKafka_0_10 processor more than eleven,all of the processors get into an indefinite stuck state.
> !1.png|thumbnail!
> like in the picture ,look like everything is all right, but all processor(even processor is not consumeKafka_0_10 ) are stop work.All of the flowfile don't move.
> And when i stop them , some of consumeKafka_0_10 processor are warning
> {{Was interrupted while trying to communicate with Kafka with lease {}. Will roll back session and discard any partially received data.}}
> in the nifi-app.log
> {noformat}
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=099a83f6-7ddc-3df8-fe07-5b901799ade3] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.SplitJson
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=fdd4850d-a619-3f17-6287-198cac3bb52a] to run
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling SplitJson[id=f3f85270-6bb8-3f65-4347-6700426d6a2e] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=6aacc9a9-c7e8-32b0-ed3c-3f89e72c30cf] to run
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,130 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=02aaae90-5ada-343e-6c15-3cfff7cfded0] to run
> 2017-12-05 10:12:02,130 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=278dc7a5-6d2e-336a-45c3-56cb7039dc7c] to run
> 2017-12-05 10:12:02,131 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=60fd28d8-b5f0-3a5b-e31a-e3ba7b73d7cb] to run
> 2017-12-05 10:12:02,131 INFO [StandardProcessScheduler Thread-2] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=efa69212-d0c9-3c83-0985-b22115b5e861] to run
> 2017-12-05 10:12:02,724 INFO [Flow Service Tasks Thread-1] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@a987d5e // Another save pending = false
> 2017-12-05 10:12:07,130 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=2fedc410-9641-3bda-cc40-59cf739a5400] to run
> 2017-12-05 10:12:07,130 WARN [Timer-Driven Process Thread-8] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=a2b469df-a2ef-355f-cc68-7b5dea6abb8b] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@2b5a2eed. Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,130 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=d925dcdc-edef-3184-2185-47a0ebf3ea3b] to run
> 2017-12-05 10:12:07,131 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=8b6fef2d-3db8-3558-89a3-abce7b0e8797] to run
> 2017-12-05 10:12:07,131 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=e1af6798-96ba-31ec-c135-ff2f10d1306b] to run
> 2017-12-05 10:12:07,131 WARN [Timer-Driven Process Thread-7] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=d4b01bbd-62a3-3584-8391-371d52b83bce] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@7060d90d. Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,132 INFO [StandardProcessScheduler Thread-8] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=b88105fe-cb1e-3c4e-f91d-22f5542cecd7] to run
> 2017-12-05 10:12:07,132 WARN [Timer-Driven Process Thread-2] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=4c067c00-fbc2-3571-5b7a-37b4957afb87] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@158837ab. Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=182347e6-14ef-38c2-6873-5602f0405296] to run
> 2017-12-05 10:12:07,133 WARN [Timer-Driven Process Thread-3] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=4035760a-8b9e-3e34-51e9-e292c58e6ed7] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@4967b312. Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=f7285790-6159-3726-662e-5717235a31dc] to run
> 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-1] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConsumeKafka_0_10[id=fae14360-943d-3a81-f3b1-675f5505e265] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=86b794b6-5875-3eac-9fdb-447743459b7d] to run
> 2017-12-05 10:12:07,133 WARN [Timer-Driven Process Thread-9] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=716f4415-e4ec-36da-8441-375bdddc690b] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@35eec0c6. Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,134 WARN [Timer-Driven Process Thread-6] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=d2e3d4a1-3257-38a0-9e7b-2e0c5c7f7395] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@261dc487. Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling EvaluateJsonPath[id=2fb57630-0d77-3947-954c-dc3b1fd6174e] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=4cdd0444-8047-3447-9f6c-f8f536b8228a] to run
> 2017-12-05 10:12:07,134 WARN [Timer-Driven Process Thread-1] o.a.n.p.kafka.pubsub.ConsumeKafka_0_10 ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] Was interrupted while trying to communicate with Kafka with lease org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@4f649784. Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ReplaceText[id=d4372710-f550-3d31-10ae-8ae7e9300b44] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=df40050e-ebf1-368f-df61-43b5eb7bbdb6] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling PutSQL[id=8c979a96-03de-3beb-ec3a-1c91027162c9] to run
> {noformat}
> Sorry,my english is so bad.Please forgive me.:D
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)