You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Matthias J. Sax (Jira)" <ji...@apache.org> on 2020/03/05 05:18:00 UTC
[jira] [Updated] (KAFKA-9645) Records could not find corresponding
partition/task
[ https://issues.apache.org/jira/browse/KAFKA-9645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias J. Sax updated KAFKA-9645:
-----------------------------------
Component/s: (was: streams)
consumer
> Records could not find corresponding partition/task
> ---------------------------------------------------
>
> Key: KAFKA-9645
> URL: https://issues.apache.org/jira/browse/KAFKA-9645
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 2.6.0
> Reporter: Boyang Chen
> Assignee: Boyang Chen
> Priority: Major
>
> We could be hitting the illegal state when stream kicks off a rebalance with all tasks closed:
> ```
> [2020-03-03T18:36:09-08:00] (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04 02:36:09,105] WARN [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] stream-thread [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] Detected that the thread is being fenced. This implies that this thread missed a rebalance and dropped out of the consumer group. Will close out all assigned tasks and rejoin the consumer group. (org.apache.kafka.streams.processor.internals.StreamThread)
> [2020-03-03T18:36:09-08:00] (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04 02:36:09,105] INFO [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] [Consumer clientId=stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1-restore-consumer, groupId=null] Subscribed to partition(s): stream-soak-test-KSTREAM-AGGREGATE-STATE-STORE-0000000049-changelog-2 (org.apache.kafka.clients.consumer.KafkaConsumer)
> [2020-03-03T18:36:10-08:00] (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04 02:36:09,286] INFO [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] [Producer clientId=stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1-1_1-producer, transactionalId=stream-soak-test-1_1] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms. (org.apache.kafka.clients.producer.KafkaProducer)
> [2020-03-03T18:36:10-08:00] (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04 02:36:09,287] INFO [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] stream-thread [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] task [1_1] Closed dirty (org.apache.kafka.streams.processor.internals.StreamTask)
> [2020-03-03T18:36:10-08:00] (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04 02:36:09,287] INFO [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] [Consumer clientId=stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1-restore-consumer, groupId=null] Unsubscribed all topics or patterns and assigned partitions (org.apache.kafka.clients.consumer.KafkaConsumer)
> [2020-03-03T18:36:10-08:00] (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04 02:36:09,290] INFO [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] [Producer clientId=stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1-3_2-producer, transactionalId=stream-soak-test-3_2] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms. (org.apache.kafka.clients.producer.KafkaProducer)
> [2020-03-03T18:36:10-08:00] (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04 02:36:09,292] INFO [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] stream-thread [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] task [3_2] Closed dirty (org.apache.kafka.streams.processor.internals.StreamTask)
> [2020-03-03T18:36:10-08:00] (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04 02:36:09,293] ERROR [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] stream-thread [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] Unable to locate active task for received-record partition node-name-repartition-1. Current tasks: TaskManager
> [2020-03-03T18:36:10-08:00] (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) > MetadataState:
> [2020-03-03T18:36:10-08:00] (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) > Tasks:
> (org.apache.kafka.streams.processor.internals.StreamThread)
> [2020-03-03T18:36:10-08:00] (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04 02:36:09,293] ERROR [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] stream-thread [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] Encountered the following exception during processing and the thread is going to shut down: (org.apache.kafka.streams.processor.internals.StreamThread)
> [2020-03-03T18:36:10-08:00] (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) java.lang.NullPointerException: Task was unexpectedly missing for partition node-name-repartition-1
> at org.apache.kafka.streams.processor.internals.StreamThread.addRecordsToTasks(StreamThread.java:984)
> at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:820)
> ```
> We should be more careful in this case by avoiding processing data when no corresponding task is found.
> at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:751)
> at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:725)
> [2020-03-03T18:36:10-08:00] (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04 02:36:09,294] INFO [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] stream-thread [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] State transition from RUNNING to PENDING_SHUTDOWN (org.apache.kafka.streams.processor.internals.StreamThread)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)