You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/04/01 06:54:02 UTC
[GitHub] [druid] wanglihui-git opened a new issue #12385: When a kafka node goes down in single-replica topic mode, the Kafka indexing task cannot be started
wanglihui-git opened a new issue #12385:
URL: https://github.com/apache/druid/issues/12385
### Affected Version
0.18.1——0.22.1
### Description
Due to the large amount of data in the production environment, our kafka cluster had to use a single-replica topic. When a kafka node goes down, the kafka indexing task cannot be started. The normal running Supervisor can still run continuously, but after the reset operation, it can't run either.
If this happens in the production environment, and the kafka node is down and cannot be recovered in a short time, how can the Druid task increase the reliability of it?
The following is a screenshot of my test. The error message is: 'Timeout of 60000ms expired before the position for partition topic-0 could be determined'.After a while, the Supervisors state changed to 'LOST_CONTACT_WITH_STREAM'.
![Uploading image.png…]()
Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] wanglihui-git commented on issue #12385: When a kafka node goes down in single-replica topic mode, the Kafka indexing task cannot be started
Posted by GitBox <gi...@apache.org>.
wanglihui-git commented on issue #12385:
URL: https://github.com/apache/druid/issues/12385#issuecomment-1086557387
> There is nothing you can do on the druid side to increase reliability if the data source itself is unavailable. aren't the producers running into the very same problem that druid is running into?
Kafka node downtime seems to have no effect on the producer, we are using kafka 1.0.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] abhishekagarwal87 commented on issue #12385: When a kafka node goes down in single-replica topic mode, the Kafka indexing task cannot be started
Posted by GitBox <gi...@apache.org>.
abhishekagarwal87 commented on issue #12385:
URL: https://github.com/apache/druid/issues/12385#issuecomment-1085677990
There is nothing you can do on the druid side to increase reliability if the data source itself is unavailable. aren't the producers running into the very same problem that druid is running into?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org