You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Yevgeniy Korin (Jira)" <ji...@apache.org> on 2021/10/21 10:56:00 UTC
[jira] [Updated] (KAFKA-13392) Timeout Exception triggering
reassign partitions with --bootstrap-server option
[ https://issues.apache.org/jira/browse/KAFKA-13392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yevgeniy Korin updated KAFKA-13392:
-----------------------------------
Description:
*Scenario when we faced with this issue:*
One of three brokers is down. Add another (fourth) broker and try to reassign partitions using '--bootstrap-server'
option.
*What's failed:*
{code:java}
/opt/kafka/bin/kafka-reassign-partitions.sh --bootstrap-server xxx.xxx.xxx.xxx:9092 --reassignment-json-file /tmp/reassignment-20211021130718.json --throttle 100000000 --execute{code}
failed with
{code:java}
Error: org.apache.kafka.common.errors.TimeoutException: Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1, nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1, nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
at kafka.admin.ReassignPartitionsCommand$.modifyInterBrokerThrottle(ReassignPartitionsCommand.scala:1435)
at kafka.admin.ReassignPartitionsCommand$.modifyReassignmentThrottle(ReassignPartitionsCommand.scala:1412)
at kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:974)
at kafka.admin.ReassignPartitionsCommand$.handleAction(ReassignPartitionsCommand.scala:255)
at kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:216)
at kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1, nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: incrementalAlterConfigs{code}
*Expected behavio**:*
partition reassignment process started.
*Workaround:*
Trigger partition reassignment process using '--zookeeper' option:
{code:java}
/opt/kafka/bin/kafka-reassign-partitions.sh --zookeeper zookeeper.my.company:2181/kafka-cluster --reassignment-json-file /tmp/reassignment-20211021130718.json --throttle 100000000 --execute{code}
*Additional info:*
We are able to trigger partition reassignment using '--bootstrap-server' option with no exceptions when all four brokers are alive.
was:
*Scenario when we faced with this issue:*
One of three brokers is down. Add another (fourth) broker and try to reassign partitions using `--bootstrap-server` option.
*What's failed:*
`/opt/kafka/bin/kafka-reassign-partitions.sh --bootstrap-server xxx.xxx.xxx.xxx:9092 --reassignment-json-file /tmp/reassignment-20211021130718.json --throttle 100000000 --execute` failed with
Error: org.apache.kafka.common.errors.TimeoutException: Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1, nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1, nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
at kafka.admin.ReassignPartitionsCommand$.modifyInterBrokerThrottle(ReassignPartitionsCommand.scala:1435)
at kafka.admin.ReassignPartitionsCommand$.modifyReassignmentThrottle(ReassignPartitionsCommand.scala:1412)
at kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:974)
at kafka.admin.ReassignPartitionsCommand$.handleAction(ReassignPartitionsCommand.scala:255)
at kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:216)
at kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1, nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: incrementalAlterConfigs
*Expected behavio**:*
partition reassignment process started.
*Workaround:*
Trigger partition reassignment process using `–zookeeper` option:
/opt/kafka/bin/kafka-reassign-partitions.sh --zookeeper zookeeper.my.company:2181/kafka-cluster --reassignment-json-file /tmp/reassignment-20211021130718.json --throttle 100000000 --execute
Additional info:
We were able to trigger partition reassignment using `--bootstrap-server` option when all four brokers are alive.
> Timeout Exception triggering reassign partitions with --bootstrap-server option
> -------------------------------------------------------------------------------
>
> Key: KAFKA-13392
> URL: https://issues.apache.org/jira/browse/KAFKA-13392
> Project: Kafka
> Issue Type: Bug
> Components: admin
> Affects Versions: 2.8.0
> Reporter: Yevgeniy Korin
> Priority: Minor
>
> *Scenario when we faced with this issue:*
> One of three brokers is down. Add another (fourth) broker and try to reassign partitions using '--bootstrap-server'
> option.
> *What's failed:*
> {code:java}
> /opt/kafka/bin/kafka-reassign-partitions.sh --bootstrap-server xxx.xxx.xxx.xxx:9092 --reassignment-json-file /tmp/reassignment-20211021130718.json --throttle 100000000 --execute{code}
> failed with
> {code:java}
> Error: org.apache.kafka.common.errors.TimeoutException: Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1, nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
> java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1, nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
> at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
> at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
> at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
> at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
> at kafka.admin.ReassignPartitionsCommand$.modifyInterBrokerThrottle(ReassignPartitionsCommand.scala:1435)
> at kafka.admin.ReassignPartitionsCommand$.modifyReassignmentThrottle(ReassignPartitionsCommand.scala:1412)
> at kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:974)
> at kafka.admin.ReassignPartitionsCommand$.handleAction(ReassignPartitionsCommand.scala:255)
> at kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:216)
> at kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
> Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1, nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
> Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: incrementalAlterConfigs{code}
> *Expected behavio**:*
> partition reassignment process started.
> *Workaround:*
> Trigger partition reassignment process using '--zookeeper' option:
> {code:java}
> /opt/kafka/bin/kafka-reassign-partitions.sh --zookeeper zookeeper.my.company:2181/kafka-cluster --reassignment-json-file /tmp/reassignment-20211021130718.json --throttle 100000000 --execute{code}
> *Additional info:*
> We are able to trigger partition reassignment using '--bootstrap-server' option with no exceptions when all four brokers are alive.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)