You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Apurva Sharma <ap...@gmail.com> on 2016/09/15 22:57:10 UTC
Partition creation issues (0.9.0.1)
Kafka Topic Creation issues (via Kafka-Manager with
auto.create.topics.enable = false)
Version: 0.9.0.1
We created a topic "web" via Kafka-Manager (our brokers are configured for
autocreate to be false) and then clicked on Generate Partitions and
according to the tool, the topic has been created cleanly with partitions
assigned correctly to brokers.
However, when we look into the individual broker logs:
[2016-09-15 18:46:10,268] ERROR [ReplicaFetcherThread-5-1006], Error for
partition [web,2] to broker
1006:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This
server does not host this topic-partition.
(kafka.server.ReplicaFetcherThread)
[2016-09-15 18:46:10,391] ERROR [ReplicaFetcherThread-4-1005], Error for
partition [web,1] to broker
1005:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This
server does not host this topic-partition.
(kafka.server.ReplicaFetcherThread)
[2016-09-15 18:46:10,391] WARN [Replica Manager on Broker 1001]: While
recording the replica LEO, the partition [web,5] hasn't been created.
(kafka.server.ReplicaManager)
[2016-09-15 18:46:10,407] ERROR [ReplicaFetcherThread-4-1004], Error for
partition [web,16] to broker
1004:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This
server does not host this topic-partition.
(kafka.server.ReplicaFetcherThread)
[2016-09-15 18:46:10,472] ERROR [ReplicaFetcherThread-11-1003], Error for
partition [web,23] to broker
1003:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This
server does not host this topic-partition.
(kafka.server.ReplicaFetcherThread)
[2016-09-15 18:46:10,520] WARN [Replica Manager on Broker 1001]: While
recording the replica LEO, the partition [web,29] hasn't been created.
(kafka.server.ReplicaManager)
[2016-09-15 18:46:10,667] ERROR [ReplicaFetcherThread-11-1004], Error for
partition [web,8] to broker
1004:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This
server does not host this topic-partition.
(kafka.server.ReplicaFetcherThread)
[2016-09-15 18:46:10,895] WARN [Replica Manager on Broker 1001]: While
recording the replica LEO, the partition [web,21] hasn't been created.
(kafka.server.ReplicaManager)
[2016-09-15 18:46:10,931] WARN [Replica Manager on Broker 1001]: While
recording the replica LEO, the partition [web,13] hasn't been created.
(kafka.server.ReplicaManager)
Indeed, when we actually inspect if the partitions were actually created,
we see disparity with what's reported by the tool. (Many partitions are
actually not present on the brokers yet - leading me to believe that the
partitioning process is stuck)
| ZK Id | Broker Id | According to Kafka Manager |
Actual Partitions Created |
|---------|----------------------------|--------------------------------------------|------------------------------------------------|
| 1001 | broker-1001.xxxx.xxxx.xxxx | 1,2,5,8,9,13,16,21,23,29,30,31 |
web-1,web-16,web-2,web-23,web-8,web-9 |
| 1002 | broker-1004.xxxx.xxxx.xxxx | 2,3,6,9,10,14,16,17,22,24,30,31 |
web-10,web-16,web-3,web-9 |
| 1003 | broker-1002.xxxx.xxxx.xxxx | 3,4,7,10,11,15,17,18,23,24,25,31 |
web-10,web-11,web-3,web-4 |
| 1004 | broker-1003.xxxx.xxxx.xxxx | 0,4,5,8,11,12,16,18,19,24,25,26 |
web-12,web-18,web-19,web-25,web-26 |
| 1005 | broker-1005.xxxx.xxxx.xxxx | 1,5,6,9,12,13,17,19,20,25,26,27 |
web-12,web-13,web-19,web-26,web-27,web-5 |
| 1006 | broker-1006.xxxx.xxxx.xxxx | 2,6,7,10,13,14,18,20,21,26,27,28 |
web-27 |
| 1007 | broker-1007.xxxx.xxxx.xxxx | 0,3,7,11,14,15,19,21,22,27,28,29 |
web-21,web-22,web-7 |
| 1008 | broker-1008.xxxx.xxxx.xxxx | 0,1,4,8,12,15,20,22,23,28,29,30 |
web-1,web-15,web-22,web-29,web-8 |
Finally, when we try to repartition the topic we get:
"Yikes! Partition reassignment currently in progress for. Aborting
operation" (This seems expected because the initial partitioning has not
completed for 3 days - so clearly it's stuck somewhere).
We have created topics with the Kafka-Manager tool before without issues,
but this has started happening consistently for the past 2-3 topics that we
tried creating through the tool.
Any help on this is greatly appreciated.
--
Regards,
Apurva
Re: Partition creation issues (0.9.0.1)
Posted by Apurva Sharma <ap...@gmail.com>.
Bumping for visibility - appreciate any responses on this.
Thanks,
Apurva
On Thu, Sep 15, 2016 at 3:57 PM, Apurva Sharma <ap...@gmail.com>
wrote:
>
> Kafka Topic Creation issues (via Kafka-Manager with
> auto.create.topics.enable = false)
> Version: 0.9.0.1
>
> We created a topic "web" via Kafka-Manager (our brokers are configured for
> autocreate to be false) and then clicked on Generate Partitions and
> according to the tool, the topic has been created cleanly with partitions
> assigned correctly to brokers.
> However, when we look into the individual broker logs:
>
> [2016-09-15 18:46:10,268] ERROR [ReplicaFetcherThread-5-1006], Error for
> partition [web,2] to broker 1006:org.apache.kafka.common.errors.
> UnknownTopicOrPartitionException: This server does not host this
> topic-partition. (kafka.server.ReplicaFetcherThread)
> [2016-09-15 18:46:10,391] ERROR [ReplicaFetcherThread-4-1005], Error for
> partition [web,1] to broker 1005:org.apache.kafka.common.errors.
> UnknownTopicOrPartitionException: This server does not host this
> topic-partition. (kafka.server.ReplicaFetcherThread)
> [2016-09-15 18:46:10,391] WARN [Replica Manager on Broker 1001]: While
> recording the replica LEO, the partition [web,5] hasn't been created.
> (kafka.server.ReplicaManager)
> [2016-09-15 18:46:10,407] ERROR [ReplicaFetcherThread-4-1004], Error for
> partition [web,16] to broker 1004:org.apache.kafka.common.errors.
> UnknownTopicOrPartitionException: This server does not host this
> topic-partition. (kafka.server.ReplicaFetcherThread)
> [2016-09-15 18:46:10,472] ERROR [ReplicaFetcherThread-11-1003], Error for
> partition [web,23] to broker 1003:org.apache.kafka.common.errors.
> UnknownTopicOrPartitionException: This server does not host this
> topic-partition. (kafka.server.ReplicaFetcherThread)
> [2016-09-15 18:46:10,520] WARN [Replica Manager on Broker 1001]: While
> recording the replica LEO, the partition [web,29] hasn't been created.
> (kafka.server.ReplicaManager)
> [2016-09-15 18:46:10,667] ERROR [ReplicaFetcherThread-11-1004], Error for
> partition [web,8] to broker 1004:org.apache.kafka.common.errors.
> UnknownTopicOrPartitionException: This server does not host this
> topic-partition. (kafka.server.ReplicaFetcherThread)
> [2016-09-15 18:46:10,895] WARN [Replica Manager on Broker 1001]: While
> recording the replica LEO, the partition [web,21] hasn't been created.
> (kafka.server.ReplicaManager)
> [2016-09-15 18:46:10,931] WARN [Replica Manager on Broker 1001]: While
> recording the replica LEO, the partition [web,13] hasn't been created.
> (kafka.server.ReplicaManager)
>
> Indeed, when we actually inspect if the partitions were actually created,
> we see disparity with what's reported by the tool. (Many partitions are
> actually not present on the brokers yet - leading me to believe that the
> partitioning process is stuck)
>
>
> | ZK Id | Broker Id | According to Kafka Manager |
> Actual Partitions Created |
> |---------|----------------------------|--------------------
> ------------------------|------------------------------------------------|
> | 1001 | broker-1001.xxxx.xxxx.xxxx | 1,2,5,8,9,13,16,21,23,29,30,31 |
> web-1,web-16,web-2,web-23,web-8,web-9 |
> | 1002 | broker-1004.xxxx.xxxx.xxxx | 2,3,6,9,10,14,16,17,22,24,30,31 |
> web-10,web-16,web-3,web-9 |
> | 1003 | broker-1002.xxxx.xxxx.xxxx | 3,4,7,10,11,15,17,18,23,24,25,31 |
> web-10,web-11,web-3,web-4 |
> | 1004 | broker-1003.xxxx.xxxx.xxxx | 0,4,5,8,11,12,16,18,19,24,25,26 |
> web-12,web-18,web-19,web-25,web-26 |
> | 1005 | broker-1005.xxxx.xxxx.xxxx | 1,5,6,9,12,13,17,19,20,25,26,27 |
> web-12,web-13,web-19,web-26,web-27,web-5 |
> | 1006 | broker-1006.xxxx.xxxx.xxxx | 2,6,7,10,13,14,18,20,21,26,27,28 |
> web-27 |
> | 1007 | broker-1007.xxxx.xxxx.xxxx | 0,3,7,11,14,15,19,21,22,27,28,29 |
> web-21,web-22,web-7 |
> | 1008 | broker-1008.xxxx.xxxx.xxxx | 0,1,4,8,12,15,20,22,23,28,29,30 |
> web-1,web-15,web-22,web-29,web-8 |
>
> Finally, when we try to repartition the topic we get:
> "Yikes! Partition reassignment currently in progress for. Aborting
> operation" (This seems expected because the initial partitioning has not
> completed for 3 days - so clearly it's stuck somewhere).
>
> We have created topics with the Kafka-Manager tool before without issues,
> but this has started happening consistently for the past 2-3 topics that we
> tried creating through the tool.
> Any help on this is greatly appreciated.
>
> --
> Regards,
> Apurva
>
--
Regards,
Apurva