You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Jun Rao (JIRA)" <ji...@apache.org> on 2012/10/16 18:19:03 UTC
[jira] [Created] (KAFKA-575) Partition.makeFollower() reads broker
info from ZK
Jun Rao created KAFKA-575:
-----------------------------
Summary: Partition.makeFollower() reads broker info from ZK
Key: KAFKA-575
URL: https://issues.apache.org/jira/browse/KAFKA-575
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 0.8
Reporter: Jun Rao
Priority: Blocker
To follow a new leader, Partition.makeFollower() has to obtain the broker info of the new leader. Currently, it reads that info from ZK for every affected partition. This increases the time for a leader to truly available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-575) Partition.makeFollower() reads broker
info from ZK
Posted by "Swapnil Ghike (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Swapnil Ghike updated KAFKA-575:
--------------------------------
Attachment: kafka-575-v3.patch
Minor change + passed sanity test.
> Partition.makeFollower() reads broker info from ZK
> --------------------------------------------------
>
> Key: KAFKA-575
> URL: https://issues.apache.org/jira/browse/KAFKA-575
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8
> Reporter: Jun Rao
> Assignee: Swapnil Ghike
> Priority: Blocker
> Labels: bugs
> Attachments: kafka-575-v1.patch, kafka-575-v2.patch, kafka-575-v3.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> To follow a new leader, Partition.makeFollower() has to obtain the broker info of the new leader. Currently, it reads that info from ZK for every affected partition. This increases the time for a leader to truly available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-575) Partition.makeFollower() reads broker
info from ZK
Posted by "Swapnil Ghike (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Swapnil Ghike updated KAFKA-575:
--------------------------------
Attachment: kafka-575-v2.patch
Made the change.
> Partition.makeFollower() reads broker info from ZK
> --------------------------------------------------
>
> Key: KAFKA-575
> URL: https://issues.apache.org/jira/browse/KAFKA-575
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8
> Reporter: Jun Rao
> Assignee: Swapnil Ghike
> Priority: Blocker
> Labels: bugs
> Attachments: kafka-575-v1.patch, kafka-575-v2.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> To follow a new leader, Partition.makeFollower() has to obtain the broker info of the new leader. Currently, it reads that info from ZK for every affected partition. This increases the time for a leader to truly available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (KAFKA-575) Partition.makeFollower() reads broker
info from ZK
Posted by "Swapnil Ghike (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Swapnil Ghike reassigned KAFKA-575:
-----------------------------------
Assignee: Swapnil Ghike
> Partition.makeFollower() reads broker info from ZK
> --------------------------------------------------
>
> Key: KAFKA-575
> URL: https://issues.apache.org/jira/browse/KAFKA-575
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8
> Reporter: Jun Rao
> Assignee: Swapnil Ghike
> Priority: Blocker
> Labels: bugs
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> To follow a new leader, Partition.makeFollower() has to obtain the broker info of the new leader. Currently, it reads that info from ZK for every affected partition. This increases the time for a leader to truly available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-575) Partition.makeFollower() reads broker
info from ZK
Posted by "Swapnil Ghike (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Swapnil Ghike updated KAFKA-575:
--------------------------------
Attachment: kafka-575-v3-correct.patch
Sorry, please ignore the last patch. This one is the correct one.
> Partition.makeFollower() reads broker info from ZK
> --------------------------------------------------
>
> Key: KAFKA-575
> URL: https://issues.apache.org/jira/browse/KAFKA-575
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8
> Reporter: Jun Rao
> Assignee: Swapnil Ghike
> Priority: Blocker
> Labels: bugs
> Attachments: kafka-575-v1.patch, kafka-575-v2.patch, kafka-575-v3-correct.patch, kafka-575-v3.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> To follow a new leader, Partition.makeFollower() has to obtain the broker info of the new leader. Currently, it reads that info from ZK for every affected partition. This increases the time for a leader to truly available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-575) Partition.makeFollower() reads
broker info from ZK
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486079#comment-13486079 ]
Jun Rao commented on KAFKA-575:
-------------------------------
Patch looks good. Just one comment:
1. ControllerBrokerRequestBatch.sendRequestsToBrokers(): Instead of including all live brokers in the leaderAndIsr request, we probably can just include brokers that are leaders in each LeaderAndIsr request.
> Partition.makeFollower() reads broker info from ZK
> --------------------------------------------------
>
> Key: KAFKA-575
> URL: https://issues.apache.org/jira/browse/KAFKA-575
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8
> Reporter: Jun Rao
> Assignee: Swapnil Ghike
> Priority: Blocker
> Labels: bugs
> Attachments: kafka-575-v1.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> To follow a new leader, Partition.makeFollower() has to obtain the broker info of the new leader. Currently, it reads that info from ZK for every affected partition. This increases the time for a leader to truly available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (KAFKA-575) Partition.makeFollower() reads broker
info from ZK
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jun Rao resolved KAFKA-575.
---------------------------
Resolution: Fixed
Fix Version/s: 0.8
Thanks for patch v3. +1. Committed to 0.8.
> Partition.makeFollower() reads broker info from ZK
> --------------------------------------------------
>
> Key: KAFKA-575
> URL: https://issues.apache.org/jira/browse/KAFKA-575
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8
> Reporter: Jun Rao
> Assignee: Swapnil Ghike
> Priority: Blocker
> Labels: bugs
> Fix For: 0.8
>
> Attachments: kafka-575-v1.patch, kafka-575-v2.patch, kafka-575-v3-correct.patch, kafka-575-v3.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> To follow a new leader, Partition.makeFollower() has to obtain the broker info of the new leader. Currently, it reads that info from ZK for every affected partition. This increases the time for a leader to truly available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-575) Partition.makeFollower() reads broker
info from ZK
Posted by "Swapnil Ghike (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Swapnil Ghike updated KAFKA-575:
--------------------------------
Attachment: kafka-575-v1.patch
1. Live broker info included in LeaderIsrRequest.
ControllerBrokerRequestBatch.sendrequestsToBrokers() passes the live brokers list to LeaderIsrRequest.
Thus Partition.makeFollower no longer needs to read the broker info from zookeeper. Only the controller reads this info from zookeeper with this patch.
2. Partition.makeLeaderOrFollower feels unnecessary because it reuses only a couple lines of code, and introduces an extra step in understanding the logic.
Hence, got rid of makeLeaderOrFollower and copied the lock synchronization and discarding the incoming request part to makeLeader and makeFollower separately.
> Partition.makeFollower() reads broker info from ZK
> --------------------------------------------------
>
> Key: KAFKA-575
> URL: https://issues.apache.org/jira/browse/KAFKA-575
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8
> Reporter: Jun Rao
> Assignee: Swapnil Ghike
> Priority: Blocker
> Labels: bugs
> Attachments: kafka-575-v1.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> To follow a new leader, Partition.makeFollower() has to obtain the broker info of the new leader. Currently, it reads that info from ZK for every affected partition. This increases the time for a leader to truly available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (KAFKA-575) Partition.makeFollower() reads broker
info from ZK
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jun Rao closed KAFKA-575.
-------------------------
> Partition.makeFollower() reads broker info from ZK
> --------------------------------------------------
>
> Key: KAFKA-575
> URL: https://issues.apache.org/jira/browse/KAFKA-575
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8
> Reporter: Jun Rao
> Assignee: Swapnil Ghike
> Priority: Blocker
> Labels: bugs
> Fix For: 0.8
>
> Attachments: kafka-575-v1.patch, kafka-575-v2.patch, kafka-575-v3-correct.patch, kafka-575-v3.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> To follow a new leader, Partition.makeFollower() has to obtain the broker info of the new leader. Currently, it reads that info from ZK for every affected partition. This increases the time for a leader to truly available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (KAFKA-575) Partition.makeFollower() reads
broker info from ZK
Posted by "Jun Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477125#comment-13477125 ]
Jun Rao commented on KAFKA-575:
-------------------------------
One way to address this issue is to include the broker info in the leaderAndIsr request. This way, Partition.makeFollower() doesn't need to read from ZK any more.
> Partition.makeFollower() reads broker info from ZK
> --------------------------------------------------
>
> Key: KAFKA-575
> URL: https://issues.apache.org/jira/browse/KAFKA-575
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8
> Reporter: Jun Rao
> Priority: Blocker
> Labels: bugs
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> To follow a new leader, Partition.makeFollower() has to obtain the broker info of the new leader. Currently, it reads that info from ZK for every affected partition. This increases the time for a leader to truly available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira