You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by arrawatia <gi...@git.apache.org> on 2016/09/07 07:45:23 UTC

[GitHub] kafka pull request #1830: [WIP] KIP-78 : Cluster Identifier (KAFKA-4093)

GitHub user arrawatia opened a pull request:

    https://github.com/apache/kafka/pull/1830

    [WIP] KIP-78 : Cluster Identifier (KAFKA-4093)

    This is a first draft that includes the following changes:
    1. Changes to the broker code. 
      - generation of cluster id and storing it in Zookeeper
      - update protocol to add cluster id to metadata request and response
      - add ClusterResourceListener interface and ClusterMetadataListeners utility class
      - update broker code to sending ClusterResource events to the metric reporters
    2. Changes to client code.
      - update Cluster and Metadata code
      - update clients for sending ClusterResource events to interceptors, de(serializers) and metric reporters
      - update tests for interceptors
    
    Changes that still need to be made:
    1. Update / create integration tests for Serializers and MetricReporter for clients. Integration tests for protocol changes and MetricReporters for broker.
    2. Upgrade system tests.
    
    @ijuma Can you please take a look ?

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/arrawatia/kafka kip-78

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/1830.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1830
    
----
commit 880f903e161a29d6daec5ad0e85e190b14023310
Author: arrawatia <su...@gmail.com>
Date:   2016-08-29T23:13:26Z

    First cut for cluster id.
    
    1. Add v2 for Metadata Request and Response in Protocol to include
    cluster_id
    2. Updated the MetadataRequest and MetadataResponse to add cluster_id
    and Update tests
    3. Add cluster_id to KafkaServer and KafkaApis
    4. cluster_id = dummy for now.

commit e76de82ab12b6d4b2f77905536a79c1178b3b378
Author: arrawatia <su...@gmail.com>
Date:   2016-08-30T01:24:24Z

    Broker Id generation.

commit 4bd44b94112c9cf09fe1819618adc81072f81ad6
Author: arrawatia <su...@gmail.com>
Date:   2016-08-30T05:35:02Z

    Change ZK representation to JSON from string.

commit 6efc32f3ec1e91c88428847a91fd5fc7da312e01
Author: arrawatia <su...@gmail.com>
Date:   2016-08-30T14:52:52Z

    Add tests and change from using internal sun classes to JAXB for base64 en(de)coding

commit 2a417a8da218a5113ef4261ba2e429007a6818bd
Author: arrawatia <su...@gmail.com>
Date:   2016-08-31T17:53:07Z

    Make cluster_id nullable and change NO_CLUSTER_ID to null.

commit e91da1e8b33ff7fb716a5151c194d2192768ec23
Author: arrawatia <su...@gmail.com>
Date:   2016-09-01T16:58:11Z

    First pass for client changes.

commit 93beebac65bce7a2f9ce0ab07b628db2b2131e9e
Author: arrawatia <su...@gmail.com>
Date:   2016-09-02T08:07:39Z

    Add cluster.id to meta.properties

commit 7823bc14f26d685adbdac0fdb6c6ae86615addd2
Author: arrawatia <su...@gmail.com>
Date:   2016-09-02T08:11:56Z

    Change to ClusterListerners utility class.

commit 170ce9272bda1b5e6d0a90395e6a9cfdb6b41c58
Author: arrawatia <su...@gmail.com>
Date:   2016-09-02T17:54:37Z

    Merge remote-tracking branch 'confluentinc/trunk' into kip-78

commit eca2ab1381cbf5b9597031a5f9879298e67a85a1
Author: arrawatia <su...@gmail.com>
Date:   2016-09-02T18:38:17Z

    Revert change for adding cluster.id to meta.properties.
    
    Wait for it to be included in the KIP.

commit 0753422d1ab0f99f25047ecbcc8ae6c126015de7
Author: arrawatia <su...@gmail.com>
Date:   2016-09-02T18:38:44Z

    Make check style tests pass.

commit fe0e5571d60a997705f694db75437afd4c069a65
Author: arrawatia <su...@gmail.com>
Date:   2016-09-02T18:50:20Z

    Rename ClusterListener -> ClusterResourceListener and ClusterResourceMeta -> ClusterResource.

commit 83a4b01d1a4373603ae5a46195029602c3384cef
Author: arrawatia <su...@gmail.com>
Date:   2016-09-03T06:38:31Z

    Add ClusterListeners on KafkaServer to send events to MetricReporters(both Codahale and KafkaMetricrReporter).

commit 6782cd5ebf71de3c1c09542494f9e27064da3c0c
Author: arrawatia <su...@gmail.com>
Date:   2016-09-07T00:57:25Z

    Update consumer tests to test for cluster id in the correct order.

commit 29e9c83a3a0c6b6ff0588a562d5f9f9712ffd35c
Author: arrawatia <su...@gmail.com>
Date:   2016-09-07T00:59:19Z

    Make check style tests pass.

commit 71df287d48a4a26422f7fb45227abd4267bc362d
Author: arrawatia <su...@gmail.com>
Date:   2016-09-07T00:59:29Z

    Merge remote-tracking branch 'confluentinc/trunk' into kip-78

commit 1ee9666bfcce9c4a6a1ff8bff2c46a88c3e20dc5
Author: arrawatia <su...@gmail.com>
Date:   2016-09-07T01:30:17Z

    Merge branch 'trunk' of github.com:apache/kafka into kip-78

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] kafka pull request #1830: KAFKA-4093 : Cluster Identifiers (KIP-78)

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/kafka/pull/1830


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---