You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Aditya Auradkar <aa...@linkedin.com> on 2015/07/01 04:44:05 UTC
Re: Review Request 33378: Patch for KAFKA-2136
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/
-----------------------------------------------------------
(Updated July 1, 2015, 2:44 a.m.)
Review request for kafka, Joel Koshy and Jun Rao.
Bugs: KAFKA-2136
https://issues.apache.org/jira/browse/KAFKA-2136
Repository: kafka
Description (updated)
-------
Addressing Joel's comments
Merging
Chaning variable name
Addressing Joel's comments
Diffs (updated)
-----
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 56281ee15cc33dfc96ff64d5b1e596047c7132a4
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 07e65d4a54ba4eef5b787eba3e71cbe9f6a920bd
clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 8686d83aa52e435c6adafbe9ff4bd1602281072a
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java fabeae3083a8ea55cdacbb9568f3847ccd85bab4
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 37ec0b79beafcf5735c386b066eb319fb697eff5
clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java 419541011d652becf0cda7a5e62ce813cddb1732
clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java e3cc1967e407b64cc734548c19e30de700b64ba8
core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
core/src/main/scala/kafka/consumer/SimpleConsumer.scala c16f7edd322709060e54c77eb505c44cbd77a4ec
core/src/main/scala/kafka/server/AbstractFetcherThread.scala 83fc47417dd7fe4edf030217fa7fd69d99b170b0
core/src/main/scala/kafka/server/DelayedFetch.scala de6cf5bdaa0e70394162febc63b50b55ca0a92db
core/src/main/scala/kafka/server/DelayedProduce.scala 05078b24ef28f2f4e099afa943e43f1d00359fda
core/src/main/scala/kafka/server/KafkaApis.scala d63bc18d795a6f0e6994538ca55a9a46f7fb8ffd
core/src/main/scala/kafka/server/OffsetManager.scala 5cca85cf727975f6d3acb2223fd186753ad761dc
core/src/main/scala/kafka/server/ReplicaFetcherThread.scala b31b432a226ba79546dd22ef1d2acbb439c2e9a3
core/src/main/scala/kafka/server/ReplicaManager.scala 59c9bc3ac3a8afc07a6f8c88c5871304db588d17
core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 5717165f2344823fabe8f7cfafae4bb8af2d949a
core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79
Diff: https://reviews.apache.org/r/33378/diff/
Testing
-------
New tests added
Thanks,
Aditya Auradkar
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
> On Aug. 5, 2015, 4:44 a.m., Jun Rao wrote:
> > clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java, line 55
> > <https://reviews.apache.org/r/33378/diff/10/?file=1010009#file1010009line55>
> >
> > We actually will need different constructors for different versions. We want to reuse those request/response objects on the broker side. So, the broker will need to construct different version of the response based on the version of the request. You can take a look at OffsetCommitRequest as an example.
Good point.
- Aditya
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review94167
-----------------------------------------------------------
On July 13, 2015, 8:36 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated July 13, 2015, 8:36 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel's comments
>
> For now the patch will publish a zero delay and return a response
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 56281ee15cc33dfc96ff64d5b1e596047c7132a4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 07e65d4a54ba4eef5b787eba3e71cbe9f6a920bd
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 8686d83aa52e435c6adafbe9ff4bd1602281072a
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java fabeae3083a8ea55cdacbb9568f3847ccd85bab4
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 37ec0b79beafcf5735c386b066eb319fb697eff5
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java 419541011d652becf0cda7a5e62ce813cddb1732
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java e3cc1967e407b64cc734548c19e30de700b64ba8
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala c16f7edd322709060e54c77eb505c44cbd77a4ec
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala 83fc47417dd7fe4edf030217fa7fd69d99b170b0
> core/src/main/scala/kafka/server/DelayedFetch.scala de6cf5bdaa0e70394162febc63b50b55ca0a92db
> core/src/main/scala/kafka/server/DelayedProduce.scala 05078b24ef28f2f4e099afa943e43f1d00359fda
> core/src/main/scala/kafka/server/KafkaApis.scala d63bc18d795a6f0e6994538ca55a9a46f7fb8ffd
> core/src/main/scala/kafka/server/OffsetManager.scala 5cca85cf727975f6d3acb2223fd186753ad761dc
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala b31b432a226ba79546dd22ef1d2acbb439c2e9a3
> core/src/main/scala/kafka/server/ReplicaManager.scala 59c9bc3ac3a8afc07a6f8c88c5871304db588d17
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 5717165f2344823fabe8f7cfafae4bb8af2d949a
> core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Jun Rao <ju...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review94167
-----------------------------------------------------------
Thanks for the patch. A few comments below.
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java (line 77)
<https://reviews.apache.org/r/33378/#comment148702>
With this, we are bounding the constructor to the v1 protocol. This is fine for FetchResponse for now since the broker uses the scala objects for dealing with the send file api. However, we should add a comment to indicate this is the constructor for v1 protocol.
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java (line 55)
<https://reviews.apache.org/r/33378/#comment148701>
We actually will need different constructors for different versions. We want to reuse those request/response objects on the broker side. So, the broker will need to construct different version of the response based on the version of the request. You can take a look at OffsetCommitRequest as an example.
core/src/main/scala/kafka/api/FetchResponse.scala (lines 193 - 195)
<https://reviews.apache.org/r/33378/#comment148704>
No need to wrap single line statement with {}.
core/src/main/scala/kafka/api/ProducerResponse.scala (line 40)
<https://reviews.apache.org/r/33378/#comment148709>
Perhaps it's worth adding a commment that readFrom() only reads v1 format.
core/src/main/scala/kafka/api/ProducerResponse.scala (lines 49 - 50)
<https://reviews.apache.org/r/33378/#comment148707>
Coding style: no space before :.
Also, could requestVersion be just version?
core/src/main/scala/kafka/api/ProducerResponse.scala (lines 95 - 97)
<https://reviews.apache.org/r/33378/#comment148706>
Coding style: no need to wrap single line statement with {}.
- Jun Rao
On July 13, 2015, 8:36 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated July 13, 2015, 8:36 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel's comments
>
> For now the patch will publish a zero delay and return a response
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 56281ee15cc33dfc96ff64d5b1e596047c7132a4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 07e65d4a54ba4eef5b787eba3e71cbe9f6a920bd
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 8686d83aa52e435c6adafbe9ff4bd1602281072a
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java fabeae3083a8ea55cdacbb9568f3847ccd85bab4
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 37ec0b79beafcf5735c386b066eb319fb697eff5
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java 419541011d652becf0cda7a5e62ce813cddb1732
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java e3cc1967e407b64cc734548c19e30de700b64ba8
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala c16f7edd322709060e54c77eb505c44cbd77a4ec
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala 83fc47417dd7fe4edf030217fa7fd69d99b170b0
> core/src/main/scala/kafka/server/DelayedFetch.scala de6cf5bdaa0e70394162febc63b50b55ca0a92db
> core/src/main/scala/kafka/server/DelayedProduce.scala 05078b24ef28f2f4e099afa943e43f1d00359fda
> core/src/main/scala/kafka/server/KafkaApis.scala d63bc18d795a6f0e6994538ca55a9a46f7fb8ffd
> core/src/main/scala/kafka/server/OffsetManager.scala 5cca85cf727975f6d3acb2223fd186753ad761dc
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala b31b432a226ba79546dd22ef1d2acbb439c2e9a3
> core/src/main/scala/kafka/server/ReplicaManager.scala 59c9bc3ac3a8afc07a6f8c88c5871304db588d17
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 5717165f2344823fabe8f7cfafae4bb8af2d949a
> core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review94876
-----------------------------------------------------------
Jun/Joel - Thanks for the comments. I'd like to address these all at once after KAFKA-2084 is committed because I will need to rebase after that.
- Aditya Auradkar
On July 13, 2015, 8:36 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated July 13, 2015, 8:36 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel's comments
>
> For now the patch will publish a zero delay and return a response
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 56281ee15cc33dfc96ff64d5b1e596047c7132a4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 07e65d4a54ba4eef5b787eba3e71cbe9f6a920bd
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 8686d83aa52e435c6adafbe9ff4bd1602281072a
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java fabeae3083a8ea55cdacbb9568f3847ccd85bab4
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 37ec0b79beafcf5735c386b066eb319fb697eff5
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java 419541011d652becf0cda7a5e62ce813cddb1732
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java e3cc1967e407b64cc734548c19e30de700b64ba8
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala c16f7edd322709060e54c77eb505c44cbd77a4ec
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala 83fc47417dd7fe4edf030217fa7fd69d99b170b0
> core/src/main/scala/kafka/server/DelayedFetch.scala de6cf5bdaa0e70394162febc63b50b55ca0a92db
> core/src/main/scala/kafka/server/DelayedProduce.scala 05078b24ef28f2f4e099afa943e43f1d00359fda
> core/src/main/scala/kafka/server/KafkaApis.scala d63bc18d795a6f0e6994538ca55a9a46f7fb8ffd
> core/src/main/scala/kafka/server/OffsetManager.scala 5cca85cf727975f6d3acb2223fd186753ad761dc
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala b31b432a226ba79546dd22ef1d2acbb439c2e9a3
> core/src/main/scala/kafka/server/ReplicaManager.scala 59c9bc3ac3a8afc07a6f8c88c5871304db588d17
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 5717165f2344823fabe8f7cfafae4bb8af2d949a
> core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review94822
-----------------------------------------------------------
core/src/main/scala/kafka/server/ReplicaManager.scala (line 312)
<https://reviews.apache.org/r/33378/#comment149463>
Good point. I think we should commit 2084 before this patch.
While rebasing, I can refactor this patch to pass the throttleTime to the callback that sends the response. i.e.
quotaManagers.recordAndMaybeThrottle (clientId, value, callback) {
// add to delay queue
// pass in the computed throttle time to the callback.
}
If I do that, I dont need to pass the throttleTime to the responseCallback in ReplicaManager
- Aditya Auradkar
On July 13, 2015, 8:36 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated July 13, 2015, 8:36 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel's comments
>
> For now the patch will publish a zero delay and return a response
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 56281ee15cc33dfc96ff64d5b1e596047c7132a4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 07e65d4a54ba4eef5b787eba3e71cbe9f6a920bd
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 8686d83aa52e435c6adafbe9ff4bd1602281072a
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java fabeae3083a8ea55cdacbb9568f3847ccd85bab4
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 37ec0b79beafcf5735c386b066eb319fb697eff5
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java 419541011d652becf0cda7a5e62ce813cddb1732
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java e3cc1967e407b64cc734548c19e30de700b64ba8
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala c16f7edd322709060e54c77eb505c44cbd77a4ec
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala 83fc47417dd7fe4edf030217fa7fd69d99b170b0
> core/src/main/scala/kafka/server/DelayedFetch.scala de6cf5bdaa0e70394162febc63b50b55ca0a92db
> core/src/main/scala/kafka/server/DelayedProduce.scala 05078b24ef28f2f4e099afa943e43f1d00359fda
> core/src/main/scala/kafka/server/KafkaApis.scala d63bc18d795a6f0e6994538ca55a9a46f7fb8ffd
> core/src/main/scala/kafka/server/OffsetManager.scala 5cca85cf727975f6d3acb2223fd186753ad761dc
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala b31b432a226ba79546dd22ef1d2acbb439c2e9a3
> core/src/main/scala/kafka/server/ReplicaManager.scala 59c9bc3ac3a8afc07a6f8c88c5871304db588d17
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 5717165f2344823fabe8f7cfafae4bb8af2d949a
> core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Joel Koshy <jj...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review96186
-----------------------------------------------------------
core/src/main/scala/kafka/api/FetchResponse.scala (line 183)
<https://reviews.apache.org/r/33378/#comment151509>
Pre-existing, but if you don't mind, see if you can apply that pattern here as well. We generally like to avoid the `_*` notation for clarity. (Never mind if it becomes too unweildy.)
core/src/main/scala/kafka/server/KafkaApis.scala (line 356)
<https://reviews.apache.org/r/33378/#comment151508>
Can you use this pattern:
`groupBy { case (topicAndPartition, _) => topicAndPartition.topic },...`
- Joel Koshy
On Aug. 24, 2015, 5:33 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated Aug. 24, 2015, 5:33 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel and Juns comments
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala b9efec2efbd3ea0ee12b911f453c47e66ad34894
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
> core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
> core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
> core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
> core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
> core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala caf98e8f2e09d39ab8234b9f4b9478686865e908
> core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Joel Koshy <jj...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review96470
-----------------------------------------------------------
Ship it!
Ship It!
- Joel Koshy
On Aug. 25, 2015, 6:30 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated Aug. 25, 2015, 6:30 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel and Juns comments
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala b9efec2efbd3ea0ee12b911f453c47e66ad34894
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
> core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
> core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
> core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
> core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
> core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala caf98e8f2e09d39ab8234b9f4b9478686865e908
> core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/
-----------------------------------------------------------
(Updated Aug. 25, 2015, 6:30 p.m.)
Review request for kafka, Joel Koshy and Jun Rao.
Bugs: KAFKA-2136
https://issues.apache.org/jira/browse/KAFKA-2136
Repository: kafka
Description (updated)
-------
Changes are
- Addressing Joel's comments
- protocol changes to the fetch request and response to return the throttle_time_ms to clients
- New producer/consumer metrics to expose the avg and max delay time for a client
- Test cases.
- Addressed Joel and Juns comments
Diffs
-----
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
core/src/main/scala/kafka/api/FetchResponse.scala b9efec2efbd3ea0ee12b911f453c47e66ad34894
core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala caf98e8f2e09d39ab8234b9f4b9478686865e908
core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52
Diff: https://reviews.apache.org/r/33378/diff/
Testing
-------
New tests added
Thanks,
Aditya Auradkar
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/
-----------------------------------------------------------
(Updated Aug. 25, 2015, 6:30 p.m.)
Review request for kafka, Joel Koshy and Jun Rao.
Bugs: KAFKA-2136
https://issues.apache.org/jira/browse/KAFKA-2136
Repository: kafka
Description (updated)
-------
Addressing Joel's comments
Merging
Chaning variable name
Addressing Joel's comments
Addressing Joel's comments
Addressing comments
Addressing joels comments
Addressing joels comments
Addressed Joels comments
Diffs (updated)
-----
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
core/src/main/scala/kafka/api/FetchResponse.scala b9efec2efbd3ea0ee12b911f453c47e66ad34894
core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala caf98e8f2e09d39ab8234b9f4b9478686865e908
core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52
Diff: https://reviews.apache.org/r/33378/diff/
Testing
-------
New tests added
Thanks,
Aditya Auradkar
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/
-----------------------------------------------------------
(Updated Aug. 24, 2015, 5:33 p.m.)
Review request for kafka, Joel Koshy and Jun Rao.
Bugs: KAFKA-2136
https://issues.apache.org/jira/browse/KAFKA-2136
Repository: kafka
Description (updated)
-------
Changes are
- Addressing Joel's comments
- protocol changes to the fetch request and response to return the throttle_time_ms to clients
- New producer/consumer metrics to expose the avg and max delay time for a client
- Test cases.
- Addressed Joel and Juns comments
Diffs
-----
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
core/src/main/scala/kafka/api/FetchResponse.scala b9efec2efbd3ea0ee12b911f453c47e66ad34894
core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala caf98e8f2e09d39ab8234b9f4b9478686865e908
core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52
Diff: https://reviews.apache.org/r/33378/diff/
Testing
-------
New tests added
Thanks,
Aditya Auradkar
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/
-----------------------------------------------------------
(Updated Aug. 24, 2015, 5:33 p.m.)
Review request for kafka, Joel Koshy and Jun Rao.
Bugs: KAFKA-2136
https://issues.apache.org/jira/browse/KAFKA-2136
Repository: kafka
Description (updated)
-------
Addressing Joel's comments
Merging
Chaning variable name
Addressing Joel's comments
Addressing Joel's comments
Addressing comments
Addressing joels comments
Addressing joels comments
Diffs (updated)
-----
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
core/src/main/scala/kafka/api/FetchResponse.scala b9efec2efbd3ea0ee12b911f453c47e66ad34894
core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala caf98e8f2e09d39ab8234b9f4b9478686865e908
core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52
Diff: https://reviews.apache.org/r/33378/diff/
Testing
-------
New tests added
Thanks,
Aditya Auradkar
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
> On Aug. 22, 2015, 12:45 a.m., Joel Koshy wrote:
> > clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java, line 107
> > <https://reviews.apache.org/r/33378/diff/12-13/?file=1043780#file1043780line107>
> >
> > This will probably need a versionId as well (as is done in the Scala response) - i.e., when we move the broker over to use these protocol schemas.
Makes sense. Do you want me to tackle this in this patch or should it be tackled in the patch that migrates the broker to use these schemas?
- Aditya
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review96100
-----------------------------------------------------------
On Aug. 24, 2015, 5:33 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated Aug. 24, 2015, 5:33 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel and Juns comments
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala b9efec2efbd3ea0ee12b911f453c47e66ad34894
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
> core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
> core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
> core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
> core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
> core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala caf98e8f2e09d39ab8234b9f4b9478686865e908
> core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
> On Aug. 22, 2015, 12:45 a.m., Joel Koshy wrote:
> > clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java, line 107
> > <https://reviews.apache.org/r/33378/diff/12-13/?file=1043780#file1043780line107>
> >
> > This will probably need a versionId as well (as is done in the Scala response) - i.e., when we move the broker over to use these protocol schemas.
>
> Aditya Auradkar wrote:
> Makes sense. Do you want me to tackle this in this patch or should it be tackled in the patch that migrates the broker to use these schemas?
>
> Joel Koshy wrote:
> I think it would be safer to do it in this patch itself.
Do you think we actually need to check the version id? The Struct contains the schema which should be sufficient to understand the version number right?
- Aditya
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review96100
-----------------------------------------------------------
On Aug. 24, 2015, 5:33 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated Aug. 24, 2015, 5:33 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel and Juns comments
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala b9efec2efbd3ea0ee12b911f453c47e66ad34894
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
> core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
> core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
> core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
> core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
> core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala caf98e8f2e09d39ab8234b9f4b9478686865e908
> core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Joel Koshy <jj...@gmail.com>.
> On Aug. 22, 2015, 12:45 a.m., Joel Koshy wrote:
> > clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java, line 107
> > <https://reviews.apache.org/r/33378/diff/12-13/?file=1043780#file1043780line107>
> >
> > This will probably need a versionId as well (as is done in the Scala response) - i.e., when we move the broker over to use these protocol schemas.
>
> Aditya Auradkar wrote:
> Makes sense. Do you want me to tackle this in this patch or should it be tackled in the patch that migrates the broker to use these schemas?
I think it would be safer to do it in this patch itself.
- Joel
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review96100
-----------------------------------------------------------
On Aug. 24, 2015, 5:33 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated Aug. 24, 2015, 5:33 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel and Juns comments
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala b9efec2efbd3ea0ee12b911f453c47e66ad34894
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
> core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
> core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
> core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
> core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
> core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala caf98e8f2e09d39ab8234b9f4b9478686865e908
> core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Joel Koshy <jj...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review96100
-----------------------------------------------------------
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java (line 33)
<https://reviews.apache.org/r/33378/#comment151319>
of FetchResponse
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java (line 107)
<https://reviews.apache.org/r/33378/#comment151321>
This will probably need a versionId as well (as is done in the Scala response) - i.e., when we move the broker over to use these protocol schemas.
- Joel Koshy
On Aug. 21, 2015, 11:30 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated Aug. 21, 2015, 11:30 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel and Juns comments
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala b9efec2efbd3ea0ee12b911f453c47e66ad34894
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
> core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
> core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
> core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
> core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
> core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala caf98e8f2e09d39ab8234b9f4b9478686865e908
> core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/
-----------------------------------------------------------
(Updated Aug. 21, 2015, 11:30 p.m.)
Review request for kafka, Joel Koshy and Jun Rao.
Bugs: KAFKA-2136
https://issues.apache.org/jira/browse/KAFKA-2136
Repository: kafka
Description (updated)
-------
Changes are
- Addressing Joel's comments
- protocol changes to the fetch request and response to return the throttle_time_ms to clients
- New producer/consumer metrics to expose the avg and max delay time for a client
- Test cases.
- Addressed Joel and Juns comments
Diffs
-----
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
core/src/main/scala/kafka/api/FetchResponse.scala b9efec2efbd3ea0ee12b911f453c47e66ad34894
core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala caf98e8f2e09d39ab8234b9f4b9478686865e908
core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52
Diff: https://reviews.apache.org/r/33378/diff/
Testing
-------
New tests added
Thanks,
Aditya Auradkar
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/
-----------------------------------------------------------
(Updated Aug. 21, 2015, 11:29 p.m.)
Review request for kafka, Joel Koshy and Jun Rao.
Bugs: KAFKA-2136
https://issues.apache.org/jira/browse/KAFKA-2136
Repository: kafka
Description (updated)
-------
Addressing Joel's comments
Merging
Chaning variable name
Addressing Joel's comments
Addressing Joel's comments
Addressing comments
Addressing joels comments
Diffs (updated)
-----
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
core/src/main/scala/kafka/api/FetchResponse.scala b9efec2efbd3ea0ee12b911f453c47e66ad34894
core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala caf98e8f2e09d39ab8234b9f4b9478686865e908
core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52
Diff: https://reviews.apache.org/r/33378/diff/
Testing
-------
New tests added
Thanks,
Aditya Auradkar
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Joel Koshy <jj...@gmail.com>.
> On Aug. 21, 2015, 12:13 a.m., Joel Koshy wrote:
> > core/src/main/scala/kafka/api/FetchResponse.scala, line 175
> > <https://reviews.apache.org/r/33378/diff/12/?file=1043787#file1043787line175>
> >
> > Since (in the event of multiple calls) this grouping would be repeated, should we just have `responseSize` take the `FetchResponse` object and have that look up the `lazy val dataGroupedByTopic`? That said, I think the original version should have had `sizeInBytes` as a `lazy val` as well.
>
> Aditya Auradkar wrote:
> FetchResponse.responseSize is called from KafkaApis in order to figure out what value to record. We cannot pass in a FetchResponse object because the object doesn't exist yet because the throttle time is not available.
>
> Should I change the signature to accept a dataGroupedByTopic instead of a TopicPartition -> FetchResponsePartitionData map.
Got it - yes we could do that. There is also the pre-existing issue of `sizeInBytes` breaking the laziness of `dataGroupedByTopic` which we can address.
- Joel
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review96009
-----------------------------------------------------------
On Aug. 21, 2015, 11:30 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated Aug. 21, 2015, 11:30 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel and Juns comments
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala b9efec2efbd3ea0ee12b911f453c47e66ad34894
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
> core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
> core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
> core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
> core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
> core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala caf98e8f2e09d39ab8234b9f4b9478686865e908
> core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Joel Koshy <jj...@gmail.com>.
> On Aug. 21, 2015, 12:13 a.m., Joel Koshy wrote:
> > core/src/main/scala/kafka/api/FetchResponse.scala, line 175
> > <https://reviews.apache.org/r/33378/diff/12/?file=1043787#file1043787line175>
> >
> > Since (in the event of multiple calls) this grouping would be repeated, should we just have `responseSize` take the `FetchResponse` object and have that look up the `lazy val dataGroupedByTopic`? That said, I think the original version should have had `sizeInBytes` as a `lazy val` as well.
>
> Aditya Auradkar wrote:
> FetchResponse.responseSize is called from KafkaApis in order to figure out what value to record. We cannot pass in a FetchResponse object because the object doesn't exist yet because the throttle time is not available.
>
> Should I change the signature to accept a dataGroupedByTopic instead of a TopicPartition -> FetchResponsePartitionData map.
>
> Joel Koshy wrote:
> Got it - yes we could do that. There is also the pre-existing issue of `sizeInBytes` breaking the laziness of `dataGroupedByTopic` which we can address.
I realized later that this now causes two explicit group-by's on the server-side and one on the consumer-side. So the non-lazy val may have worked better in practice.
- Joel
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review96009
-----------------------------------------------------------
On Aug. 25, 2015, 6:30 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated Aug. 25, 2015, 6:30 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel and Juns comments
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala b9efec2efbd3ea0ee12b911f453c47e66ad34894
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
> core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
> core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
> core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
> core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
> core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala caf98e8f2e09d39ab8234b9f4b9478686865e908
> core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala c4b5803917e700965677d53624f1460c1a52bf52
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
> On Aug. 21, 2015, 12:13 a.m., Joel Koshy wrote:
> > core/src/main/scala/kafka/server/ClientQuotaManager.scala, line 142
> > <https://reviews.apache.org/r/33378/diff/12/?file=1043792#file1043792line142>
> >
> > any specific reason for this change?
recordAndMaybeThrottle returns an int value as delay. Nicer to have delayTime return an int as well
> On Aug. 21, 2015, 12:13 a.m., Joel Koshy wrote:
> > core/src/main/scala/kafka/api/FetchResponse.scala, line 175
> > <https://reviews.apache.org/r/33378/diff/12/?file=1043787#file1043787line175>
> >
> > Since (in the event of multiple calls) this grouping would be repeated, should we just have `responseSize` take the `FetchResponse` object and have that look up the `lazy val dataGroupedByTopic`? That said, I think the original version should have had `sizeInBytes` as a `lazy val` as well.
FetchResponse.responseSize is called from KafkaApis in order to figure out what value to record. We cannot pass in a FetchResponse object because the object doesn't exist yet because the throttle time is not available.
Should I change the signature to accept a dataGroupedByTopic instead of a TopicPartition -> FetchResponsePartitionData map.
- Aditya
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review96009
-----------------------------------------------------------
On Aug. 18, 2015, 8:24 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated Aug. 18, 2015, 8:24 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel and Juns comments
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
> core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
> core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
> core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
> core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
> core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala 97dcca8c96f955acb3d92b29d7faa1e031ba71d4
> core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala 14a7f4538041d557c190127e3d5f85edf2a0e7c1
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Joel Koshy <jj...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review96009
-----------------------------------------------------------
core/src/main/scala/kafka/api/FetchResponse.scala (line 172)
<https://reviews.apache.org/r/33378/#comment151206>
Since (in the event of multiple calls) this grouping would be repeated, should we just have `responseSize` take the `FetchResponse` object and have that look up the `lazy val dataGroupedByTopic`? That said, I think the original version should have had `sizeInBytes` as a `lazy val` as well.
core/src/main/scala/kafka/server/ClientQuotaManager.scala (line 115)
<https://reviews.apache.org/r/33378/#comment151207>
`throttleTimeMs` for consistency
core/src/main/scala/kafka/server/ClientQuotaManager.scala (line 119)
<https://reviews.apache.org/r/33378/#comment151209>
Maybe make this explicitly zero, and `delayTime` can move below as a `val`
core/src/main/scala/kafka/server/ClientQuotaManager.scala (line 142)
<https://reviews.apache.org/r/33378/#comment151208>
any specific reason for this change?
- Joel Koshy
On Aug. 18, 2015, 8:24 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated Aug. 18, 2015, 8:24 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel and Juns comments
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
> core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
> core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
> core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
> core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
> core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala 97dcca8c96f955acb3d92b29d7faa1e031ba71d4
> core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala 14a7f4538041d557c190127e3d5f85edf2a0e7c1
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/
-----------------------------------------------------------
(Updated Aug. 18, 2015, 8:24 p.m.)
Review request for kafka, Joel Koshy and Jun Rao.
Bugs: KAFKA-2136
https://issues.apache.org/jira/browse/KAFKA-2136
Repository: kafka
Description (updated)
-------
Changes are
- Addressing Joel's comments
- protocol changes to the fetch request and response to return the throttle_time_ms to clients
- New producer/consumer metrics to expose the avg and max delay time for a client
- Test cases.
- Addressed Joel and Juns comments
Diffs
-----
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala 97dcca8c96f955acb3d92b29d7faa1e031ba71d4
core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala 14a7f4538041d557c190127e3d5f85edf2a0e7c1
Diff: https://reviews.apache.org/r/33378/diff/
Testing
-------
New tests added
Thanks,
Aditya Auradkar
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/
-----------------------------------------------------------
(Updated Aug. 18, 2015, 8:24 p.m.)
Review request for kafka, Joel Koshy and Jun Rao.
Bugs: KAFKA-2136
https://issues.apache.org/jira/browse/KAFKA-2136
Repository: kafka
Description (updated)
-------
Addressing Joel's comments
Merging
Chaning variable name
Addressing Joel's comments
Addressing Joel's comments
Addressing comments
Diffs (updated)
-----
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 9dc669728e6b052f5c6686fcf1b5696a50538ab4
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 0baf16e55046a2f49f6431e01d52c323c95eddf0
clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java df073a0e76cc5cc731861b9604d0e19a928970e0
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java 715504b32950666e9aa5a260fa99d5f897b2007a
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java febfc70dabc23671fd8a85cf5c5b274dff1e10fb
clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java a7c83cac47d41138d47d7590a3787432d675c1b0
clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java 8b2aca85fa738180e5420985fddc39a4bf9681ea
core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
core/src/main/scala/kafka/consumer/SimpleConsumer.scala 7ebc0405d1f309bed9943e7116051d1d8276f200
core/src/main/scala/kafka/server/AbstractFetcherThread.scala f84306143c43049e3aa44e42beaffe7eb2783163
core/src/main/scala/kafka/server/ClientQuotaManager.scala 9f8473f1c64d10c04cf8cc91967688e29e54ae2e
core/src/main/scala/kafka/server/KafkaApis.scala 67f0cad802f901f255825aa2158545d7f5e7cc3d
core/src/main/scala/kafka/server/ReplicaFetcherThread.scala fae22d2af8daccd528ac24614290f46ae8f6c797
core/src/main/scala/kafka/server/ReplicaManager.scala d829e180c3943a90861a12ec184f9b4e4bbafe7d
core/src/main/scala/kafka/server/ThrottledResponse.scala 1f80d5480ccf7c411a02dd90296a7046ede0fae2
core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala b4c2a228c3c9872e5817ac58d3022e4903e317b7
core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala 97dcca8c96f955acb3d92b29d7faa1e031ba71d4
core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala 14a7f4538041d557c190127e3d5f85edf2a0e7c1
Diff: https://reviews.apache.org/r/33378/diff/
Testing
-------
New tests added
Thanks,
Aditya Auradkar
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/
-----------------------------------------------------------
(Updated Aug. 18, 2015, 8:20 p.m.)
Review request for kafka, Joel Koshy and Jun Rao.
Bugs: KAFKA-2136
https://issues.apache.org/jira/browse/KAFKA-2136
Repository: kafka
Description (updated)
-------
kafka-2005; Generate html report for system tests; patched by Ashish Singh; reviewed by Jun Rao
kafka-2266; Client Selector can drop idle connections without notifying NetworkClient; patched by Jason Gustafson; reviewed by Jun Rao
kafka-2232; make MockProducer generic; patched by Alexander Pakulov; reviewed by Jun Rao
kafka-2164; ReplicaFetcherThread: suspicious log message on reset offset; patched by Alexey Ozeritski; reviewed by Jun Rao
kafka-2101; Metric metadata-age is reset on a failed update; patched by Tim Brooks; reviewed by Jun Rao
kafka-2195; Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest; patched by Andrii Biletskyi; reviewed by Jun Rao
kafka-2270; incorrect package name in unit tests; patched by Proneet Verma; reviewed by Jun Rao
kafka-2272; listeners endpoint parsing fails if the hostname has capital letter; patched by Sriharsha Chintalapani; reviewed by Jun Rao
kafka-2264; SESSION_TIMEOUT_MS_CONFIG in ConsumerConfig should be int; patched by Manikumar Reddy; reviewed by Jun Rao
kafka-2252; Socket connection closing is logged, but not corresponding opening of socket; patched by Gwen Shapira; reviewed by Jun Rao
kafka-2262; LogSegmentSize validation should be consistent; patched by Manikumar Reddy; reviewed by Jun Rao
trivial fix for stylecheck error on Jenkins
kafka-2249; KafkaConfig does not preserve original Properties; patched by Gwen Shapira; reviewed by Jun Rao
kafka-2265; creating a topic with large number of partitions takes a long time; patched by Manikumar Reddy; reviewed by Jun Rao
kafka-2234; Partition reassignment of a nonexistent topic prevents future reassignments; patched by Manikumar Reddy; reviewed by Jun Rao
trivial change to fix unit test failure introduced in kafka-2234
kafka-1758; corrupt recovery file prevents startup; patched by Manikumar Reddy; reviewed by Neha Narkhede and Jun Rao
kafka-1646; Improve consumer read performance for Windows; patched by Honghai Chen; reviewed by Jay Kreps and Jun Rao
kafka-2012; Broker should automatically handle corrupt index files; patched by Manikumar Reddy; reviewed by Jun Rao
kafka-2290; OffsetIndex should open RandomAccessFile consistently; patched by Chris Black; reviewed by Jun Rao
kafka-2235; LogCleaner offset map overflow; patched by Ivan Simoneko; reviewed by Jun Rao
KAFKA-2245; Add response tests for consumer coordinator; reviewed by Joel Koshy
KAFKA-2293; Fix incorrect format specification in Partition.scala; reviewed by Joel Koshy
kafka-2168; New consumer poll() can block other calls like position(), commit(), and close() indefinitely; patched by Jason Gustafson; reviewed by Jay Kreps, Ewen Cheslack-Postava, Guozhang Wang and Jun Rao
KAFKA-2294; javadoc compile error due to illegal <p/> , build failing (jdk 8); patched by Jeff Maxwell; reviewed by Jakob Homan
KAFKA-2281: avoid unnecessary value copying if logAsString is false; reviewed by Guozhang Wang
KAFKA-2168: minor follow-up patch; reviewed by Guozhang Wang
KAFKA-1740: merge offset manager into consumer coordinator; reviewed by Onur Karaman and Jason Gustafson
kafka-2248; Use Apache Rat to enforce copyright headers; patched by Ewen Cheslack-Postava; reviewed by Gwen Shapira, Joel Joshy and Jun Rao
kafka-2132; Move Log4J appender to a separate module; patched by Ashish Singh; reviewed by Gwen Shapira, Aditya Auradkar and Jun Rao
KAFKA-2314: proper MirrorMaker's message handler help message; reviewed by Guozhang Wang
kafka-1367; Broker topic metadata not kept in sync with ZooKeeper; patched by Ashish Singh; reviewed by Jun Rao
KAFKA-2304 Supported enabling JMX in Kafka Vagrantfile patch by Stevo Slavic reviewed by Ewen Cheslack-Postava
KAFKA-2306: add another metric for buffer exhausted; reviewed by Guozhang Wang
KAFKA-2317: follow-up of KAFKA1367; reviewed by Guozhang Wang
KAFKA-2313: javadoc fix for KafkaConsumer deserialization; reviewed by Guozhang Wang
KAFKA-2298; Client Selector can drop connections on InvalidReceiveException without notifying NetworkClient; reviewed by Jason Gustafson and Joel Koshy
Trivial commit - explicitly exclude build/rat-report.xml from rat check
KAFKA-2308: make MemoryRecords idempotent; reviewed by Guozhang Wang
KAFKA-2316: Drop java 1.6 support; patched by Sriharsha Chintalapani reviewed by Ismael Juma and Gwen Shapira
KAFKA-2327; broker doesn't start if config defines advertised.host but not advertised.port
Added unit tests as well. These fail without the fix, but pass with the fix.
Author: Geoff Anderson <ge...@confluent.io>
Closes #73 from granders/KAFKA-2327 and squashes the following commits:
52a2085 [Geoff Anderson] Cleaned up unecessary toString calls
23b3340 [Geoff Anderson] Fixes KAFKA-2327
KAFKA-1740 follow-up: add state checking in handling heartbeat request; reviewed by Onur Karaman, Ewen Cheslack-Postavam and Guozhang Wang
Trivial commit - explicitly exclude build/** from rat check
KAFKA-2312: use atomic long for thread id reference; reviewed by Ewen Cheslack-Postava, Jason Gustafson, Ismael Juma and Guozhang Wang
kafka-972; MetadataRequest returns stale list of brokers; patched by Ashish Singh; reviewed by Jun Rao
KAFKA-2198: kafka-topics.sh exits with 0 status on failures; patched by Manikumar Reddy reviewed by Gwen Shapira
KAFKA-2123: add callback in commit api and use a delayed queue for async requests; reviewed by Ewen Cheslack-Postava and Guozhang Wang
KAFKA-2335; fix comment about thread safety
Author: Jason Gustafson <ja...@confluent.io>
Closes #78 from hachikuji/KAFKA-2335 and squashes the following commits:
c697998 [Jason Gustafson] KAFKA-2335; fix comment about consumer thread safety
[MINOR] fix new consumer heartbeat reschedule bug
This commit fixes a minor issue introduced in the patch for KAFKA-2123. The schedule method requires the time the task should be executed, not a delay.
Author: Jason Gustafson <ja...@confluent.io>
Closes #79 from hachikuji/KAFKA-2123-fix and squashes the following commits:
6eb7ec6 [Jason Gustafson] [Minor] fix new consumer heartbeat reschedule bug
KAFKA-2032: validate consumer's partition-assignment config; reviewed by Jason Rosenberg, Sriharsha Chintalapani and Guozhang Wang
kafka-2323; Simplify ScalaTest dependency versions; patched by Ismael Juma; reviewed by Jun Rao
kafka-2241; AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer); patched by Dong Lin; reviewed by Jun Rao
kafka-2174; Wrong TopicMetadata deserialization; patched by Alexey Ozeritskiy; reviewed by Jun Rao
kafka-2271; transient unit test failure in KafkaConfigConfigDefTest.testFromPropsToProps; patched by Jason Gustafson; reviewed by Jun Rao
kafka-2103; kafka.producer.AsyncProducerTest failure; patched by Ewen Cheslack-Postava; reviewed by Jun Rao
KAFKA-2324; Update to Scala 2.11.7
Author: Ismael Juma <is...@juma.me.uk>
Closes #82 from ijuma/kafka-2324 and squashes the following commits:
d71bf5c [Ismael Juma] KAFKA-2324; Update to Scala 2.11.7
KAFKA-2345; Attempt to delete a topic already marked for deletion throws ZkNodeExistsException; patched by Ashish Singh; reviewed by Sriharsha Chintalapani and Ismael Juma
Adding a file missed while committing KAFKA-2345
KAFKA-2337; Verify that metric names will not collide when creating new topics; patched by Grant Henke; reviewed by Edward Ribeiro, Ashish Singh and Gwen Shapira
KAFKA-2328; merge-kafka-pr.py script should not leave user in a detached branch
The right command to get the branch name is `git rev-parse --abbrev-ref HEAD` instead of `git rev-parse HEAD`. The latter gives the commit hash causing a detached branch when we checkout to it. Seems like a bug we inherited from the Spark script.
Author: Ismael Juma <is...@juma.me.uk>
Closes #84 from ijuma/kafka-2328-merge-script-no-detached-branch and squashes the following commits:
ae201dd [Ismael Juma] KAFKA-2328; merge-kafka-pr.py script should not leave user in a detached branch
KAFKA-2342; KafkaConsumer rebalance with in-flight fetch can cause invalid position
Author: Jason Gustafson <ja...@confluent.io>
Closes #88 from hachikuji/KAFKA-2342 and squashes the following commits:
cabb017 [Jason Gustafson] KAFKA-2342; KafkaConsumer rebalance with in-flight fetch can cause invalid position
KAFKA-2344; kafka-merge-pr improvements
The first 4 commits are adapted from changes that have been done to the Spark version and the last one is the feature that gwenshap asked for.
Author: Ismael Juma <is...@juma.me.uk>
Reviewers: Gwen Shapira <cs...@gmail.com>
Closes #90 from ijuma/kafka-2344-merge-pr-improvements and squashes the following commits:
900c371 [Ismael Juma] Allow reviewers to be entered during merge
ac06347 [Ismael Juma] Allow primary author to be overridden during merge
b309829 [Ismael Juma] Set JIRA resolution to "Fixed" instead of relying on default transition
0c69a64 [Ismael Juma] Check return value of doctest.testmod()
061cdce [Ismael Juma] Fix instructions on how to install the `jira-python` library
KAFKA-2355; Add an unit test to validate the deletion of a partition marked as deleted; patched by Edward Ribeiro, reviewed by Ashish Singh, Ismael Juma and Grant Henke
KAFKA-2353; SocketServer catch exception and close connection properly; reviewed by Gwen Shapira, Ismael Juma and Guozhang Wang
KAFKA-2348; Drop support for Scala 2.9
`testAll` passed locally.
Author: Ismael Juma <is...@juma.me.uk>
Reviewers: Harsha, Ewen, Guozhang
Closes #87 from ijuma/kafka-2348-drop-support-for-scala-2.9 and squashes the following commits:
cf9796a [Ismael Juma] KAFKA-2348; Drop support for Scala 2.9
KAFKA-2321; Introduce CONTRIBUTING.md
Author: Ismael Juma <is...@juma.me.uk>
Reviewers: Guozhang
Closes #97 from ijuma/kafka-2321 and squashes the following commits:
4834464 [Ismael Juma] KAFKA-2321; Introduce CONTRIBUTING.md
KAFKA-2301: Warn ConsumerOffsetChecker as deprecated; reviewed by Ewen Cheslack-Postava and Guozhang Wang
KAFKA-2381: Fix concurrent modification on assigned partition while looping over it; reviewed by Jason Gustafson, Aditya Auradkar, Ewen Cheslack-Postava, Ismael Juma and Guozhang Wang
KAFKA-2347: Add setConsumerRebalanceListener method to ZookeeperConsumerConnector java api; reviewed by Jiangjie Qin, Ismael Juma, Grant Henke and Guozhang Wang
KAFKA-2275: Add ListTopics() API to the Java consumer; reviewed by Jason Gustafson, Edward Ribeiro and Guozhang Wang
KAFKA-2089: Fix transient MetadataTest failure; reviewed by Jiangjie Qin and Guozhang Wang
KAFKA-2276; KIP-25 initial patch
Initial patch for KIP-25
Note that to install ducktape, do *not* use pip to install ducktape. Instead:
```
$ git clone gitgithub.com:confluentinc/ducktape.git
$ cd ducktape
$ python setup.py install
```
Author: Geoff Anderson <ge...@confluent.io>
Author: Geoff <gr...@gmail.com>
Author: Liquan Pei <li...@gmail.com>
Reviewers: Ewen, Gwen, Jun, Guozhang
Closes #70 from granders/KAFKA-2276 and squashes the following commits:
a62fb6c [Geoff Anderson] fixed checkstyle errors
a70f0f8 [Geoff Anderson] Merged in upstream trunk.
8b62019 [Geoff Anderson] Merged in upstream trunk.
47b7b64 [Geoff Anderson] Created separate tools jar so that the clients package does not pull in dependencies on the Jackson JSON tools or argparse4j.
a9e6a14 [Geoff Anderson] Merged in upstream changes
d18db7b [Geoff Anderson] fixed :rat errors (needed to add licenses)
321fdf8 [Geoff Anderson] Ignore tests/ and vagrant/ directories when running rat build task
795fc75 [Geoff Anderson] Merged in changes from upstream trunk.
1d93f06 [Geoff Anderson] Updated provisioning to use java 7 in light of KAFKA-2316
2ea4e29 [Geoff Anderson] Tweaked README, changed default log collection behavior on VerifiableProducer
0eb6fdc [Geoff Anderson] Merged in system-tests
69dd7be [Geoff Anderson] Merged in trunk
4034dd6 [Geoff Anderson] Merged in upstream trunk
ede6450 [Geoff] Merge pull request #4 from confluentinc/move_muckrake
7751545 [Geoff Anderson] Corrected license headers
e6d532f [Geoff Anderson] java 7 -> java 6
8c61e2d [Geoff Anderson] Reverted jdk back to 6
f14c507 [Geoff Anderson] Removed mode = "test" from Vagrantfile and Vagrantfile.local examples. Updated testing README to clarify aws setup.
98b7253 [Geoff Anderson] Updated consumer tests to pre-populate kafka logs
e6a41f1 [Geoff Anderson] removed stray println
b15b24f [Geoff Anderson] leftover KafkaBenchmark in super call
0f75187 [Geoff Anderson] Rmoved stray allow_fail. kafka_benchmark_test -> benchmark_test
f469f84 [Geoff Anderson] Tweaked readme, added example Vagrantfile.local
3d73857 [Geoff Anderson] Merged downstream changes
42dcdb1 [Geoff Anderson] Tweaked behavior of stop_node, clean_node to generally fail fast
7f7c3e0 [Geoff Anderson] Updated setup.py for kafkatest
c60125c [Geoff Anderson] TestEndToEndLatency -> EndToEndLatency
4f476fe [Geoff Anderson] Moved aws scripts to vagrant directory
5af88fc [Geoff Anderson] Updated README to include aws quickstart
e5edf03 [Geoff Anderson] Updated example aws Vagrantfile.local
96533c3 [Geoff] Update aws-access-keys-commands
25a413d [Geoff] Update aws-example-Vagrantfile.local
884b20e [Geoff Anderson] Moved a bunch of files to kafkatest directory
fc7c81c [Geoff Anderson] added setup.py
632be12 [Geoff] Merge pull request #3 from confluentinc/verbose-client
51a94fd [Geoff Anderson] Use argparse4j instead of joptsimple. ThroughputThrottler now has more intuitive behavior when targetThroughput is 0.
a80a428 [Geoff Anderson] Added shell program for VerifiableProducer.
d586fb0 [Geoff Anderson] Updated comments to reflect that throttler is not message-specific
6842ed1 [Geoff Anderson] left out a file from last commit
1228eef [Geoff Anderson] Renamed throttler
9100417 [Geoff Anderson] Updated command-line options for VerifiableProducer. Extracted throughput logic to make it reusable.
0a5de8e [Geoff Anderson] Fixed checkstyle errors. Changed name to VerifiableProducer. Added synchronization for thread safety on println statements.
475423b [Geoff Anderson] Convert class to string before adding to json object.
bc009f2 [Geoff Anderson] Got rid of VerboseProducer in core (moved to clients)
c0526fe [Geoff Anderson] Updates per review comments.
8b4b1f2 [Geoff Anderson] Minor updates to VerboseProducer
2777712 [Geoff Anderson] Added some metadata to producer output.
da94b8c [Geoff Anderson] Added number of messages option.
07cd1c6 [Geoff Anderson] Added simple producer which prints status of produced messages to stdout.
a278988 [Geoff Anderson] fixed typos
f1914c3 [Liquan Pei] Merge pull request #2 from confluentinc/system_tests
81e4156 [Liquan Pei] Bootstrap Kafka system tests
KAFKA-2100; Client Error doesn't preserve or display original server error code when it is an unknown code; Reviewed by Gwen, Guozhang and Ewen
MINOR: Fixed ConsumerRecord constructor javadoc
Refactoring of ConsumerRecord made in https://github.com/apache/kafka/commit/0699ff2ce60abb466cab5315977a224f1a70a4da#diff-fafe8d3a3942f3c6394927881a9389b2 left ConsumerRecord constructor javadoc inconsistent with implementation.
This patch fixes ConsumerRecord constructor javadoc to be inline with implementation.
Author: Stevo Slavi? <ss...@gmail.com>
Reviewers: Ismael, Guozhang
Closes #85 from sslavic/patch-3 and squashes the following commits:
c289c4f [Stevo Slavi?] MINOR: Fixed ConsumerRecord constructor javadoc
KAFKA-2350; KafkaConsumer pause/resume API
Author: Jason Gustafson <ja...@confluent.io>
Reviewers: Ismael, Ashish, Guozhang
Closes #100 from hachikuji/KAFKA-2350 and squashes the following commits:
250e823 [Jason Gustafson] KAFKA-2350; KafkaConsumer pause/resume API
KAFKA-2026: fix logging of unsued options always showing null; reviewed by Ewen Cheslack-Postava and Jiangjie Qin
KAFKA-2384: Override commit message title in kafka-merge-pr.py
Author: Ismael Juma <is...@juma.me.uk>
Reviewers: Guozhang
Closes #105 from ijuma/kafka-2384-override-commit-message-title and squashes the following commits:
e042242 [Ismael Juma] Support overriding of commit message title in kafka-merge-pr.py
MINOR: Added to .gitignore Kafka server logs directory
When running Kafka server from sources, logs directory gets created in root of repository, and kafka server logs end up there. Currently that directory is not ignored by git.
This change adds root logs directory to .gitignore so that Kafka server logs are ignored and do not get tracked by git.
Author: Stevo Slavi? <ss...@gmail.com>
Reviewers: Ismael Juma
Closes #94 from sslavic/patch-7 and squashes the following commits:
c7b62a7 [Stevo Slavi?] MINOR: Added to .gitignore Kafka server logs
KAFKA-2386; increase timeouts for transient test failure in ConsumerCoordinatorResponseTests
There are two race conditions in the test case "testGenerationIdIncrementsOnRebalance." First, a delay before the second join group request can timeout the initial group and cause the generationId to unexpectedly reset. Second, a delay in the join group request handling will timeout the request itself and cause the test to fail. This commit doesn't address these race conditions, but increases the timeouts to make them more unlikely. If the problem reoccurs, then we'll probably need a better solution.
Author: Jason Gustafson <ja...@confluent.io>
Reviewers: Gwen Shapira <cs...@gmail.com>
Closes #107 from hachikuji/KAFKA-2386 and squashes the following commits:
a53460a [Jason Gustafson] KAFKA-2386; increase timeouts for transient test failure in ConsumerCoordinatorResponseTest
KAFKA-2055; Fix transient ConsumerBounceTest.testSeekAndCommitWithBrokerFailure failure.
…kerFailures failure;
Author: lvfangmin <lv...@gmail.com>
Reviewers: Guozhang
Closes #98 from lvfangmin/KAFKA-2055 and squashes the following commits:
057a1f1 [lvfangmin] KAFKA-2055; Fix transient ConsumerBounceTest.testSeekAndCommitWithBrokerFailures failure;
HOTFIX; Encode/decode to utf-8 for commit title IO in kafka-merge-pr.py after KAFKA-2384
This fix should be fine for Linux and OS X. Not sure about Windows though. This is a very specific fix for new functionality added in KAFKA-2384. There are other places where a similar error could occur, but are less likely.
The script doesn't really support Unicode input at the moment.
Author: Ismael Juma <is...@juma.me.uk>
Reviewers: Guozhang
Closes #109 from ijuma/kafka-2384-hotfix and squashes the following commits:
0ab8958 [Ismael Juma] Encode/decode to utf-8 for commit title IO in kafka-merge-pr.py
kafka-2205; Generalize TopicConfigManager to handle multiple entity configs; patched by Aditya Auradkar; reviewed Jun Rao
KAFKA-2405; Don't kill the JVM on session establishment failure
As noted in the JIRA https://issues.apache.org/jira/browse/KAFKA-2405 currently the KafkaHealthCheck causes the JVM to terminate in cases where session establishment with Zookeeper fails. I don't know if retrying (after a while) is a better way to fix this but at least, IMO, the session establishment failure shouldn't kill the JVM. This commit removes the `System.exit()` call.
Author: Jaikiran Pai <ja...@gmail.com>
Reviewers: Gwen Shapira <cs...@gmail.com>
Closes #111 from jaikiran/kafka-2405 and squashes the following commits:
0255fdb [Jaikiran Pai] KAFKA-2405 Don't kill the JVM on session establishment failure
KAFKA-2288; Follow-up to KAFKA-2249 - reduce logging and testing; Reviewd by Jun Rao
KAFKA-2407: Only create log directory when it will be used
Author: Grant Henke <gr...@gmail.com>
Reviewers: Gwen Shapira
Closes #115 from granthenke/log-fix and squashes the following commits:
de36138 [Grant Henke] Small comment fix
49a8dd4 [Grant Henke] KAFKA-2407: Only create log directory when it will be used
KAFKA-2401: Fix transient failure in ProducerSendTest.testCloseWithZeroTimeoutFromSenderThread
Author: Jiangjie Qin <be...@gmail.com>
Reviewers: Guozhang
Closes #113 from becketqin/KAFKA-2401 and squashes the following commits:
7d4223d [Jiangjie Qin] KAFKA-2401: fix transient failure in ProducerSendTest.testCloseWithZeroTimeoutFromSenderThread
KAFKA-2393: Correctly Handle InvalidTopicException in KafkaApis.getTo…
…picMetadata()
Author: Grant Henke <gr...@gmail.com>
Reviewers: Jun Rao <ju...@gmail.com>
Closes #117 from granthenke/invalid-topic and squashes the following commits:
0abda5f [Grant Henke] KAFKA-2393: Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata()
MINOR: ConsumerRecords are organized per topic partition
ConsumerRecords has records organized per topic partition, not per topic as ConsumerRecords javadoc suggested.
Author: Stevo Slavi? <ss...@gmail.com>
Reviewers: Jason, Guozhang
Closes #92 from sslavic/patch-6 and squashes the following commits:
b08a58d [Stevo Slavi?] MINOR: ConsumerRecords are organized per topic partition
MINOR: auto.offset.reset docs not in sync with validation
In this commit https://github.com/apache/kafka/commit/0699ff2ce60abb466cab5315977a224f1a70a4da#diff-5533ddc72176acd1c32f5abbe94aa672 among other things auto.offset.reset possible options were changed from smallest to earliest and from largest to latest, but not in documentation for that configuration property.
This patch fixes documentation for auto.offset.reset consumer configuration property so it is in sync with validation logic.
Author: Stevo Slavi? <ss...@gmail.com>
Reviewers: Jason, Ismael, Guozhang
Closes #91 from sslavic/patch-5 and squashes the following commits:
f4c9656 [Stevo Slavi?] MINOR: auto.offset.reset docs not in sync with validation
MINOR: Fixed javadoc for committed return value
If no offset has been committed, then committed method does not return (null) value, instead NoOffsetForPartitionException is thrown in that case.
Author: Stevo Slavi? <ss...@gmail.com>
Reviewers: Ismael, Guozhang
Closes #89 from sslavic/patch-4 and squashes the following commits:
5c0a152 [Stevo Slavi?] MINOR: Fixed javadoc for committed return value
MINOR: Fix typo in ReplicaVerificationTool output
Author: Andrew Otto <ac...@gmail.com>
Reviewers: Ismael Juma, Guozhang Wang
Closes #101 from ottomata/trunk and squashes the following commits:
10b76f3 [Andrew Otto] MINOR - Fix typo in ReplicaVerificationTool output
KAFKA-2400: Expose heartbeat interval in KafkaConsumer configuration
Author: Jason Gustafson <ja...@confluent.io>
Reviewers: Guozhang Wang
Closes #116 from hachikuji/KAFKA-2400 and squashes the following commits:
3c1b1dd [Jason Gustafson] KAFKA-2400; expose heartbeat interval in KafkaConsumer configuration
KAFKA-2340: Improve KafkaConsumer Fetcher test coverage
Author: Jason Gustafson <ja...@confluent.io>
Reviewers: Guozhang Wang
Closes #112 from hachikuji/KAFKA-2340 and squashes the following commits:
cc49ca2 [Jason Gustafson] KAFKA-2340; improve KafkaConsumer Fetcher test coverage
KAFKA-2413; New consumer's subscribe(Topic...) api fails if called more than once
Author: Onur Karaman <ok...@linkedin.com>
Reviewers: Ashish Singh, Ismael Juma, Jason Gustafson
Closes #122 from onurkaraman/KAFKA-2413 and squashes the following commits:
cc340fc [Onur Karaman] fix ConsumerCoordinator updateConsumer
Minor: Fixes to Selector's javadoc
Author: Ismael Juma <is...@juma.me.uk>
Closes #126 from ijuma/minor-selector-javadoc-fixes and squashes the following commits:
a26f529 [Ismael Juma] Minor fixes to Selector's javadoc
MINOR: documentation fix in StringDecoder
Author: Dave Cromberge <da...@gmail.com>
Closes #124 from davecromberge/documentation-fix and squashes the following commits:
7b9b1ce [Dave Cromberge] minor documentation fix to StringDecoder
KAFKA-2415; Fix transient failure in LogRecoveryTest
Author: Jiangjie Qin <be...@gmail.com>
Author: Jiangjie Qin <jq...@jqin-ld1.linkedin.biz>
Reviewers: Ismael Juma, Gwen Shapira
Closes #121 from becketqin/KAFKA-2415 and squashes the following commits:
7a9f453 [Jiangjie Qin] Addressed Ismael's comment
346103c [Jiangjie Qin] KAFKA-2415: Fix transient failure in LogRecoveryTest
KAFKA-1997: Follow-up patch, hardcode key/value serializer in mirror maker to byte serializer.
Hardcode the key/value serializer to ByteArraySerializer according to Jun’s comments.
Author: Jiangjie Qin <jq...@jqin-ld1.linkedin.biz>
Reviewers: Guozhang Wang
Closes #120 from becketqin/KAFKA-1997 and squashes the following commits:
7f2e5a6 [Jiangjie Qin] KAFKA-1997: Follow-up patch, hardcode key/value serializer in mirror maker to byte serializer.
KAFKA-2202: fix consumerTimeoutMs computation on ConsumerPerformance; reviewed by Guozhang Wang
KAFKA-2336: Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment; Reviewed by Jiangjie Qin, Gwen Shapira
KAFKA-2408: ConsoleConsumerService direct log output to file
console consumer writes to System.out, while (some) log4j loggers operate in other threads.
This occasionally led to funky interleaved output which disrupted parsing of consumed messages by ConsoleConsumerService, leading to spurious test failures.
This fix directs log output to a separate file.
Author: Geoff Anderson <ge...@confluent.io>
Reviewers: Ewen Cheslack-Postava
Closes #123 from granders/KAFKA-2408 and squashes the following commits:
247b0e0 [Geoff Anderson] Updated line counting to use wc -l
66d6f4f [Geoff Anderson] lower -> uperrcase constants
e67f554 [Geoff Anderson] Changed incorrect license header
af67e01 [Geoff Anderson] Merged in upstream trunk
8f89044 [Geoff Anderson] Added another lifecycle check. Wait for log file to exist before exmaning contents.
521a84b [Geoff Anderson] Updated console consumer to directo log output directly to file rather than stdout
KAFKA-2300: Error in controller log when broker tries to rejoin cluster
Author: flavio junqueira <fp...@apache.org>
Reviewers: Ismael Juma, Guozhang Wang
Closes #102 from fpj/2300 and squashes the following commits:
7bd2edb [flavio junqueira] KAFKA-2300: Removed unnecessary s" occurrences.
aa6ec90 [flavio junqueira] KAFKA-2300: Wrapped all occurences of sendRequestToBrokers with try/catch and fixed string typo.
f1261b1 [flavio junqueira] Fixed some style issues.
9b6390a [flavio junqueira] Updated package name and removed unnecessary imports.
dbd1bf3 [flavio junqueira] KAFKA-2300: Error in controller log when broker tries to rejoin cluster
MINOR: Fix hard coded strings in ProduceResponse
Author: Grant Henke <gr...@gmail.com>
Reviewers: Ismael Juma, Ewen Cheslack-Postava and Guozhang Wang
Closes #131 from granthenke/minor-string and squashes the following commits:
3c6250d [Grant Henke] MINOR: Fix hard coded strings in ProduceResponse
KAFKA-2429: Add annotations to mark classes as stable/unstable
This also marks the consumer as unstable to show an example of using these annotations.
Author: Ewen Cheslack-Postava <me...@ewencp.org>
Reviewers: Gwen Shapira
Closes #133 from ewencp/stability-annotations and squashes the following commits:
09c15c3 [Ewen Cheslack-Postava] KAFKA-2429: Add annotations to mark classes as stable/unstable
KAFKA-1782: fix JUnit3 Misuse
Author: Ewen Cheslack-Postava <me...@ewencp.org>
Reviewers: Ewen Cheslack-Postava, Guozhang Wang
Closes #135 from ewencp/kafka-1782-junit3-misusage and squashes the following commits:
0ae6258 [Ewen Cheslack-Postava] KAFKA-1782: Junit3 Misusage
KAFKA-2430; Listing of PR commits in commit message should be optional
If there is a single commit in the PR, then it's never listed.
Author: Ismael Juma <is...@juma.me.uk>
Reviewers: Guozhang Wang
Closes #136 from ijuma/kafka-2430-optional-listing-commits and squashes the following commits:
64f1aec [Ismael Juma] Listing of PR commits in commit message should be optional
KAFKA-2406: Throttle ISR propagation
This is a follow up patch for KAFKA-2406. Further test to verify if this change alone is enough to solve the problem or not.
Author: Jiangjie Qin <be...@gmail.com>
Author: Jiangjie Qin <jq...@jqin-ld1.linkedin.biz>
Reviewers: Jun Rao <ju...@gmail.com>
Closes #114 from becketqin/KAFKA-2406
MINOR: expose vagrant base box as variable
Added base_box variable to Vagrantfile. This makes it possible to override the base box in Vagrantfile.local.
Author: Geoff Anderson <ge...@confluent.io>
Reviewers: Guozhang Wang
Closes #137 from granders/minor-expose-vagrant-box and squashes the following commits:
44936f7 [Geoff Anderson] Added base_box variable to Vagrantfile. This makes it possible to override the base box in Vagrantfile.local.
KAFKA-2366; Initial patch for Copycat
This is an initial patch implementing the basics of Copycat for KIP-26.
The intent here is to start a review of the key pieces of the core API and get a reasonably functional, baseline, non-distributed implementation of Copycat in place to get things rolling. The current patch has a number of known issues that need to be addressed before a final version:
* Some build-related issues. Specifically, requires some locally-installed dependencies (see below), ignores checkstyle for the runtime data library because it's lifted from Avro currently and likely won't last in its current form, and some Gradle task dependencies aren't quite right because I haven't gotten rid of the dependency on `core` (which should now be an easy patch since new consumer groups are in a much better state).
* This patch currently depends on some Confluent trunk code because I prototyped with our Avro serializers w/ schema-registry support. We need to figure out what we want to provide as an example built-in set of serializers. Unlike core Kafka where we could ignore the issue, providing only ByteArray or String serializers, this is pretty central to how Copycat works.
* This patch uses a hacked up version of Avro as its runtime data format. Not sure if we want to go through the entire API discussion just to get some basic code committed, so I filed KAFKA-2367 to handle that separately. The core connector APIs and the runtime data APIs are entirely orthogonal.
* This patch needs some updates to get aligned with recent new consumer changes (specifically, I'm aware of the ConcurrentModificationException issue on exit). More generally, the new consumer is in flux but Copycat depends on it, so there are likely to be some negative interactions.
* The layout feels a bit awkward to me right now because I ported it from a Maven layout. We don't have nearly the same level of granularity in Kafka currently (core and clients, plus the mostly ignored examples, log4j-appender, and a couple of contribs). We might want to reorganize, although keeping data+api separate from runtime and connector plugins is useful for minimizing dependencies.
* There are a variety of other things (e.g., I'm not happy with the exception hierarchy/how they are currently handled, TopicPartition doesn't really need to be duplicated unless we want Copycat entirely isolated from the Kafka APIs, etc), but I expect those we'll cover in the review.
Before commenting on the patch, it's probably worth reviewing https://issues.apache.org/jira/browse/KAFKA-2365 and https://issues.apache.org/jira/browse/KAFKA-2366 to get an idea of what I had in mind for a) what we ultimately want with all the Copycat patches and b) what we aim to cover in this initial patch. My hope is that we can use a WIP patch (after the current obvious deficiencies are addressed) while recognizing that we want to make iterative progress with a bunch of subsequent PRs.
Author: Ewen Cheslack-Postava <me...@ewencp.org>
Reviewers: Ismael Juma, Gwen Shapira
Closes #99 from ewencp/copycat and squashes the following commits:
a3a47a6 [Ewen Cheslack-Postava] Simplify Copycat exceptions, make them a subclass of KafkaException.
8c108b0 [Ewen Cheslack-Postava] Rename Coordinator to Herder to avoid confusion with the consumer coordinator.
7bf8075 [Ewen Cheslack-Postava] Make Copycat CLI speific to standalone mode, clean up some config and get rid of config storage in standalone mode.
656a003 [Ewen Cheslack-Postava] Clarify and expand the explanation of the Copycat Coordinator interface.
c0e5fdc [Ewen Cheslack-Postava] Merge remote-tracking branch 'origin/trunk' into copycat
0fa7a36 [Ewen Cheslack-Postava] Mark Copycat classes as unstable and reduce visibility of some classes where possible.
d55d31e [Ewen Cheslack-Postava] Reorganize Copycat code to put it all under one top-level directory.
b29cb2c [Ewen Cheslack-Postava] Merge remote-tracking branch 'origin/trunk' into copycat
d713a21 [Ewen Cheslack-Postava] Address Gwen's review comments.
6787a85 [Ewen Cheslack-Postava] Make Converter generic to match serializers since some serialization formats do not require a base class of Object; update many other classes to have generic key and value class type parameters to match this change.
b194c73 [Ewen Cheslack-Postava] Split Copycat converter option into two options for key and value.
0b5a1a0 [Ewen Cheslack-Postava] Normalize naming to use partition for both source and Kafka, adjusting naming in CopycatRecord classes to clearly differentiate.
e345142 [Ewen Cheslack-Postava] Remove Copycat reflection utils, use existing Utils and ConfigDef functionality from clients package.
be5c387 [Ewen Cheslack-Postava] Minor cleanup
122423e [Ewen Cheslack-Postava] Style cleanup
6ba87de [Ewen Cheslack-Postava] Remove most of the Avro-based mock runtime data API, only preserving enough schema functionality to support basic primitive types for an initial patch.
4674d13 [Ewen Cheslack-Postava] Address review comments, clean up some code styling.
25b5739 [Ewen Cheslack-Postava] Fix sink task offset commit concurrency issue by moving it to the worker thread and waking up the consumer to ensure it exits promptly.
0aefe21 [Ewen Cheslack-Postava] Add log4j settings for Copycat.
220e42d [Ewen Cheslack-Postava] Replace Avro serializer with JSON serializer.
1243a7c [Ewen Cheslack-Postava] Merge remote-tracking branch 'origin/trunk' into copycat
5a618c6 [Ewen Cheslack-Postava] Remove offset serializers, instead reusing the existing serializers and removing schema projection support.
e849e10 [Ewen Cheslack-Postava] Remove duplicated TopicPartition implementation.
dec1379 [Ewen Cheslack-Postava] Switch to using new consumer coordinator instead of manually assigning partitions. Remove dependency of copycat-runtime on core.
4a9b4f3 [Ewen Cheslack-Postava] Add some helpful Copycat-specific build and test targets that cover all Copycat packages.
31cd1ca [Ewen Cheslack-Postava] Add CLI tools for Copycat.
e14942c [Ewen Cheslack-Postava] Add Copycat file connector.
0233456 [Ewen Cheslack-Postava] Add copycat-avro and copycat-runtime
11981d2 [Ewen Cheslack-Postava] Add copycat-data and copycat-api
KAFKA-2084; Add per-client-id byte-rate metrics and quota manager; reviewed by Joel Koshy, Dong Lin, Jun Rao and Edward Ribeiro
KAFKA-1782: Follow up - add missing @Test annotations.
Author: Ewen Cheslack-Postava <me...@ewencp.org>
Reviewers: Guozhang
Closes #140 from ewencp/kafka-1782-followup and squashes the following commits:
fe36bd1 [Ewen Cheslack-Postava] Fix missing annotation of Before and After in ControllerFailoverTest.
1dcaf39 [Ewen Cheslack-Postava] KAFKA-1782: Follow up - add missing @Test annotations.
KAFKA-2438; add maxParallelForks to build.gradle to speedup tests.
Author: Sriharsha Chintalapani <ha...@hortonworks.com>
Reviewers: Gwen Shapira
Closes #143 from harshach/KAFKA-2438
Fixing bug
Addressing Joel's comments
Merging
Chaning variable name
Addressing Joel's comments
Addressing Joel's comments
Addressing comments
Diffs (updated)
-----
.gitignore 1f3ba7dbbf524ee1cc71164065e667ffe78952a1
.rat-excludes 01d629817c8a4b2db3c11d8e6a7fa13a7e4856c2
CONTRIBUTING.md PRE-CREATION
README.md a9a5d1ef2b74440a7a63fef02078a1f54f107b8f
Vagrantfile 55c67ddda458199c0b5168c8f0c367d368a6f402
bin/copycat-standalone.sh PRE-CREATION
bin/kafka-configs.sh PRE-CREATION
bin/kafka-run-class.sh 8c3fa286145344fb527307012c1d1000d855aa18
bin/kafka-verifiable-producer.sh PRE-CREATION
build.gradle 30d1cf2f1ff9ed3f86a060da8099bb0774b4cf91
checkstyle/import-control.xml f2e6cec267e67ce8e261341e373718e14a8e8e03
clients/src/main/java/org/apache/kafka/clients/ClientRequest.java ed4c0d98596cc294757f35df8c8cbc8e36ff42de
clients/src/main/java/org/apache/kafka/clients/Metadata.java 07f1cdb1fe920b0c7a5f2d101ddc40c689e1b247
clients/src/main/java/org/apache/kafka/clients/NetworkClient.java 48fe7961e2215372d8033ece4af739ea06c6457b
clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java 8f587bc0705b65b3ef37c86e0c25bb43ab8803de
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerCommitCallback.java PRE-CREATION
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java 1e905240e7459a4a0a0573ae5d8ac19217cae197
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRebalanceCallback.java 74dfdba0ecbca04947adba5eabb1cb5190a0cd5f
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java 49d9527afeb253617c1fc5bd8ab9c9dd4a6f0ecb
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java 1ca75f83d3667f7d01da1ae2fd9488fb79562364
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerWakeupException.java PRE-CREATION
clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java d1d1ec178f60dc47d408f52a89e52886c1a093a2
clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java f50da825756938c193d7f07bee953e000e2627d9
clients/src/main/java/org/apache/kafka/clients/consumer/OffsetResetStrategy.java PRE-CREATION
clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkClient.java PRE-CREATION
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Coordinator.java c1496a0851526f3c7d3905ce4bdff2129c83a6c1
clients/src/main/java/org/apache/kafka/clients/consumer/internals/DelayedTask.java PRE-CREATION
clients/src/main/java/org/apache/kafka/clients/consumer/internals/DelayedTaskQueue.java PRE-CREATION
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 56281ee15cc33dfc96ff64d5b1e596047c7132a4
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Heartbeat.java e7cfaaad296fa6e325026a5eee1aaf9b9c0fe1fe
clients/src/main/java/org/apache/kafka/clients/consumer/internals/NoAvailableBrokersException.java PRE-CREATION
clients/src/main/java/org/apache/kafka/clients/consumer/internals/RequestFuture.java PRE-CREATION
clients/src/main/java/org/apache/kafka/clients/consumer/internals/RequestFutureAdapter.java PRE-CREATION
clients/src/main/java/org/apache/kafka/clients/consumer/internals/RequestFutureListener.java PRE-CREATION
clients/src/main/java/org/apache/kafka/clients/consumer/internals/SendFailedException.java PRE-CREATION
clients/src/main/java/org/apache/kafka/clients/consumer/internals/StaleMetadataException.java PRE-CREATION
clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java cee75410127dd1b86c1156563003216d93a086b3
clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java 5a37580ec69af08b97cf5b43b241790ba8c129dd
clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java e66491cc82f11641df6516e7d7abb4a808c27368
clients/src/main/java/org/apache/kafka/clients/producer/internals/ErrorLoggingCallback.java 678d1c6accd2a119b54dbb4740604d25fa8f9de4
clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java 87dbd64f30f35dbf31d3820f9819a63c6c0d1e58
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 07e65d4a54ba4eef5b787eba3e71cbe9f6a920bd
clients/src/main/java/org/apache/kafka/clients/tools/ProducerPerformance.java 13f4d5958052afcc8ad66eacbcae50b6fd149398
clients/src/main/java/org/apache/kafka/common/annotation/InterfaceStability.java PRE-CREATION
clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java c4fa058692f50abb4f47bd344119d805c60123f5
clients/src/main/java/org/apache/kafka/common/errors/ConsumerCoordinatorNotAvailableException.java PRE-CREATION
clients/src/main/java/org/apache/kafka/common/errors/DisconnectException.java PRE-CREATION
clients/src/main/java/org/apache/kafka/common/errors/IllegalGenerationException.java PRE-CREATION
clients/src/main/java/org/apache/kafka/common/errors/NotCoordinatorForConsumerException.java PRE-CREATION
clients/src/main/java/org/apache/kafka/common/errors/OffsetLoadInProgressException.java PRE-CREATION
clients/src/main/java/org/apache/kafka/common/errors/UnknownConsumerIdException.java PRE-CREATION
clients/src/main/java/org/apache/kafka/common/metrics/Quota.java d82bb0c055e631425bc1ebbc7d387baac76aeeaa
clients/src/main/java/org/apache/kafka/common/metrics/QuotaViolationException.java a451e5385c9eca76b38b425e8ac856b2715fcffe
clients/src/main/java/org/apache/kafka/common/metrics/Sensor.java ca823fd4639523018311b814fde69b6177e73b97
clients/src/main/java/org/apache/kafka/common/metrics/stats/Rate.java 98429da34418f7f1deba1b5e44e2e6025212edb3
clients/src/main/java/org/apache/kafka/common/network/Selector.java effb1e63081ed2c1fff6d08d4ecdf8a3cb43e40a
clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 5b898c8f8ad5d0469f469b600c4b2eb13d1fc662
clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java b2db2403868b1e7361b8514cfed2e76ef785edee
clients/src/main/java/org/apache/kafka/common/requests/AbstractRequest.java 5e5308ec0e333179a9abbf4f3b750ea25ab57967
clients/src/main/java/org/apache/kafka/common/requests/ConsumerMetadataRequest.java 04b90bfe62456a6739fe0299f1564dbd1850fe58
clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 8686d83aa52e435c6adafbe9ff4bd1602281072a
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
clients/src/main/java/org/apache/kafka/common/requests/HeartbeatRequest.java 51d081fa296fd7c170a90a634d432067afcfe772
clients/src/main/java/org/apache/kafka/common/requests/JoinGroupRequest.java 6795682258e6b329cc3caa245b950b4dbcf0cf45
clients/src/main/java/org/apache/kafka/common/requests/JoinGroupResponse.java 8d418cd24cf6d105e9687a4a2492b8ed13738338
clients/src/main/java/org/apache/kafka/common/requests/ListOffsetRequest.java 19267ee8aad5a2f5a84cecd6fc563f00329d5035
clients/src/main/java/org/apache/kafka/common/requests/MetadataRequest.java 7e0ce159a2ddd041fc06116038bd5831bbca278b
clients/src/main/java/org/apache/kafka/common/requests/MetadataResponse.java 44e2ce61899889601b6aee71fa7f7ddb4a65a255
clients/src/main/java/org/apache/kafka/common/requests/OffsetCommitRequest.java 8bf6cbb79a92b0878096e099ec9169d21e6d7023
clients/src/main/java/org/apache/kafka/common/requests/OffsetCommitResponse.java 70844d65369f6ff300cbeb513dbb6650050c7eec
clients/src/main/java/org/apache/kafka/common/requests/OffsetFetchRequest.java deec1fa480d5a5c5884a1c007b076aa64e902472
clients/src/main/java/org/apache/kafka/common/requests/OffsetFetchResponse.java 512a0ef7e619d54e74122c38119209f5cf9590e3
clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java fabeae3083a8ea55cdacbb9568f3847ccd85bab4
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 37ec0b79beafcf5735c386b066eb319fb697eff5
clients/src/main/java/org/apache/kafka/common/utils/Utils.java f73eedb030987f018d8446bb1dcd98d19fa97331
clients/src/test/java/org/apache/kafka/clients/MetadataTest.java 928087d29deb80655ca83726c1ebc45d76468c1f
clients/src/test/java/org/apache/kafka/clients/MockClient.java d9c97e966c0e2fb605b67285f4275abb89f8813e
clients/src/test/java/org/apache/kafka/clients/consumer/MockConsumerTest.java 677edd385f35d4262342b567262c0b874876d25b
clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkClientTest.java PRE-CREATION
clients/src/test/java/org/apache/kafka/clients/consumer/internals/CoordinatorTest.java b06c4a73e2b4e9472cd772c8bc32bf4a29f431bb
clients/src/test/java/org/apache/kafka/clients/consumer/internals/DelayedTaskQueueTest.java PRE-CREATION
clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java 419541011d652becf0cda7a5e62ce813cddb1732
clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatTest.java ecc78cedf59a994fcf084fa7a458fe9ed5386b00
clients/src/test/java/org/apache/kafka/clients/consumer/internals/RequestFutureTest.java PRE-CREATION
clients/src/test/java/org/apache/kafka/clients/consumer/internals/SubscriptionStateTest.java e000cf8e10ebfacd6c9ee68d7b88ff8c157f73c6
clients/src/test/java/org/apache/kafka/clients/producer/MockProducerTest.java 6372f1a7f7f77d96ba7be05eb927c004f7fefb73
clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
clients/src/test/java/org/apache/kafka/common/metrics/MetricsTest.java 544e120594de78c43581a980b1e4087b4fb98ccb
clients/src/test/java/org/apache/kafka/common/network/SelectorTest.java d23b4b6a7060eeefa9f47f292fd818c881d789c1
clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java e3cc1967e407b64cc734548c19e30de700b64ba8
clients/src/test/java/org/apache/kafka/common/utils/UtilsTest.java 2ebe3c21f611dc133a2dbb8c7dfb0845f8c21498
clients/src/test/java/org/apache/kafka/test/MockSerializer.java e75d2e4e58ae0cdbe276d3a3b652e47795984791
config/copycat-console-sink.properties PRE-CREATION
config/copycat-console-source.properties PRE-CREATION
config/copycat-file-sink.properties PRE-CREATION
config/copycat-file-source.properties PRE-CREATION
config/copycat-log4j.properties PRE-CREATION
config/copycat-standalone.properties PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/connector/Connector.java PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/connector/ConnectorContext.java PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/connector/CopycatRecord.java PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/connector/Task.java PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/errors/CopycatException.java PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/sink/SinkConnector.java PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/sink/SinkRecord.java PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/sink/SinkTask.java PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/sink/SinkTaskContext.java PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/source/SourceConnector.java PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/source/SourceRecord.java PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/source/SourceTask.java PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/source/SourceTaskContext.java PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/storage/Converter.java PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/storage/OffsetStorageReader.java PRE-CREATION
copycat/api/src/main/java/org/apache/kafka/copycat/util/ConnectorUtils.java PRE-CREATION
copycat/api/src/test/java/org/apache/kafka/copycat/connector/ConnectorReconfigurationTest.java PRE-CREATION
copycat/api/src/test/java/org/apache/kafka/copycat/util/ConnectorUtilsTest.java PRE-CREATION
copycat/data/src/main/java/org/apache/kafka/copycat/data/DataRuntimeException.java PRE-CREATION
copycat/data/src/main/java/org/apache/kafka/copycat/data/DataTypeException.java PRE-CREATION
copycat/data/src/main/java/org/apache/kafka/copycat/data/ObjectProperties.java PRE-CREATION
copycat/data/src/main/java/org/apache/kafka/copycat/data/Schema.java PRE-CREATION
copycat/data/src/main/java/org/apache/kafka/copycat/data/SchemaBuilder.java PRE-CREATION
copycat/data/src/main/java/org/apache/kafka/copycat/data/SchemaParseException.java PRE-CREATION
copycat/file/src/main/java/org/apache/kafka/copycat/file/FileStreamSinkConnector.java PRE-CREATION
copycat/file/src/main/java/org/apache/kafka/copycat/file/FileStreamSinkTask.java PRE-CREATION
copycat/file/src/main/java/org/apache/kafka/copycat/file/FileStreamSourceConnector.java PRE-CREATION
copycat/file/src/main/java/org/apache/kafka/copycat/file/FileStreamSourceTask.java PRE-CREATION
copycat/file/src/test/java/org/apache/kafka/copycat/file/FileStreamSinkConnectorTest.java PRE-CREATION
copycat/file/src/test/java/org/apache/kafka/copycat/file/FileStreamSinkTaskTest.java PRE-CREATION
copycat/file/src/test/java/org/apache/kafka/copycat/file/FileStreamSourceConnectorTest.java PRE-CREATION
copycat/file/src/test/java/org/apache/kafka/copycat/file/FileStreamSourceTaskTest.java PRE-CREATION
copycat/json/src/main/java/org/apache/kafka/copycat/json/JsonConverter.java PRE-CREATION
copycat/json/src/main/java/org/apache/kafka/copycat/json/JsonDeserializer.java PRE-CREATION
copycat/json/src/main/java/org/apache/kafka/copycat/json/JsonSchema.java PRE-CREATION
copycat/json/src/main/java/org/apache/kafka/copycat/json/JsonSerializer.java PRE-CREATION
copycat/json/src/test/java/org/apache/kafka/copycat/json/JsonConverterTest.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/cli/CopycatStandalone.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/cli/WorkerConfig.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/runtime/ConnectorConfig.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/runtime/Copycat.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/runtime/Herder.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/runtime/SinkTaskContextImpl.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/runtime/SourceTaskOffsetCommitter.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/runtime/Worker.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/runtime/WorkerSinkTask.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/runtime/WorkerSinkTaskThread.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/runtime/WorkerSourceTask.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/runtime/WorkerTask.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/runtime/standalone/StandaloneConnectorContext.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/runtime/standalone/StandaloneHerder.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/storage/FileOffsetBackingStore.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/storage/MemoryOffsetBackingStore.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/storage/OffsetBackingStore.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/storage/OffsetStorageReaderImpl.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/storage/OffsetStorageWriter.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/util/Callback.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/util/ConnectorTaskId.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/util/FutureCallback.java PRE-CREATION
copycat/runtime/src/main/java/org/apache/kafka/copycat/util/ShutdownableThread.java PRE-CREATION
copycat/runtime/src/test/java/org/apache/kafka/copycat/runtime/WorkerSinkTaskTest.java PRE-CREATION
copycat/runtime/src/test/java/org/apache/kafka/copycat/runtime/WorkerSourceTaskTest.java PRE-CREATION
copycat/runtime/src/test/java/org/apache/kafka/copycat/runtime/WorkerTest.java PRE-CREATION
copycat/runtime/src/test/java/org/apache/kafka/copycat/runtime/standalone/StandaloneHerderTest.java PRE-CREATION
copycat/runtime/src/test/java/org/apache/kafka/copycat/storage/FileOffsetBackingStoreTest.java PRE-CREATION
copycat/runtime/src/test/java/org/apache/kafka/copycat/storage/OffsetStorageWriterTest.java PRE-CREATION
copycat/runtime/src/test/java/org/apache/kafka/copycat/util/MockTime.java PRE-CREATION
copycat/runtime/src/test/java/org/apache/kafka/copycat/util/ShutdownableThreadTest.java PRE-CREATION
copycat/runtime/src/test/java/org/apache/kafka/copycat/util/TestBackgroundThreadExceptionHandler.java PRE-CREATION
copycat/runtime/src/test/java/org/apache/kafka/copycat/util/ThreadedTest.java PRE-CREATION
core/src/main/scala/kafka/admin/AdminUtils.scala f06edf41c732a7b794e496d0048b0ce6f897e72b
core/src/main/scala/kafka/admin/ConfigCommand.scala PRE-CREATION
core/src/main/scala/kafka/admin/ReassignPartitionsCommand.scala 912b718ec9ad58b1c2d42337ed85bbb1b88162d6
core/src/main/scala/kafka/admin/TopicCommand.scala dacbdd089f96c67b7bbc1c0cd36490d4fe094c1b
core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
core/src/main/scala/kafka/api/TopicMetadata.scala bd866bc8d65ef21d7ac24d8c1234ff7eecfe9d6e
core/src/main/scala/kafka/cluster/EndPoint.scala e9008e6d758be04bebe3cc70952c13dc55dc58fb
core/src/main/scala/kafka/cluster/Partition.scala 730a232482fdf77be5704cdf5941cfab3828db88
core/src/main/scala/kafka/common/OffsetMetadataAndError.scala 6b4242c7cd1df9b3465db0fec35a25102c76cd60
core/src/main/scala/kafka/common/Topic.scala ad759786d1c22f67c47808c0b8f227eb2b1a9aa8
core/src/main/scala/kafka/common/TopicAlreadyMarkedForDeletionException.scala PRE-CREATION
core/src/main/scala/kafka/common/TopicAndPartition.scala df3db912f5daef6a25b4b2dd2220d2cc3795bce6
core/src/main/scala/kafka/consumer/ConsumerConfig.scala 01993177130ff9d9f6106afa2783b24ebcedd20c
core/src/main/scala/kafka/consumer/SimpleConsumer.scala c16f7edd322709060e54c77eb505c44cbd77a4ec
core/src/main/scala/kafka/controller/ControllerChannelManager.scala 9f521fa7a54a8038ca6051d121c73b76519810e7
core/src/main/scala/kafka/controller/KafkaController.scala 69bba243a9a511cc5292b43da0cc48e421a428b0
core/src/main/scala/kafka/controller/PartitionLeaderSelector.scala 3b15ab4eef22c6f50a7483e99a6af40fb55aca9f
core/src/main/scala/kafka/controller/PartitionStateMachine.scala 92fd92d135b24931de970347eae49ad24dd1f5c4
core/src/main/scala/kafka/controller/TopicDeletionManager.scala 64ecb499f24bc801d48f86e1612d927cc08e006d
core/src/main/scala/kafka/coordinator/ConsumerCoordinator.scala 51e89c87ee2c20fc7f976536f01fa1055fb8e670
core/src/main/scala/kafka/coordinator/CoordinatorMetadata.scala c39e6de34ee531c6dfa9107b830752bd7f8fbe59
core/src/main/scala/kafka/javaapi/consumer/ConsumerConnector.java ca74ca8abf03478b6de4ec1f82fbac379e7603f1
core/src/main/scala/kafka/log/FileMessageSet.scala 2522604bd985c513527fa0c863a7df677ff7a503
core/src/main/scala/kafka/log/Log.scala 84e7b8fe9dd014884b60c4fbe13c835cf02a40e4
core/src/main/scala/kafka/log/LogCleaner.scala c9ade7208798fbd92d4ff49e183fe5f8925c82a9
core/src/main/scala/kafka/log/LogConfig.scala a907da09e1ccede3b446459225e407cd1ae6d8b3
core/src/main/scala/kafka/log/LogManager.scala e781ebac2677ebb22e0c1fef0cf7e5ad57c74ea4
core/src/main/scala/kafka/log/LogSegment.scala ed039539ac18ea4d65144073915cf112f7374631
core/src/main/scala/kafka/log/OffsetIndex.scala a1082aefd60ed4207f90feec29e700471d175ef8
core/src/main/scala/kafka/network/RequestChannel.scala 357d8b97cb336857500213efade77950833c2096
core/src/main/scala/kafka/network/SocketServer.scala 91319fa010b140cca632e5fa8050509bd2295fc9
core/src/main/scala/kafka/producer/KafkaLog4jAppender.scala 5d36a019e3dbfb93737a9cd23404dcd1c5d836d1
core/src/main/scala/kafka/serializer/Decoder.scala 54d0b935704ae4f2f9763f77c1ca4b859937d31a
core/src/main/scala/kafka/server/AbstractFetcherThread.scala 83fc47417dd7fe4edf030217fa7fd69d99b170b0
core/src/main/scala/kafka/server/ClientQuotaManager.scala PRE-CREATION
core/src/main/scala/kafka/server/ConfigHandler.scala PRE-CREATION
core/src/main/scala/kafka/server/DelayedOperation.scala 123078d97a7bfe2121655c00f3b2c6af21c53015
core/src/main/scala/kafka/server/KafkaApis.scala d63bc18d795a6f0e6994538ca55a9a46f7fb8ffd
core/src/main/scala/kafka/server/KafkaConfig.scala 2d75186a110075e0c322db4b9f7a8c964a7a3e88
core/src/main/scala/kafka/server/KafkaHealthcheck.scala ea0c996ae5d1c03428d8cbc7434b7ea5b53e63ca
core/src/main/scala/kafka/server/KafkaServer.scala b320ce9f6a12c0ee392e91beb82e8804d167f9f4
core/src/main/scala/kafka/server/OffsetManager.scala 5cca85cf727975f6d3acb2223fd186753ad761dc
core/src/main/scala/kafka/server/ReplicaFetcherThread.scala b31b432a226ba79546dd22ef1d2acbb439c2e9a3
core/src/main/scala/kafka/server/ReplicaManager.scala 59c9bc3ac3a8afc07a6f8c88c5871304db588d17
core/src/main/scala/kafka/server/ThrottledResponse.scala PRE-CREATION
core/src/main/scala/kafka/server/TopicConfigManager.scala b675a7e45ea4f4179f8b15fe221fd988aff13aa0
core/src/main/scala/kafka/tools/ConsumerOffsetChecker.scala 3d52f62c88a509a655cf1df6232b738c25fa9b69
core/src/main/scala/kafka/tools/ConsumerPerformance.scala 903318d15893af08104a97499798c9ad0ba98013
core/src/main/scala/kafka/tools/MirrorMaker.scala 459aaec4c3d96cd3d1c2ac6363ba4553d77dc36b
core/src/main/scala/kafka/tools/ProducerPerformance.scala 0ebfa59090a823d7a9f81405ac81d6907a5cfcc5
core/src/main/scala/kafka/tools/ReplicaVerificationTool.scala 1366172243b41de6aa9f90fca9848dbb96921c6d
core/src/main/scala/kafka/utils/CoreUtils.scala d0a8fa701564b4c13b3cd6501e1b6218d77e8e06
core/src/main/scala/kafka/utils/ReplicationUtils.scala 60687332b4c9bee4d4c0851314cfb4b02d5d3489
core/src/main/scala/kafka/utils/ShutdownableThread.scala fc226c863095b7761290292cd8755cd7ad0f155c
core/src/main/scala/kafka/utils/ZkUtils.scala 78475e3d5ec477cef00caeaa34ff2d196466be96
core/src/test/scala/integration/kafka/api/ConsumerBounceTest.scala f56096b826bdbf760411a54ba067a6a83eca8a10
core/src/test/scala/integration/kafka/api/ConsumerTest.scala 17b17b9b1520c7cc2e2ba96cdb1f9ff06e47bcad
core/src/test/scala/integration/kafka/api/IntegrationTestHarness.scala 07b1ff47bfc3cd3f948c9533c8dc977fa36d996f
core/src/test/scala/integration/kafka/api/ProducerBounceTest.scala ce70a0a449883723a9b59ea48da34ba30b3f6daf
core/src/test/scala/integration/kafka/api/ProducerCompressionTest.scala 83de81cb3f79a6966dd5ef462733d8a22cd6d467
core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala ee94011894b46864614b97bbd2a98375a7d3f20b
core/src/test/scala/integration/kafka/api/ProducerSendTest.scala 9ce4bd5ee130ce3cb252b2883a3fd3c9acd742a5
core/src/test/scala/integration/kafka/api/QuotasTest.scala PRE-CREATION
core/src/test/scala/kafka/tools/TestEndToEndLatency.scala 99b77a1ffbd8b957e6ac4f3d86bb36fcb1340a07
core/src/test/scala/other/kafka/StressTestLog.scala c0e248d669c7bd653f512af7f72d895c38772f83
core/src/test/scala/other/kafka/TestLinearWriteSpeed.scala 3034c4f9b0d026e25ce045689d9a9f99a59a10ec
core/src/test/scala/other/kafka/TestOffsetManager.scala 8047da4ff5b995611bdae53f4f04d0791f1a34b4
core/src/test/scala/unit/kafka/KafkaConfigTest.scala 4cb92de169d465ccdafb8c24b5b46b0f259c8e43
core/src/test/scala/unit/kafka/admin/AddPartitionsTest.scala df5c6ba20f01e497ce896af790cbab40369f1776
core/src/test/scala/unit/kafka/admin/AdminTest.scala efb2f8e79b3faef78722774b951fea828cd50374
core/src/test/scala/unit/kafka/admin/ConfigCommandTest.scala PRE-CREATION
core/src/test/scala/unit/kafka/admin/DeleteConsumerGroupTest.scala 1913ad6d3e6eb29a0c939c4f59f6b688c8c925fa
core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala fa8ce259a2832ab86f9dda8c1d409b2c42d43ae9
core/src/test/scala/unit/kafka/admin/TopicCommandTest.scala c7136f20972614ac47aa57ab13e3c94ef775a4b7
core/src/test/scala/unit/kafka/api/ApiUtilsTest.scala 255442526d94157b7a0b5a92e1d6a900aacb7536
core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 5717165f2344823fabe8f7cfafae4bb8af2d949a
core/src/test/scala/unit/kafka/cluster/BrokerEndPointTest.scala bb2506ca186bb2e5a606263fb0c50ae95769ff68
core/src/test/scala/unit/kafka/common/ConfigTest.scala 7bff96caca1dc715a81b48cec84cb3c163f7faa3
core/src/test/scala/unit/kafka/common/TopicTest.scala 0fb25880c24adef906cd06359b624e7c8eb94ca6
core/src/test/scala/unit/kafka/consumer/ConsumerIteratorTest.scala db5302ff02851f9f1a59419b1e071286bff0e142
core/src/test/scala/unit/kafka/consumer/PartitionAssignorTest.scala 1910fcbfb4353fa6adba0ba125c49f1aa8132770
core/src/test/scala/unit/kafka/consumer/TopicFilterTest.scala 4f124af5c3e946045a78ad1519c37372a72c8985
core/src/test/scala/unit/kafka/consumer/ZookeeperConsumerConnectorTest.scala 359b0f5d14f82d14ee423cde271bb35b7034766d
core/src/test/scala/unit/kafka/controller/ControllerFailoverTest.scala PRE-CREATION
core/src/test/scala/unit/kafka/coordinator/ConsumerCoordinatorResponseTest.scala PRE-CREATION
core/src/test/scala/unit/kafka/coordinator/ConsumerGroupMetadataTest.scala b69c993a3fa49f7f01dd28e13ce465c2a89eeba5
core/src/test/scala/unit/kafka/coordinator/CoordinatorMetadataTest.scala 08854c5e6ec249368206298b2ac2623df18f266a
core/src/test/scala/unit/kafka/coordinator/PartitionAssignorTest.scala 887cee5a582b5737ba838920399bb9b24bf22382
core/src/test/scala/unit/kafka/integration/AutoOffsetResetTest.scala 139dc9a104c024e35fd9bc5ac9333e6bd208b571
core/src/test/scala/unit/kafka/integration/FetcherTest.scala facebd8f81c67f26f41a96bce343227bc9b55893
core/src/test/scala/unit/kafka/integration/KafkaServerTestHarness.scala 87c631573aa1e0275d6618c4ac3b33e76fa6abd3
core/src/test/scala/unit/kafka/integration/MinIsrConfigTest.scala 70a5b3aa00a1f647f23ad353b8b1fb1ece020505
core/src/test/scala/unit/kafka/integration/PrimitiveApiTest.scala 6a758a7db71bdc4794b1e5f264f8ca6d410ff2ba
core/src/test/scala/unit/kafka/integration/ProducerConsumerTestHarness.scala 4614a922e739098dbb0ff560d831e26045e32023
core/src/test/scala/unit/kafka/integration/RollingBounceTest.scala 12d0733f5edf436315f884bc193da533d9d4a4ee
core/src/test/scala/unit/kafka/integration/TopicMetadataTest.scala 995b05901491bb0dbf0df210d44bd1d7f66fdc82
core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala e4bf2df48dd59a251b646b7f96d63ec4b924fc0b
core/src/test/scala/unit/kafka/javaapi/consumer/ZookeeperConsumerConnectorTest.scala 74c761dec7afc98667032bdec8359a9aa7c2ecc2
core/src/test/scala/unit/kafka/javaapi/message/BaseMessageSetTestCases.scala 726399e3c7a4157223b5037ff2a03da51236e876
core/src/test/scala/unit/kafka/javaapi/message/ByteBufferMessageSetTest.scala 383fcef02994fde07e669651e522b9e5ee239dd8
core/src/test/scala/unit/kafka/log/BrokerCompressionTest.scala 375555f0684bbd0bfaf64b765ce04a928e257f0a
core/src/test/scala/unit/kafka/log/CleanerTest.scala 8b8249a35322a60ca94cb385a6cad25943dd1cc9
core/src/test/scala/unit/kafka/log/FileMessageSetTest.scala cec1caecc51507ae339ebf8f3b8a028b12a1a056
core/src/test/scala/unit/kafka/log/LogCleanerIntegrationTest.scala 471ddff9bff1bdfa277c071e59e5c6b749b9c74f
core/src/test/scala/unit/kafka/log/LogConfigTest.scala f3546adee490891e0d8d0214bef00b1dd7f42227
core/src/test/scala/unit/kafka/log/LogManagerTest.scala 01dfbc4f8d21f6905327cd4ed6c61d657adc0143
core/src/test/scala/unit/kafka/log/LogSegmentTest.scala 03fb3512c4a4450eac83d4cd4b0919baeaa22942
core/src/test/scala/unit/kafka/log/LogTest.scala 8e095d652851f05365e1d3bbe3e9e1c3345b7a40
core/src/test/scala/unit/kafka/log/OffsetIndexTest.scala 9213a5d9e95d0a5f3009b2cf50baa99765e4cfef
core/src/test/scala/unit/kafka/log/OffsetMapTest.scala 12ce39e665afd4cb1d8aa8d7d4d7df18b219a141
core/src/test/scala/unit/kafka/log4j/KafkaLog4jAppenderTest.scala 41366a14590d318fced0e83d6921d8035fa882da
core/src/test/scala/unit/kafka/message/BaseMessageSetTestCases.scala dd8847f5f709a0a7b2e7037d5beda9a0dfc054d6
core/src/test/scala/unit/kafka/message/ByteBufferMessageSetTest.scala 07bc317bcd40cf40fbff1cbd5039672508f293d9
core/src/test/scala/unit/kafka/message/MessageCompressionTest.scala 76987d4fa68fd39cf0ce6bc47b05eb32b17bbedb
core/src/test/scala/unit/kafka/message/MessageTest.scala 11c0f817ca06e4a53bce1009ad6a36aa007cf93a
core/src/test/scala/unit/kafka/message/MessageWriterTest.scala b08a3432ad1ae49372e4b6cea9b247f6be2889da
core/src/test/scala/unit/kafka/metrics/KafkaTimerTest.scala 7df74050f99567395d4d01c1600ca77cd917652e
core/src/test/scala/unit/kafka/metrics/MetricsTest.scala b42101b85fa4aa6a2249e7b04ec0f61e51c81c3f
core/src/test/scala/unit/kafka/network/SocketServerTest.scala 7dc2fad542ea553ee888543dd215eb41ea57d509
core/src/test/scala/unit/kafka/producer/AsyncProducerTest.scala be4bb878dc49f05caf55b36e9218ed19b1c56253
core/src/test/scala/unit/kafka/producer/ProducerTest.scala 4d2536b462c030f4f269bad024847839e63337c2
core/src/test/scala/unit/kafka/producer/SyncProducerTest.scala 8c3fb7a393db019a29b284b26670f63c25d6a4c6
core/src/test/scala/unit/kafka/server/AdvertiseBrokerTest.scala e899b023b3153542f481b82e92b258e4595b23dc
core/src/test/scala/unit/kafka/server/ClientQuotaManagerTest.scala PRE-CREATION
core/src/test/scala/unit/kafka/server/DelayedOperationTest.scala f3ab3f4ff8eb1aa6b2ab87ba75f72eceb6649620
core/src/test/scala/unit/kafka/server/DynamicConfigChangeTest.scala 7877f6ca1845c2edbf96d4a9783a07a552db8f07
core/src/test/scala/unit/kafka/server/HighwatermarkPersistenceTest.scala 60cd8249e6ec03349e20bb0a7226ea9cd66e6b17
core/src/test/scala/unit/kafka/server/ISRExpirationTest.scala 90529faf11dca65c3ef6bcb27ad72557bc36f939
core/src/test/scala/unit/kafka/server/KafkaConfigConfigDefTest.scala ace6321b36d809946554d205bc926c9c76a43bd6
core/src/test/scala/unit/kafka/server/KafkaConfigTest.scala 2428dbd7197a58cf4cad42ef82b385dab3a2b15e
core/src/test/scala/unit/kafka/server/LeaderElectionTest.scala f1977d850a5bf1125260949101fa3485b29bd4e6
core/src/test/scala/unit/kafka/server/LogOffsetTest.scala e57c1dec2dee4b9a64ffe5906d12676fd877319b
core/src/test/scala/unit/kafka/server/LogRecoveryTest.scala 7688f26fe42c4481fe2da7a1d50459805be0ebc1
core/src/test/scala/unit/kafka/server/OffsetCommitTest.scala 528525b719ec916e16f8b3ae3715bec4b5dcc47d
core/src/test/scala/unit/kafka/server/ReplicaFetchTest.scala a3a03db88c4c3b2949c4ff7f26f2f00498884b18
core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79
core/src/test/scala/unit/kafka/server/ServerGenerateBrokerIdTest.scala 12269cde06aab953dcd898821a8afaa5c4290e77
core/src/test/scala/unit/kafka/server/ServerShutdownTest.scala 95534e36c29023565fedf6bf0fe537ef270c2420
core/src/test/scala/unit/kafka/server/ServerStartupTest.scala 60e10b3d5adda152a01425d98d45ca373a63bebd
core/src/test/scala/unit/kafka/server/SimpleFetchTest.scala 09a0961f73969aca3b054ed3ff6b181a6500364a
core/src/test/scala/unit/kafka/server/ThrottledResponseExpirationTest.scala PRE-CREATION
core/src/test/scala/unit/kafka/utils/ByteBoundedBlockingQueueTest.scala fe8d2aeb642bc075b65ae19d3c85ef4b3f00104a
core/src/test/scala/unit/kafka/utils/CommandLineUtilsTest.scala e832a0179c721546abc725d8b1ee5d78fccdad1f
core/src/test/scala/unit/kafka/utils/IteratorTemplateTest.scala fbd245cad0afab146354fdcc76193d784e1997d9
core/src/test/scala/unit/kafka/utils/JsonTest.scala 93550e8f24071f88eb1ea5b41373efee27e4b8b7
core/src/test/scala/unit/kafka/utils/ReplicationUtilsTest.scala c96c0ffd958d63c09880d436b2e5ae96f51ead36
core/src/test/scala/unit/kafka/utils/SchedulerTest.scala cfea63b88e2590cbd7a1e7cb2c8cacf054bd5568
core/src/test/scala/unit/kafka/utils/TestUtils.scala 17e9fe4c159a29033fe9a287db6ced2fdc3fa9d1
core/src/test/scala/unit/kafka/utils/timer/TimerTaskListTest.scala 052aecdc32ae14afe6aabf942ed7c0028ed9f979
core/src/test/scala/unit/kafka/utils/timer/TimerTest.scala 8507592d1f374248cc7f56ef786544562ca8ad9c
core/src/test/scala/unit/kafka/zk/ZKEphemeralTest.scala 2be161985657293e83a3fd4d3d15faa30e9a119e
core/src/test/scala/unit/kafka/zk/ZKPathTest.scala 64c3ba28614a94f7126aa5ec5fa1f99845f45259
core/src/test/scala/unit/kafka/zk/ZooKeeperTestHarness.scala 1f4d10d25ab3b907689ead7701d2c868ba703952
gradle/buildscript.gradle 5e45c06e8bb8b7c4cb681684023686a7b2252fb7
gradle/license.gradle b4b62ebe277719b28bd131dde9e2a1cdb030e2a9
gradle/rat.gradle PRE-CREATION
gradle/resources/rat-output-to-html.xsl PRE-CREATION
kafka-merge-pr.py PRE-CREATION
kafka-patch-review.py b5a2e950110d44d20d21e3d72be1783ceb49627d
log4j-appender/src/main/java/org/apache/kafka/log4jappender/KafkaLog4jAppender.java PRE-CREATION
log4j-appender/src/test/java/org/apache/kafka/log4jappender/KafkaLog4jAppenderTest.java PRE-CREATION
log4j-appender/src/test/java/org/apache/kafka/log4jappender/MockKafkaLog4jAppender.java PRE-CREATION
scala.gradle cabb59c2a6289bc3a2673ddfa3842addf9a5bb44
settings.gradle 83f764e6a4a15a5fdba232dce74a369870f26b45
system_test/system_test_runner.py 5078d4479fab71722751a28c3c8f5c0f61baadec
tests/.gitignore PRE-CREATION
tests/README.md PRE-CREATION
tests/kafkatest/__init__.py PRE-CREATION
tests/kafkatest/sanity_checks/__init__.py PRE-CREATION
tests/kafkatest/sanity_checks/test_console_consumer.py PRE-CREATION
tests/kafkatest/services/__init__.py PRE-CREATION
tests/kafkatest/services/console_consumer.py PRE-CREATION
tests/kafkatest/services/kafka.py PRE-CREATION
tests/kafkatest/services/performance.py PRE-CREATION
tests/kafkatest/services/templates/console_consumer.properties PRE-CREATION
tests/kafkatest/services/templates/console_consumer_log4j.properties PRE-CREATION
tests/kafkatest/services/templates/kafka.properties PRE-CREATION
tests/kafkatest/services/templates/zookeeper.properties PRE-CREATION
tests/kafkatest/services/verifiable_producer.py PRE-CREATION
tests/kafkatest/services/zookeeper.py PRE-CREATION
tests/kafkatest/tests/__init__.py PRE-CREATION
tests/kafkatest/tests/benchmark_test.py PRE-CREATION
tests/kafkatest/tests/kafka_test.py PRE-CREATION
tests/kafkatest/tests/replication_test.py PRE-CREATION
tests/setup.py PRE-CREATION
tools/src/main/java/org/apache/kafka/clients/tools/ThroughputThrottler.java PRE-CREATION
tools/src/main/java/org/apache/kafka/clients/tools/VerifiableProducer.java PRE-CREATION
topics.json ff011ed381e781b9a177036001d44dca3eac586f
vagrant/aws/aws-access-keys-commands PRE-CREATION
vagrant/aws/aws-example-Vagrantfile.local PRE-CREATION
vagrant/aws/aws-init.sh PRE-CREATION
vagrant/base.sh 6f28dfed67877fd252a9c7cedad1eaa78c648b28
vagrant/broker.sh 63f2d4f30c5a46307d0fe991bbc791a9804b180e
vagrant/system-test-Vagrantfile.local PRE-CREATION
vagrant/zk.sh 15517f826461d8adc2b7b32257cc03d7bfe11bc3
Diff: https://reviews.apache.org/r/33378/diff/
Testing
-------
New tests added
Thanks,
Aditya Auradkar
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Joel Koshy <jj...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review93979
-----------------------------------------------------------
This patch also needs a rebase.
core/src/main/scala/kafka/server/ReplicaManager.scala (line 312)
<https://reviews.apache.org/r/33378/#comment148455>
I'm a bit unclear on how you are planning to put in the right delay value in the response struct. i.e., in KAFKA-2084 you are computing the delay inside the callback. How will that value be accessed here?
- Joel Koshy
On July 13, 2015, 8:36 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated July 13, 2015, 8:36 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel's comments
>
> For now the patch will publish a zero delay and return a response
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 56281ee15cc33dfc96ff64d5b1e596047c7132a4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 07e65d4a54ba4eef5b787eba3e71cbe9f6a920bd
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 8686d83aa52e435c6adafbe9ff4bd1602281072a
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java fabeae3083a8ea55cdacbb9568f3847ccd85bab4
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 37ec0b79beafcf5735c386b066eb319fb697eff5
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java 419541011d652becf0cda7a5e62ce813cddb1732
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java e3cc1967e407b64cc734548c19e30de700b64ba8
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala c16f7edd322709060e54c77eb505c44cbd77a4ec
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala 83fc47417dd7fe4edf030217fa7fd69d99b170b0
> core/src/main/scala/kafka/server/DelayedFetch.scala de6cf5bdaa0e70394162febc63b50b55ca0a92db
> core/src/main/scala/kafka/server/DelayedProduce.scala 05078b24ef28f2f4e099afa943e43f1d00359fda
> core/src/main/scala/kafka/server/KafkaApis.scala d63bc18d795a6f0e6994538ca55a9a46f7fb8ffd
> core/src/main/scala/kafka/server/OffsetManager.scala 5cca85cf727975f6d3acb2223fd186753ad761dc
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala b31b432a226ba79546dd22ef1d2acbb439c2e9a3
> core/src/main/scala/kafka/server/ReplicaManager.scala 59c9bc3ac3a8afc07a6f8c88c5871304db588d17
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 5717165f2344823fabe8f7cfafae4bb8af2d949a
> core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/
-----------------------------------------------------------
(Updated July 13, 2015, 8:36 p.m.)
Review request for kafka, Joel Koshy and Jun Rao.
Bugs: KAFKA-2136
https://issues.apache.org/jira/browse/KAFKA-2136
Repository: kafka
Description (updated)
-------
Changes are
- Addressing Joel's comments
- protocol changes to the fetch request and response to return the throttle_time_ms to clients
- New producer/consumer metrics to expose the avg and max delay time for a client
- Test cases.
- Addressed Joel's comments
For now the patch will publish a zero delay and return a response
Diffs
-----
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 56281ee15cc33dfc96ff64d5b1e596047c7132a4
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 07e65d4a54ba4eef5b787eba3e71cbe9f6a920bd
clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 8686d83aa52e435c6adafbe9ff4bd1602281072a
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java fabeae3083a8ea55cdacbb9568f3847ccd85bab4
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 37ec0b79beafcf5735c386b066eb319fb697eff5
clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java 419541011d652becf0cda7a5e62ce813cddb1732
clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java e3cc1967e407b64cc734548c19e30de700b64ba8
core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
core/src/main/scala/kafka/consumer/SimpleConsumer.scala c16f7edd322709060e54c77eb505c44cbd77a4ec
core/src/main/scala/kafka/server/AbstractFetcherThread.scala 83fc47417dd7fe4edf030217fa7fd69d99b170b0
core/src/main/scala/kafka/server/DelayedFetch.scala de6cf5bdaa0e70394162febc63b50b55ca0a92db
core/src/main/scala/kafka/server/DelayedProduce.scala 05078b24ef28f2f4e099afa943e43f1d00359fda
core/src/main/scala/kafka/server/KafkaApis.scala d63bc18d795a6f0e6994538ca55a9a46f7fb8ffd
core/src/main/scala/kafka/server/OffsetManager.scala 5cca85cf727975f6d3acb2223fd186753ad761dc
core/src/main/scala/kafka/server/ReplicaFetcherThread.scala b31b432a226ba79546dd22ef1d2acbb439c2e9a3
core/src/main/scala/kafka/server/ReplicaManager.scala 59c9bc3ac3a8afc07a6f8c88c5871304db588d17
core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 5717165f2344823fabe8f7cfafae4bb8af2d949a
core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79
Diff: https://reviews.apache.org/r/33378/diff/
Testing
-------
New tests added
Thanks,
Aditya Auradkar
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/
-----------------------------------------------------------
(Updated July 13, 2015, 8:34 p.m.)
Review request for kafka, Joel Koshy and Jun Rao.
Bugs: KAFKA-2136
https://issues.apache.org/jira/browse/KAFKA-2136
Repository: kafka
Description (updated)
-------
Addressing Joel's comments
Merging
Chaning variable name
Addressing Joel's comments
Addressing Joel's comments
Diffs (updated)
-----
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 56281ee15cc33dfc96ff64d5b1e596047c7132a4
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 07e65d4a54ba4eef5b787eba3e71cbe9f6a920bd
clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 8686d83aa52e435c6adafbe9ff4bd1602281072a
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java fabeae3083a8ea55cdacbb9568f3847ccd85bab4
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 37ec0b79beafcf5735c386b066eb319fb697eff5
clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java 419541011d652becf0cda7a5e62ce813cddb1732
clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java e3cc1967e407b64cc734548c19e30de700b64ba8
core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
core/src/main/scala/kafka/consumer/SimpleConsumer.scala c16f7edd322709060e54c77eb505c44cbd77a4ec
core/src/main/scala/kafka/server/AbstractFetcherThread.scala 83fc47417dd7fe4edf030217fa7fd69d99b170b0
core/src/main/scala/kafka/server/DelayedFetch.scala de6cf5bdaa0e70394162febc63b50b55ca0a92db
core/src/main/scala/kafka/server/DelayedProduce.scala 05078b24ef28f2f4e099afa943e43f1d00359fda
core/src/main/scala/kafka/server/KafkaApis.scala d63bc18d795a6f0e6994538ca55a9a46f7fb8ffd
core/src/main/scala/kafka/server/OffsetManager.scala 5cca85cf727975f6d3acb2223fd186753ad761dc
core/src/main/scala/kafka/server/ReplicaFetcherThread.scala b31b432a226ba79546dd22ef1d2acbb439c2e9a3
core/src/main/scala/kafka/server/ReplicaManager.scala 59c9bc3ac3a8afc07a6f8c88c5871304db588d17
core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 5717165f2344823fabe8f7cfafae4bb8af2d949a
core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79
Diff: https://reviews.apache.org/r/33378/diff/
Testing
-------
New tests added
Thanks,
Aditya Auradkar
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
> On July 10, 2015, 5:49 p.m., Joel Koshy wrote:
> > core/src/main/scala/kafka/server/DelayedFetch.scala, line 135
> > <https://reviews.apache.org/r/33378/diff/9/?file=996359#file996359line135>
> >
> > For these, I'm wondering if we should put in the actual delay and in KAFKA-2136 just add a config to enable/disable quotas altogether.
Hey Joel.. can you elaborate? The actual delay isn't being computed in this patch.
- Aditya
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review91319
-----------------------------------------------------------
On July 1, 2015, 2:44 a.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated July 1, 2015, 2:44 a.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel's comments
>
> For now the patch will publish a zero delay and return a response
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 56281ee15cc33dfc96ff64d5b1e596047c7132a4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 07e65d4a54ba4eef5b787eba3e71cbe9f6a920bd
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 8686d83aa52e435c6adafbe9ff4bd1602281072a
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java fabeae3083a8ea55cdacbb9568f3847ccd85bab4
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 37ec0b79beafcf5735c386b066eb319fb697eff5
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java 419541011d652becf0cda7a5e62ce813cddb1732
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java e3cc1967e407b64cc734548c19e30de700b64ba8
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala c16f7edd322709060e54c77eb505c44cbd77a4ec
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala 83fc47417dd7fe4edf030217fa7fd69d99b170b0
> core/src/main/scala/kafka/server/DelayedFetch.scala de6cf5bdaa0e70394162febc63b50b55ca0a92db
> core/src/main/scala/kafka/server/DelayedProduce.scala 05078b24ef28f2f4e099afa943e43f1d00359fda
> core/src/main/scala/kafka/server/KafkaApis.scala d63bc18d795a6f0e6994538ca55a9a46f7fb8ffd
> core/src/main/scala/kafka/server/OffsetManager.scala 5cca85cf727975f6d3acb2223fd186753ad761dc
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala b31b432a226ba79546dd22ef1d2acbb439c2e9a3
> core/src/main/scala/kafka/server/ReplicaManager.scala 59c9bc3ac3a8afc07a6f8c88c5871304db588d17
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 5717165f2344823fabe8f7cfafae4bb8af2d949a
> core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
> On July 10, 2015, 5:49 p.m., Joel Koshy wrote:
> > LGTM - just a few minor comments.
Also, I filed this ticket to add metrics to the old producer and consumers:
https://issues.apache.org/jira/browse/KAFKA-2332
- Aditya
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review91319
-----------------------------------------------------------
On July 13, 2015, 8:36 p.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated July 13, 2015, 8:36 p.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - Addressing Joel's comments
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel's comments
>
> For now the patch will publish a zero delay and return a response
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 56281ee15cc33dfc96ff64d5b1e596047c7132a4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 07e65d4a54ba4eef5b787eba3e71cbe9f6a920bd
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 8686d83aa52e435c6adafbe9ff4bd1602281072a
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java fabeae3083a8ea55cdacbb9568f3847ccd85bab4
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 37ec0b79beafcf5735c386b066eb319fb697eff5
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java 419541011d652becf0cda7a5e62ce813cddb1732
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java e3cc1967e407b64cc734548c19e30de700b64ba8
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala c16f7edd322709060e54c77eb505c44cbd77a4ec
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala 83fc47417dd7fe4edf030217fa7fd69d99b170b0
> core/src/main/scala/kafka/server/DelayedFetch.scala de6cf5bdaa0e70394162febc63b50b55ca0a92db
> core/src/main/scala/kafka/server/DelayedProduce.scala 05078b24ef28f2f4e099afa943e43f1d00359fda
> core/src/main/scala/kafka/server/KafkaApis.scala d63bc18d795a6f0e6994538ca55a9a46f7fb8ffd
> core/src/main/scala/kafka/server/OffsetManager.scala 5cca85cf727975f6d3acb2223fd186753ad761dc
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala b31b432a226ba79546dd22ef1d2acbb439c2e9a3
> core/src/main/scala/kafka/server/ReplicaManager.scala 59c9bc3ac3a8afc07a6f8c88c5871304db588d17
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 5717165f2344823fabe8f7cfafae4bb8af2d949a
> core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Joel Koshy <jj...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/#review91319
-----------------------------------------------------------
LGTM - just a few minor comments.
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java (line 101)
<https://reviews.apache.org/r/33378/#comment144632>
Would prefer seeing this in this style:
```
func() {
statement;
}
```
clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java (line 68)
<https://reviews.apache.org/r/33378/#comment144638>
EPSILON?
core/src/main/scala/kafka/api/FetchResponse.scala (line 169)
<https://reviews.apache.org/r/33378/#comment144643>
A: B
core/src/main/scala/kafka/api/FetchResponse.scala (line 170)
<https://reviews.apache.org/r/33378/#comment144644>
A: B
core/src/main/scala/kafka/api/FetchResponse.scala (line 179)
<https://reviews.apache.org/r/33378/#comment144646>
Prefer placing throttleTimeSize on the next line.
core/src/main/scala/kafka/api/FetchResponse.scala (line 229)
<https://reviews.apache.org/r/33378/#comment144655>
The throttleTimeSize...
(However, see following comment)
core/src/main/scala/kafka/api/FetchResponse.scala (line 230)
<https://reviews.apache.org/r/33378/#comment144653>
Just wondering if we should move all this to `FetchResponse` class (not object).
i.e., add `headerSize` and `writeHeaderTo(buffer)` member methods since the throttle-time is really part of the header. Depending on the version, those two methods can then do the right thing. This code will then just become:
```
private val buffer = ByteBuffer.allocate(4 + fetchResponse.headerSize)
fetchResponse.writeHeaderTo(buffer)
```
core/src/main/scala/kafka/server/DelayedFetch.scala (line 135)
<https://reviews.apache.org/r/33378/#comment144657>
For these, I'm wondering if we should put in the actual delay and in KAFKA-2136 just add a config to enable/disable quotas altogether.
core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala (line 81)
<https://reviews.apache.org/r/33378/#comment144658>
delayTime: Int
- Joel Koshy
On July 1, 2015, 2:44 a.m., Aditya Auradkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33378/
> -----------------------------------------------------------
>
> (Updated July 1, 2015, 2:44 a.m.)
>
>
> Review request for kafka, Joel Koshy and Jun Rao.
>
>
> Bugs: KAFKA-2136
> https://issues.apache.org/jira/browse/KAFKA-2136
>
>
> Repository: kafka
>
>
> Description
> -------
>
> Changes are
> - protocol changes to the fetch request and response to return the throttle_time_ms to clients
> - New producer/consumer metrics to expose the avg and max delay time for a client
> - Test cases.
> - Addressed Joel's comments
>
> For now the patch will publish a zero delay and return a response
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 56281ee15cc33dfc96ff64d5b1e596047c7132a4
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 07e65d4a54ba4eef5b787eba3e71cbe9f6a920bd
> clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
> clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 8686d83aa52e435c6adafbe9ff4bd1602281072a
> clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
> clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java fabeae3083a8ea55cdacbb9568f3847ccd85bab4
> clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 37ec0b79beafcf5735c386b066eb319fb697eff5
> clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java 419541011d652becf0cda7a5e62ce813cddb1732
> clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
> clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java e3cc1967e407b64cc734548c19e30de700b64ba8
> core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
> core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
> core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
> core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
> core/src/main/scala/kafka/consumer/SimpleConsumer.scala c16f7edd322709060e54c77eb505c44cbd77a4ec
> core/src/main/scala/kafka/server/AbstractFetcherThread.scala 83fc47417dd7fe4edf030217fa7fd69d99b170b0
> core/src/main/scala/kafka/server/DelayedFetch.scala de6cf5bdaa0e70394162febc63b50b55ca0a92db
> core/src/main/scala/kafka/server/DelayedProduce.scala 05078b24ef28f2f4e099afa943e43f1d00359fda
> core/src/main/scala/kafka/server/KafkaApis.scala d63bc18d795a6f0e6994538ca55a9a46f7fb8ffd
> core/src/main/scala/kafka/server/OffsetManager.scala 5cca85cf727975f6d3acb2223fd186753ad761dc
> core/src/main/scala/kafka/server/ReplicaFetcherThread.scala b31b432a226ba79546dd22ef1d2acbb439c2e9a3
> core/src/main/scala/kafka/server/ReplicaManager.scala 59c9bc3ac3a8afc07a6f8c88c5871304db588d17
> core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 5717165f2344823fabe8f7cfafae4bb8af2d949a
> core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79
>
> Diff: https://reviews.apache.org/r/33378/diff/
>
>
> Testing
> -------
>
> New tests added
>
>
> Thanks,
>
> Aditya Auradkar
>
>
Re: Review Request 33378: Patch for KAFKA-2136
Posted by Aditya Auradkar <aa...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33378/
-----------------------------------------------------------
(Updated July 1, 2015, 2:44 a.m.)
Review request for kafka, Joel Koshy and Jun Rao.
Bugs: KAFKA-2136
https://issues.apache.org/jira/browse/KAFKA-2136
Repository: kafka
Description (updated)
-------
Changes are
- protocol changes to the fetch request and response to return the throttle_time_ms to clients
- New producer/consumer metrics to expose the avg and max delay time for a client
- Test cases.
- Addressed Joel's comments
For now the patch will publish a zero delay and return a response
Diffs
-----
clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java 56281ee15cc33dfc96ff64d5b1e596047c7132a4
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 07e65d4a54ba4eef5b787eba3e71cbe9f6a920bd
clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java 3dc8b015afd2347a41c9a9dbc02b8e367da5f75f
clients/src/main/java/org/apache/kafka/common/requests/FetchRequest.java 8686d83aa52e435c6adafbe9ff4bd1602281072a
clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java eb8951fba48c335095cc43fc3672de1c733e07ff
clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java fabeae3083a8ea55cdacbb9568f3847ccd85bab4
clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java 37ec0b79beafcf5735c386b066eb319fb697eff5
clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java 419541011d652becf0cda7a5e62ce813cddb1732
clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java 8b1805d3d2bcb9fe2bacb37d870c3236aa9532c4
clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java e3cc1967e407b64cc734548c19e30de700b64ba8
core/src/main/scala/kafka/api/FetchRequest.scala 5b38f8554898e54800abd65a7415dd0ac41fd958
core/src/main/scala/kafka/api/FetchResponse.scala 0b6b33ab6f7a732ff1322b6f48abd4c43e0d7215
core/src/main/scala/kafka/api/ProducerRequest.scala c866180d3680da03e48d374415f10220f6ca68c4
core/src/main/scala/kafka/api/ProducerResponse.scala 5d1fac4cb8943f5bfaa487f8e9d9d2856efbd330
core/src/main/scala/kafka/consumer/SimpleConsumer.scala c16f7edd322709060e54c77eb505c44cbd77a4ec
core/src/main/scala/kafka/server/AbstractFetcherThread.scala 83fc47417dd7fe4edf030217fa7fd69d99b170b0
core/src/main/scala/kafka/server/DelayedFetch.scala de6cf5bdaa0e70394162febc63b50b55ca0a92db
core/src/main/scala/kafka/server/DelayedProduce.scala 05078b24ef28f2f4e099afa943e43f1d00359fda
core/src/main/scala/kafka/server/KafkaApis.scala d63bc18d795a6f0e6994538ca55a9a46f7fb8ffd
core/src/main/scala/kafka/server/OffsetManager.scala 5cca85cf727975f6d3acb2223fd186753ad761dc
core/src/main/scala/kafka/server/ReplicaFetcherThread.scala b31b432a226ba79546dd22ef1d2acbb439c2e9a3
core/src/main/scala/kafka/server/ReplicaManager.scala 59c9bc3ac3a8afc07a6f8c88c5871304db588d17
core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala 5717165f2344823fabe8f7cfafae4bb8af2d949a
core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala 00d59337a99ac135e8689bd1ecd928f7b1423d79
Diff: https://reviews.apache.org/r/33378/diff/
Testing
-------
New tests added
Thanks,
Aditya Auradkar