You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Dong Lin <li...@gmail.com> on 2014/05/29 23:46:23 UTC
Review Request 22063: Patch for KAFKA-1472
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22063/
-----------------------------------------------------------
Review request for kafka.
Bugs: KAFKA-1472
https://issues.apache.org/jira/browse/KAFKA-1472
Repository: kafka
Description
-------
KAFKA-1472 Add the compression ratio metrics
Diffs
-----
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 3e83ae0fefb7578cccbd8dc0cd9b28f22e0951b2
clients/src/main/java/org/apache/kafka/common/record/Compressor.java 0fa6dd2d6aad3dbe33c9c05406931caae4d8ecf5
clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 428968cd38a7b12991f87868bf759926ff7e594e
Diff: https://reviews.apache.org/r/22063/diff/
Testing
-------
Thanks,
Dong Lin
Re: Review Request 22063: Patch for KAFKA-1472
Posted by Dong Lin <li...@gmail.com>.
> On May 30, 2014, 6:25 a.m., Guozhang Wang wrote:
> > clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java, line 130
> > <https://reviews.apache.org/r/22063/diff/1/?file=599342#file599342line130>
> >
> > When compression is not used, should we return the default value as 1.0?
On a second thought, I agree it is better to use 1.0 as default value for all "invalid" cases.
> On May 30, 2014, 6:25 a.m., Guozhang Wang wrote:
> > clients/src/main/java/org/apache/kafka/common/record/Compressor.java, line 87
> > <https://reviews.apache.org/r/22063/diff/1/?file=599341#file599341line87>
> >
> > When writtenUncompressed is 0, shall we treat the compression rate as 1.0 or -1?
On a second thought, I agree it is better to use 1.0 as default value for all "invalid" cases.
> On May 30, 2014, 6:25 a.m., Guozhang Wang wrote:
> > clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java, line 815
> > <https://reviews.apache.org/r/22063/diff/1/?file=599340#file599340line815>
> >
> > Should this be a Rate() or it should be an Avg()?
oops. My bad, I should use Avg().
- Dong
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22063/#review44334
-----------------------------------------------------------
On June 4, 2014, 5:31 p.m., Dong Lin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22063/
> -----------------------------------------------------------
>
> (Updated June 4, 2014, 5:31 p.m.)
>
>
> Review request for kafka.
>
>
> Bugs: KAFKA-1472
> https://issues.apache.org/jira/browse/KAFKA-1472
>
>
> Repository: kafka
>
>
> Description
> -------
>
> version 2
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 9b1f5653548ba90defdae43940a5554066770b0a
> clients/src/main/java/org/apache/kafka/common/record/Compressor.java 0fa6dd2d6aad3dbe33c9c05406931caae4d8ecf5
> clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 428968cd38a7b12991f87868bf759926ff7e594e
>
> Diff: https://reviews.apache.org/r/22063/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dong Lin
>
>
Re: Review Request 22063: Patch for KAFKA-1472
Posted by Guozhang Wang <gu...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22063/#review44334
-----------------------------------------------------------
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java
<https://reviews.apache.org/r/22063/#comment78704>
Should this be a Rate() or it should be an Avg()?
clients/src/main/java/org/apache/kafka/common/record/Compressor.java
<https://reviews.apache.org/r/22063/#comment78705>
When writtenUncompressed is 0, shall we treat the compression rate as 1.0 or -1?
clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java
<https://reviews.apache.org/r/22063/#comment78706>
When compression is not used, should we return the default value as 1.0?
- Guozhang Wang
On May 29, 2014, 9:46 p.m., Dong Lin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22063/
> -----------------------------------------------------------
>
> (Updated May 29, 2014, 9:46 p.m.)
>
>
> Review request for kafka.
>
>
> Bugs: KAFKA-1472
> https://issues.apache.org/jira/browse/KAFKA-1472
>
>
> Repository: kafka
>
>
> Description
> -------
>
> KAFKA-1472 Add the compression ratio metrics
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 3e83ae0fefb7578cccbd8dc0cd9b28f22e0951b2
> clients/src/main/java/org/apache/kafka/common/record/Compressor.java 0fa6dd2d6aad3dbe33c9c05406931caae4d8ecf5
> clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 428968cd38a7b12991f87868bf759926ff7e594e
>
> Diff: https://reviews.apache.org/r/22063/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dong Lin
>
>
Re: Review Request 22063: Patch for KAFKA-1472
Posted by Guozhang Wang <gu...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22063/#review44913
-----------------------------------------------------------
Ship it!
Ship It!
- Guozhang Wang
On June 6, 2014, 5:42 a.m., Dong Lin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22063/
> -----------------------------------------------------------
>
> (Updated June 6, 2014, 5:42 a.m.)
>
>
> Review request for kafka.
>
>
> Bugs: KAFKA-1472
> https://issues.apache.org/jira/browse/KAFKA-1472
>
>
> Repository: kafka
>
>
> Description
> -------
>
> global compression rate
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 9b1f5653548ba90defdae43940a5554066770b0a
> clients/src/main/java/org/apache/kafka/common/record/Compressor.java 0fa6dd2d6aad3dbe33c9c05406931caae4d8ecf5
> clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 428968cd38a7b12991f87868bf759926ff7e594e
>
> Diff: https://reviews.apache.org/r/22063/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dong Lin
>
>
Re: Review Request 22063: Patch for KAFKA-1472
Posted by Dong Lin <li...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22063/
-----------------------------------------------------------
(Updated June 6, 2014, 5:42 a.m.)
Review request for kafka.
Bugs: KAFKA-1472
https://issues.apache.org/jira/browse/KAFKA-1472
Repository: kafka
Description (updated)
-------
global compression rate
Diffs (updated)
-----
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 9b1f5653548ba90defdae43940a5554066770b0a
clients/src/main/java/org/apache/kafka/common/record/Compressor.java 0fa6dd2d6aad3dbe33c9c05406931caae4d8ecf5
clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 428968cd38a7b12991f87868bf759926ff7e594e
Diff: https://reviews.apache.org/r/22063/diff/
Testing
-------
Thanks,
Dong Lin
Re: Review Request 22063: Patch for KAFKA-1472
Posted by Dong Lin <li...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22063/
-----------------------------------------------------------
(Updated June 5, 2014, 5:14 p.m.)
Review request for kafka.
Bugs: KAFKA-1472
https://issues.apache.org/jira/browse/KAFKA-1472
Repository: kafka
Description (updated)
-------
Provide per-topic and global compression rate averaged across closed batches. In case of unwritable and unused MemoryRcords, use 1.0 as default compression rate.
Diffs
-----
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 9b1f5653548ba90defdae43940a5554066770b0a
clients/src/main/java/org/apache/kafka/common/record/Compressor.java 0fa6dd2d6aad3dbe33c9c05406931caae4d8ecf5
clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 428968cd38a7b12991f87868bf759926ff7e594e
Diff: https://reviews.apache.org/r/22063/diff/
Testing
-------
Thanks,
Dong Lin
Re: Review Request 22063: Patch for KAFKA-1472
Posted by Dong Lin <li...@gmail.com>.
> On June 5, 2014, 4:19 p.m., Jun Rao wrote:
> > clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java, lines 858-862
> > <https://reviews.apache.org/r/22063/diff/2/?file=603554#file603554line858>
> >
> > Could we add a global compression-rate metric too?
> >
Sure! I will add this metric too.
- Dong
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22063/#review44818
-----------------------------------------------------------
On June 4, 2014, 5:31 p.m., Dong Lin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22063/
> -----------------------------------------------------------
>
> (Updated June 4, 2014, 5:31 p.m.)
>
>
> Review request for kafka.
>
>
> Bugs: KAFKA-1472
> https://issues.apache.org/jira/browse/KAFKA-1472
>
>
> Repository: kafka
>
>
> Description
> -------
>
> version 2
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 9b1f5653548ba90defdae43940a5554066770b0a
> clients/src/main/java/org/apache/kafka/common/record/Compressor.java 0fa6dd2d6aad3dbe33c9c05406931caae4d8ecf5
> clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 428968cd38a7b12991f87868bf759926ff7e594e
>
> Diff: https://reviews.apache.org/r/22063/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dong Lin
>
>
Re: Review Request 22063: Patch for KAFKA-1472
Posted by Jun Rao <ju...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22063/#review44818
-----------------------------------------------------------
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java
<https://reviews.apache.org/r/22063/#comment79363>
Could we add a global compression-rate metric too?
- Jun Rao
On June 4, 2014, 5:31 p.m., Dong Lin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22063/
> -----------------------------------------------------------
>
> (Updated June 4, 2014, 5:31 p.m.)
>
>
> Review request for kafka.
>
>
> Bugs: KAFKA-1472
> https://issues.apache.org/jira/browse/KAFKA-1472
>
>
> Repository: kafka
>
>
> Description
> -------
>
> version 2
>
>
> Diffs
> -----
>
> clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 9b1f5653548ba90defdae43940a5554066770b0a
> clients/src/main/java/org/apache/kafka/common/record/Compressor.java 0fa6dd2d6aad3dbe33c9c05406931caae4d8ecf5
> clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 428968cd38a7b12991f87868bf759926ff7e594e
>
> Diff: https://reviews.apache.org/r/22063/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dong Lin
>
>
Re: Review Request 22063: Patch for KAFKA-1472
Posted by Dong Lin <li...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22063/
-----------------------------------------------------------
(Updated June 4, 2014, 5:31 p.m.)
Review request for kafka.
Bugs: KAFKA-1472
https://issues.apache.org/jira/browse/KAFKA-1472
Repository: kafka
Description (updated)
-------
version 2
Diffs (updated)
-----
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java 9b1f5653548ba90defdae43940a5554066770b0a
clients/src/main/java/org/apache/kafka/common/record/Compressor.java 0fa6dd2d6aad3dbe33c9c05406931caae4d8ecf5
clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java 428968cd38a7b12991f87868bf759926ff7e594e
Diff: https://reviews.apache.org/r/22063/diff/
Testing
-------
Thanks,
Dong Lin