You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/05/17 23:29:12 UTC
[jira] [Commented] (KAFKA-3554) Generate actual data with specific
compression ratio and add multi-thread support in the ProducerPerformance
tool.
[ https://issues.apache.org/jira/browse/KAFKA-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15287878#comment-15287878 ]
ASF GitHub Bot commented on KAFKA-3554:
---------------------------------------
GitHub user becketqin opened a pull request:
https://github.com/apache/kafka/pull/1399
KAFKA-3554 Improve ProducerPerformance test
1. Added multiple thread support.
2. Added value-bound to make compressed data more realistic.
3. Print out the producer metrics.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/becketqin/kafka KAFKA-3554
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/1399.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1399
----
commit c6a96a95673e89a60a19959100982ab151ffb73c
Author: Jiangjie Qin <be...@gmail.com>
Date: 2016-05-17T23:10:57Z
KAFKA-3554 ProducerPerformance test improvements.
commit 71fd4c8e92d3d9e695c8c0fcfab838de61f4ffc4
Author: Jiangjie Qin <be...@gmail.com>
Date: 2016-05-17T23:12:40Z
remove change in the server property file
----
> Generate actual data with specific compression ratio and add multi-thread support in the ProducerPerformance tool.
> ------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-3554
> URL: https://issues.apache.org/jira/browse/KAFKA-3554
> Project: Kafka
> Issue Type: Improvement
> Affects Versions: 0.9.0.1
> Reporter: Jiangjie Qin
> Assignee: Jiangjie Qin
> Fix For: 0.10.1.0
>
>
> Currently the ProducerPerformance always generate the payload with same bytes. This does not quite well to test the compressed data because the payload is extremely compressible no matter how big the payload is.
> We can make some changes to make it more useful for compressed messages. Currently I am generating the payload containing integer from a given range. By adjusting the range of the integers, we can get different compression ratios.
> API wise, we can either let user to specify the integer range or the expected compression ratio (we will do some probing to get the corresponding range for the users)
> Besides that, in many cases, it is useful to have multiple producer threads when the producer threads themselves are bottleneck. Admittedly people can run multiple ProducerPerformance to achieve similar result, but it is still different from the real case when people actually use the producer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)