You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "divijvaidya (via GitHub)" <gi...@apache.org> on 2023/02/25 16:17:57 UTC
[GitHub] [kafka] divijvaidya opened a new pull request, #13307: MINOR: Make ByteUtilsBenchmark deterministic
divijvaidya opened a new pull request, #13307:
URL: https://github.com/apache/kafka/pull/13307
# Motivation
The current implementation of the benchmark may produce different set of integers across benchmarks and hence, may not provide apples to apples comparison.
# Changes
1. With this change, we initialize random number generator with a fixed seed at the start of a benchmark. This ensures that for each benchmark, random number generator produces the same sequence of random values. Hence, the input data across benchmarks would be consistent providing a reliable apples to apples comparison.
2. We ensure that a new set of random numbers are generated per iteration, so that the bechmark calculation is performed over a diverse range of values
Sample result of a benchmark run:
```
Benchmark Mode Cnt Score Error Units
ByteUtilsBenchmark.testSizeOfUnsignedVarint thrpt 30 1302.193 ± 0.396 ops/s
ByteUtilsBenchmark.testSizeOfUnsignedVarintSimple thrpt 30 328.678 ± 0.269 ops/s
ByteUtilsBenchmark.testSizeOfVarlong thrpt 30 880.113 ± 0.676 ops/s
ByteUtilsBenchmark.testSizeOfVarlongSimple thrpt 30 109.592 ± 0.071 ops/s
JMH benchmarks done
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya closed pull request #13307: MINOR: Make ByteUtilsBenchmark deterministic
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya closed pull request #13307: MINOR: Make ByteUtilsBenchmark deterministic
URL: https://github.com/apache/kafka/pull/13307
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on pull request #13307: MINOR: Make ByteUtilsBenchmark deterministic
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on PR #13307:
URL: https://github.com/apache/kafka/pull/13307#issuecomment-1445372732
> I didn't understand why we need this change. Could you elaborate more on it? From what I can see, it should make no difference if we use the same values or not for each iteration.
Having different values for each iteration gives us ability to benchmark over diverse sample set. This increases the probability that the algorithm we are testing is optimal over a larger set of values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] divijvaidya commented on pull request #13307: MINOR: Make ByteUtilsBenchmark deterministic
Posted by "divijvaidya (via GitHub)" <gi...@apache.org>.
divijvaidya commented on PR #13307:
URL: https://github.com/apache/kafka/pull/13307#issuecomment-1447001588
@showuon I am discarding this PR as I have included it's changes in https://github.com/apache/kafka/pull/13312
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org