You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/07/17 16:11:19 UTC
[GitHub] [kafka] jolshan opened a new pull request #11073: KAFKA-13092: Perf regression in LISR requests
jolshan opened a new pull request #11073:
URL: https://github.com/apache/kafka/pull/11073
Cherry pick of 584213ed20d679b11206b67c5a65035347632f07
Main difference is that we do not try to write to partition metadata file when initializing the Log. Since we only do so in checkOrSet topic ID and there is a small bug in that path, the benchmark attempts to run that method last -- so that the log is created and we can actually record the ID to write.
Some quick runs of the benchmarks --
Master
```
Benchmark (numPartitions) (useTopicIds) Mode Cnt Score Error Units
PartitionCreationBench.makeFollower 2000 false avgt 2 2576.278 ms/op
PartitionCreationBench.makeFollower 2000 true avgt 2 5378.075 ms/op
```
PR
```
Benchmark (numPartitions) (useTopicIds) Mode Cnt Score Error Units
PartitionCreationBench.makeFollower 2000 false avgt 2 2678.498 ms/op
PartitionCreationBench.makeFollower 2000 true avgt 2 2962.268 ms/op
```
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dajac commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
dajac commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-901015825
@jolshan is it something that we must add to 2.8.1?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jolshan commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
jolshan commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-906500017
I think you may be able to just restart the build. (I think only committers can do that). If that doesn't work, I'll rebase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dajac commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
dajac commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-901015825
@jolshan is it something that we must add to 2.8.1?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dajac commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
dajac commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-906334811
Do you need to rebase the PR to include it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dajac commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
dajac commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-908364578
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dajac merged pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
dajac merged pull request #11073:
URL: https://github.com/apache/kafka/pull/11073
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jolshan commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
jolshan commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-905114876
fix here: https://github.com/apache/kafka/pull/11257
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jolshan commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
jolshan commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-906611414
Ok will do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jolshan commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
jolshan commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-901220052
Ah yes. I had a feeling I still had a pr open for this. I think I can clean it up a bit and it should be ready for review by tomorrow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dajac commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
dajac commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-906522684
> I think you may be able to just restart the build. (I think only committers can do that). If that doesn't work, I'll rebase.
Done, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dajac commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
dajac commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-908971681
@junrao @lbradstreet Thanks for your reviews.
The failures are not related. I will merge it to 2.8.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jolshan commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
jolshan commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-905024994
Fixed my issue locally but looks like 2.8 branch is broken in general.
```
> Task :streams:compileTestJava
/Users/jolshan/kafka/streams/src/test/java/org/apache/kafka/streams/integration/MetricsReporterIntegrationTest.java:73: error: stop() has private access in EmbeddedKafkaCluster
CLUSTER.stop();
^
1 error
```
Will try to contact relevant person.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jolshan commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
jolshan commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-904805600
Looks like an issue with more recent changes. I'll fix it up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dajac commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
dajac commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-906600948
@jolshan All tests failed, again. I think that you have to rebase to include the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dajac merged pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
dajac merged pull request #11073:
URL: https://github.com/apache/kafka/pull/11073
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dajac commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
dajac commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-904481315
@jolshan All the builds have failed. Could you take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dajac commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
dajac commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-908364578
@lbradstreet @junrao Could you take a look at this one as you both reviewed the original PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jolshan commented on pull request #11073: KAFKA-13092: Perf regression in LISR requests
Posted by GitBox <gi...@apache.org>.
jolshan commented on pull request #11073:
URL: https://github.com/apache/kafka/pull/11073#issuecomment-901220052
Ah yes. I had a feeling I still had a pr open for this. I think I can clean it up a bit and it should be ready for review by tomorrow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org