You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2020/07/06 19:49:36 UTC
[GitHub] [kafka] cadonna opened a new pull request #8987: KAFKA-10221: Backport fix for KAFKA-9603 to 2.5
cadonna opened a new pull request #8987:
URL: https://github.com/apache/kafka/pull/8987
KAFKA-9603 reports that the number of open files keeps increasing
in RocksDB. The reason is that bulk loading is turned on but
never turned off in segmented state stores for standby tasks.
This bug was fixed in 2.6 through PR #8661 by using code that is not present in 2.5.
So cherry-picking was not possible.
This PR backports the fix to 2.5.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] vvcephei commented on pull request #8987: KAFKA-10221: Backport fix for KAFKA-9603 to 2.5
Posted by GitBox <gi...@apache.org>.
vvcephei commented on pull request #8987:
URL: https://github.com/apache/kafka/pull/8987#issuecomment-654951589
Retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] vvcephei commented on a change in pull request #8987: KAFKA-10221: Backport fix for KAFKA-9603 to 2.5
Posted by GitBox <gi...@apache.org>.
vvcephei commented on a change in pull request #8987:
URL: https://github.com/apache/kafka/pull/8987#discussion_r450961919
##########
File path: streams/src/main/java/org/apache/kafka/streams/state/internals/AbstractRocksDBSegmentedBytesStore.java
##########
@@ -251,7 +252,7 @@ void restoreAllInternal(final Collection<KeyValue<byte[], byte[]>> records) {
// This handles the case that state store is moved to a new client and does not
// have the local RocksDB instance for the segment. In this case, toggleDBForBulkLoading
// will only close the database and open it again with bulk loading enabled.
- if (!bulkLoadSegments.contains(segment)) {
+ if (!bulkLoadSegments.contains(segment) && context instanceof ProcessorContextImpl) {
Review comment:
Woah, this is subtle. IIUC, the fix works by asserting that we should only enable bulk loading if the provided context is a ProcessorContextImpl, which is the kind of context that is only provided when adding the store to an active task.
This seems correct to me, and although it's very subtle, it also seems ok as a patch for an older codebase that won't need to be maintained much. But maybe we can have a comment, or an internal method for the check, like
```suggestion
if (!bulkLoadSegments.contains(segment) && isStoreForActiveTask(context)) {
```
so that it'll be more obvious what's going on here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] vvcephei commented on pull request #8987: KAFKA-10221: Backport fix for KAFKA-9603 to 2.5
Posted by GitBox <gi...@apache.org>.
vvcephei commented on pull request #8987:
URL: https://github.com/apache/kafka/pull/8987#issuecomment-654946672
Test this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] vvcephei merged pull request #8987: KAFKA-10221: Backport fix for KAFKA-9603 to 2.5
Posted by GitBox <gi...@apache.org>.
vvcephei merged pull request #8987:
URL: https://github.com/apache/kafka/pull/8987
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] cadonna commented on pull request #8987: KAFKA-10221: Backport fix for KAFKA-9603 to 2.5
Posted by GitBox <gi...@apache.org>.
cadonna commented on pull request #8987:
URL: https://github.com/apache/kafka/pull/8987#issuecomment-654433640
Call for review: @vvcephei @ableegoldman @guozhangwang @mjsax @abbccdda
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] cadonna commented on a change in pull request #8987: KAFKA-10221: Backport fix for KAFKA-9603 to 2.5
Posted by GitBox <gi...@apache.org>.
cadonna commented on a change in pull request #8987:
URL: https://github.com/apache/kafka/pull/8987#discussion_r450989016
##########
File path: streams/src/main/java/org/apache/kafka/streams/state/internals/AbstractRocksDBSegmentedBytesStore.java
##########
@@ -251,7 +252,7 @@ void restoreAllInternal(final Collection<KeyValue<byte[], byte[]>> records) {
// This handles the case that state store is moved to a new client and does not
// have the local RocksDB instance for the segment. In this case, toggleDBForBulkLoading
// will only close the database and open it again with bulk loading enabled.
- if (!bulkLoadSegments.contains(segment)) {
+ if (!bulkLoadSegments.contains(segment) && context instanceof ProcessorContextImpl) {
Review comment:
Good point! Will do!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] vvcephei commented on pull request #8987: KAFKA-10221: Backport fix for KAFKA-9603 to 2.5
Posted by GitBox <gi...@apache.org>.
vvcephei commented on pull request #8987:
URL: https://github.com/apache/kafka/pull/8987#issuecomment-655163245
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] vvcephei commented on pull request #8987: KAFKA-10221: Backport fix for KAFKA-9603 to 2.5
Posted by GitBox <gi...@apache.org>.
vvcephei commented on pull request #8987:
URL: https://github.com/apache/kafka/pull/8987#issuecomment-655163349
Retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] vvcephei commented on pull request #8987: KAFKA-10221: Backport fix for KAFKA-9603 to 2.5
Posted by GitBox <gi...@apache.org>.
vvcephei commented on pull request #8987:
URL: https://github.com/apache/kafka/pull/8987#issuecomment-655249937
The test failure was unrelated.
kafka.api.ConsumerBounceTest.testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup
Thanks for the backport, @cadonna ! I'll go ahead and merge now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org