You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2022/01/18 19:52:44 UTC
[GitHub] [kafka] jonathan-albrecht-ibm opened a new pull request #11690: KAFKA-13599: Upgrade RocksDB to 6.27.3
jonathan-albrecht-ibm opened a new pull request #11690:
URL: https://github.com/apache/kafka/pull/11690
RocksDB v6.27.3 has been released and it is the first release to support s390x. RocksDB is currently the only dependency in gradle/dependencies.gradle without s390x support.
RocksDB v6.27.3 has added some new options that require an update to streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBGenericOptionsToDbOptionsColumnFamilyOptionsAdapter.java but no other changes are needed to upgrade.
I have run the unit/integration tests locally on s390x and also the :streams tests on x86_64 and they pass.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jonathan-albrecht-ibm commented on a change in pull request #11690: KAFKA-13599: Upgrade RocksDB to 6.27.3
Posted by GitBox <gi...@apache.org>.
jonathan-albrecht-ibm commented on a change in pull request #11690:
URL: https://github.com/apache/kafka/pull/11690#discussion_r795859434
##########
File path: streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBGenericOptionsToDbOptionsColumnFamilyOptionsAdapter.java
##########
@@ -1662,16 +1673,95 @@ public ConcurrentTaskLimiter compactionThreadLimiter() {
return columnFamilyOptions.compactionThreadLimiter();
}
+ @Override
public Options setCompactionFilter(final AbstractCompactionFilter<? extends AbstractSlice<?>> compactionFilter) {
columnFamilyOptions.setCompactionFilter(compactionFilter);
return this;
}
+ @Override
public Options setCompactionFilterFactory(final AbstractCompactionFilterFactory<? extends AbstractCompactionFilter<?>> compactionFilterFactory) {
columnFamilyOptions.setCompactionFilterFactory(compactionFilterFactory);
return this;
}
+ @Override
+ public Options setEnableBlobFiles(final boolean enableBlobFiles) {
Review comment:
TFTR @showuon! I've added in the comments, let me know how it looks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] cadonna commented on pull request #11690: KAFKA-13599: Upgrade RocksDB to 6.27.3
Posted by GitBox <gi...@apache.org>.
cadonna commented on pull request #11690:
URL: https://github.com/apache/kafka/pull/11690#issuecomment-1027763171
Failing test are unrelated:
```
Build / JDK 11 and Scala 2.13 / kafka.admin.LeaderElectionCommandTest.[1] Type=Raft, Name=testTopicPartition, Security=PLAINTEXT
Build / JDK 11 and Scala 2.13 / kafka.api.TransactionsExpirationTest.[2] quorum=kraft
Build / JDK 17 and Scala 2.13 / kafka.admin.LeaderElectionCommandTest.[1] Type=Raft, Name=testAllTopicPartition, Security=PLAINTEXT
Build / JDK 17 and Scala 2.13 / kafka.admin.LeaderElectionCommandTest.[1] Type=Raft, Name=testTopicPartition, Security=PLAINTEXT
Build / JDK 17 and Scala 2.13 / kafka.api.TransactionsTest.testAbortTransactionTimeout()
Build / JDK 17 and Scala 2.13 / kafka.server.KRaftClusterTest.testCreateClusterAndCreateListDeleteTopic()
Build / JDK 17 and Scala 2.13 / kafka.network.SocketServerTest.remoteCloseWithIncompleteBufferedReceive()
Build / JDK 8 and Scala 2.12 / kafka.network.SocketServerTest.remoteCloseWithIncompleteBufferedReceive()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] cadonna commented on pull request #11690: KAFKA-13599: Upgrade RocksDB to 6.27.3
Posted by GitBox <gi...@apache.org>.
cadonna commented on pull request #11690:
URL: https://github.com/apache/kafka/pull/11690#issuecomment-1024311788
Sorry, I got sick and need to move the review to next week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] cadonna commented on pull request #11690: KAFKA-13599: Upgrade RocksDB to 6.27.3
Posted by GitBox <gi...@apache.org>.
cadonna commented on pull request #11690:
URL: https://github.com/apache/kafka/pull/11690#issuecomment-1016214618
@jonathan-albrecht-ibm Thank you for the PR! I will review it this or next week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jonathan-albrecht-ibm commented on pull request #11690: KAFKA-13599: Upgrade RocksDB to 6.27.3
Posted by GitBox <gi...@apache.org>.
jonathan-albrecht-ibm commented on pull request #11690:
URL: https://github.com/apache/kafka/pull/11690#issuecomment-1024362058
Thanks @cadonna. Get well soon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on a change in pull request #11690: KAFKA-13599: Upgrade RocksDB to 6.27.3
Posted by GitBox <gi...@apache.org>.
showuon commented on a change in pull request #11690:
URL: https://github.com/apache/kafka/pull/11690#discussion_r795136975
##########
File path: streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBGenericOptionsToDbOptionsColumnFamilyOptionsAdapter.java
##########
@@ -1662,16 +1673,95 @@ public ConcurrentTaskLimiter compactionThreadLimiter() {
return columnFamilyOptions.compactionThreadLimiter();
}
+ @Override
public Options setCompactionFilter(final AbstractCompactionFilter<? extends AbstractSlice<?>> compactionFilter) {
columnFamilyOptions.setCompactionFilter(compactionFilter);
return this;
}
+ @Override
public Options setCompactionFilterFactory(final AbstractCompactionFilterFactory<? extends AbstractCompactionFilter<?>> compactionFilterFactory) {
columnFamilyOptions.setCompactionFilterFactory(compactionFilterFactory);
return this;
}
+ @Override
+ public Options setEnableBlobFiles(final boolean enableBlobFiles) {
Review comment:
Started here, they are options for blobsDB. In the upstream `Options.java`, there are comments at the beginning and the end of the options. Could we add something similar here?
```java
//
// BEGIN options for blobs (integrated BlobDB)
//
```
```java
//
// END options for blobs (integrated BlobDB)
//
```
ref: https://github.com/facebook/rocksdb/blob/main/java/src/main/java/org/rocksdb/Options.java#L1996-L1998
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] cadonna commented on pull request #11690: KAFKA-13599: Upgrade RocksDB to 6.27.3
Posted by GitBox <gi...@apache.org>.
cadonna commented on pull request #11690:
URL: https://github.com/apache/kafka/pull/11690#issuecomment-1026894087
I re-triggered the builds. Once the builds are acceptable, I will merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] cadonna merged pull request #11690: KAFKA-13599: Upgrade RocksDB to 6.27.3
Posted by GitBox <gi...@apache.org>.
cadonna merged pull request #11690:
URL: https://github.com/apache/kafka/pull/11690
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jonathan-albrecht-ibm commented on pull request #11690: KAFKA-13599: Upgrade RocksDB to 6.27.3
Posted by GitBox <gi...@apache.org>.
jonathan-albrecht-ibm commented on pull request #11690:
URL: https://github.com/apache/kafka/pull/11690#issuecomment-1023667798
Hi @cadonna, just wondering if you are still able to review this week or if it will be pushed out to next? Not trying to rush anything, I appreciate the time you can spend on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jonathan-albrecht-ibm commented on pull request #11690: KAFKA-13599: Upgrade RocksDB to 6.27.3
Posted by GitBox <gi...@apache.org>.
jonathan-albrecht-ibm commented on pull request #11690:
URL: https://github.com/apache/kafka/pull/11690#issuecomment-1016498932
Thanks @cadonna!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] jonathan-albrecht-ibm commented on pull request #11690: KAFKA-13599: Upgrade RocksDB to 6.27.3
Posted by GitBox <gi...@apache.org>.
jonathan-albrecht-ibm commented on pull request #11690:
URL: https://github.com/apache/kafka/pull/11690#issuecomment-1027990865
Thanks @cadonna!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org