You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by GitBox <gi...@apache.org> on 2022/09/01 14:51:31 UTC
[GitHub] [jackrabbit-oak] gnaresh19 opened a new pull request, #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
gnaresh19 opened a new pull request, #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684
- Provides support for other compression algorithms (zstd, zlib) besides default collection compressor (snappy)
- These compressors are only valid for mongodb v4.2 and above.
- These compressors are applicable to collections only
- Currently the document compressors should be applied per collection, as there is no ability to apply the compressors for all collections at once.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] rishabhdaim commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
rishabhdaim commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r965689623
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDocumentStore.java:
##########
@@ -443,6 +454,19 @@ private void ensureIndexes(@NotNull MongoStatus mongoStatus) {
createIndex(journal, JournalEntry.MODIFIED, true, false, false);
}
+ private void createCollection(MongoDatabase db, String collectionName, MongoStatus mongoStatus) {
+ CreateCollectionOptions options = new CreateCollectionOptions();
+
+ if (mongoStatus.isVersion(4, 2)) {
+ options.storageEngineOptions(MongoDBConfig.getCollectionStorageOptions(mongoStorageOptions));
+ if (!db.listCollectionNames()
Review Comment:
```suggestion
if (!Iterables.tryFind(localDb.listCollectionNames(), s -> Objects.equals(collectionName, s)).isPresent())
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] gnaresh19 commented on pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
gnaresh19 commented on PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#issuecomment-1240962943
Thank you @rishabhdaim for the review. I addressed most of the comments as requested, just had questions on couple of them. can you pls confirm?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] rishabhdaim commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
rishabhdaim commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r965790708
##########
oak-store-document/src/test/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDBConfigTest.java:
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jackrabbit.oak.plugins.document.mongo;
+
+import com.mongodb.BasicDBObject;
+import com.mongodb.MongoClient;
+import org.apache.jackrabbit.oak.plugins.document.mongo.MongoDBConfig.CollectionCompressor;
+import org.bson.BsonDocument;
+import org.bson.conversions.Bson;
+import org.junit.Test;
+
+import java.util.Collections;
+import static org.junit.Assert.assertTrue;
+
+
+public class MongoDBConfigTest {
+
+ private MongoDBConfig mongoDBConfig;
+
+ @Test
+ public void defaultCollectionStorageOptions() {
+ Bson bson = mongoDBConfig.getCollectionStorageOptions(Collections.emptyMap());
+ BsonDocument bsonDocument = bson.toBsonDocument(BasicDBObject.class, MongoClient.getDefaultCodecRegistry());
+ String configuredCompressor = bsonDocument.getDocument(MongoDBConfig.STORAGE_ENGINE).getString(MongoDBConfig.STORAGE_CONFIG).getValue();
+ assertTrue(configuredCompressor.indexOf(CollectionCompressor.SNAPPY.getCompressionType()) > 0);
Review Comment:
I would use `Assert.Equals(string, string)` here.
##########
oak-store-document/src/test/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDBConfigTest.java:
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jackrabbit.oak.plugins.document.mongo;
+
+import com.mongodb.BasicDBObject;
+import com.mongodb.MongoClient;
+import org.apache.jackrabbit.oak.plugins.document.mongo.MongoDBConfig.CollectionCompressor;
+import org.bson.BsonDocument;
+import org.bson.conversions.Bson;
+import org.junit.Test;
+
+import java.util.Collections;
+import static org.junit.Assert.assertTrue;
+
+
+public class MongoDBConfigTest {
+
+ private MongoDBConfig mongoDBConfig;
+
+ @Test
+ public void defaultCollectionStorageOptions() {
+ Bson bson = mongoDBConfig.getCollectionStorageOptions(Collections.emptyMap());
+ BsonDocument bsonDocument = bson.toBsonDocument(BasicDBObject.class, MongoClient.getDefaultCodecRegistry());
+ String configuredCompressor = bsonDocument.getDocument(MongoDBConfig.STORAGE_ENGINE).getString(MongoDBConfig.STORAGE_CONFIG).getValue();
+ assertTrue(configuredCompressor.indexOf(CollectionCompressor.SNAPPY.getCompressionType()) > 0);
+
+ }
+
+ @Test (expected = IllegalArgumentException.class)
+ public void invalidCollectionStorageOptions() throws Exception {
+ mongoDBConfig.getCollectionStorageOptions(Collections.singletonMap(MongoDBConfig.COLLECTION_COMPRESSION_TYPE, "Invalid"));
+ }
+
+ @Test
+ public void overrideDefaultCollectionStorageOptions() {
+ Bson bson = mongoDBConfig.getCollectionStorageOptions(Collections.singletonMap(MongoDBConfig.COLLECTION_COMPRESSION_TYPE, "zstd"));
+ BsonDocument bsonDocument = bson.toBsonDocument(BasicDBObject.class, MongoClient.getDefaultCodecRegistry());
+ String configuredCompressor = bsonDocument.getDocument(MongoDBConfig.STORAGE_ENGINE).getString(MongoDBConfig.STORAGE_CONFIG).getValue();
+
+ assertTrue(configuredCompressor.indexOf(CollectionCompressor.ZSTD.getCompressionType()) > 0);
Review Comment:
I would use `Assert.Equals(string, string)` here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] rishabhdaim commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
rishabhdaim commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r965684140
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDBConfig.java:
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jackrabbit.oak.plugins.document.mongo;
+
+import com.google.common.collect.ImmutableMap;
+import org.apache.jackrabbit.oak.commons.json.JsonObject;
+import org.bson.BsonDocument;
+import org.bson.conversions.Bson;
+
+import java.util.Map;
+
+public class MongoDBConfig {
+ public static final String COLLECTION_COMPRESSION_TYPE = "collectionCompressionType";
+ public static final String STORAGE_ENGINE = "wiredTiger";
+ public static final String STORAGE_CONFIG = "configString";
+
+ enum CollectionCompressor {
+ SNAPPY("snappy"), ZLIB("zlib"), ZSTD("zstd");
+
+ private final String compressionType;
+
+ private static final Map<String, CollectionCompressor> supportedCompressionTypes = ImmutableMap.of(
+ "snappy", CollectionCompressor.SNAPPY, "zlib",
+ CollectionCompressor.ZLIB, "zstd", CollectionCompressor.ZSTD);
+
+ CollectionCompressor(String compressionType) {
+ this.compressionType = compressionType;
+ }
+
+ public String getCompressionType() {
Review Comment:
the method should be renamed to `getName()` because we want to know the name of this compressionType.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] rishabhdaim commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
rishabhdaim commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r965682682
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDBConfig.java:
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jackrabbit.oak.plugins.document.mongo;
+
+import com.google.common.collect.ImmutableMap;
+import org.apache.jackrabbit.oak.commons.json.JsonObject;
+import org.bson.BsonDocument;
+import org.bson.conversions.Bson;
+
+import java.util.Map;
+
+public class MongoDBConfig {
+ public static final String COLLECTION_COMPRESSION_TYPE = "collectionCompressionType";
+ public static final String STORAGE_ENGINE = "wiredTiger";
+ public static final String STORAGE_CONFIG = "configString";
+
+ enum CollectionCompressor {
+ SNAPPY("snappy"), ZLIB("zlib"), ZSTD("zstd");
+
+ private final String compressionType;
+
+ private static final Map<String, CollectionCompressor> supportedCompressionTypes = ImmutableMap.of(
Review Comment:
```suggestion
private static final Map<String, CollectionCompressor> COLLECTION_COMPRESSOR_MAP;
static {
ImmutableMap.Builder<String, CollectionCompressor> builder = new ImmutableMap.Builder<>();
for (CollectionCompressor value : CollectionCompressor.values()) {
builder.put(value.getName().toLowerCase(), value);
}
COLLECTION_COMPRESSOR_MAP = builder.build();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] rishabhdaim commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
rishabhdaim commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r965776008
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreService.java:
##########
@@ -81,7 +81,6 @@
import org.apache.jackrabbit.oak.plugins.blob.datastore.SharedDataStoreUtils;
import org.apache.jackrabbit.oak.plugins.document.persistentCache.PersistentCacheStats;
import org.apache.jackrabbit.oak.plugins.document.util.MongoConnection;
-import org.apache.jackrabbit.oak.plugins.document.util.SystemPropertySupplier;
Review Comment:
This is an un-intended change, please revert this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] rishabhdaim commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
rishabhdaim commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r965689623
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDocumentStore.java:
##########
@@ -443,6 +454,19 @@ private void ensureIndexes(@NotNull MongoStatus mongoStatus) {
createIndex(journal, JournalEntry.MODIFIED, true, false, false);
}
+ private void createCollection(MongoDatabase db, String collectionName, MongoStatus mongoStatus) {
+ CreateCollectionOptions options = new CreateCollectionOptions();
+
+ if (mongoStatus.isVersion(4, 2)) {
+ options.storageEngineOptions(MongoDBConfig.getCollectionStorageOptions(mongoStorageOptions));
+ if (!db.listCollectionNames()
Review Comment:
```suggestion
if (Iterables.tryFind(localDb.listCollectionNames(), s -> Objects.equals(collectionName, s)).isPresent())
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] mreutegg commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
mreutegg commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r963768603
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDocumentStore.java:
##########
@@ -383,16 +393,17 @@ private MongoDBConnection getOrCreateClusterNodesConnection(@NotNull MongoDocume
return mc;
}
- private void ensureIndexes(@NotNull MongoStatus mongoStatus) {
+ private void ensureIndexes(MongoDatabase db, @NotNull MongoStatus mongoStatus) {
Review Comment:
Please add `NotNull` annotation to `db` parameter.
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDocumentStore.java:
##########
@@ -383,16 +393,17 @@ private MongoDBConnection getOrCreateClusterNodesConnection(@NotNull MongoDocume
return mc;
}
- private void ensureIndexes(@NotNull MongoStatus mongoStatus) {
+ private void ensureIndexes(MongoDatabase db, @NotNull MongoStatus mongoStatus) {
// reading documents in the nodes collection and checking
// existing indexes is performed against the MongoDB primary
// this ensures the information is up-to-date and accurate
boolean emptyNodesCollection = execute(session -> MongoUtils.isCollectionEmpty(nodes, session), Collection.NODES);
-
+ createCollection(db, Collection.NODES.toString(), mongoStatus);
Review Comment:
What happens when the collection exists and is empty? Will this method still succeed?
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDocumentStore.java:
##########
@@ -242,7 +244,7 @@ enum DocumentReadPreference {
* How many times should be the bulk update request retries in case of
* a conflict.
* <p>
- * Default is 0 (no retries).
+ * Default is 0 (no retries).≠≠
Review Comment:
This looks unintentional.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] gnaresh19 commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
gnaresh19 commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r966186291
##########
oak-store-document/src/test/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDBConfigTest.java:
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jackrabbit.oak.plugins.document.mongo;
+
+import com.mongodb.BasicDBObject;
+import com.mongodb.MongoClient;
+import org.apache.jackrabbit.oak.plugins.document.mongo.MongoDBConfig.CollectionCompressor;
+import org.bson.BsonDocument;
+import org.bson.conversions.Bson;
+import org.junit.Test;
+
+import java.util.Collections;
+import static org.junit.Assert.assertTrue;
+
+
+public class MongoDBConfigTest {
+
+ private MongoDBConfig mongoDBConfig;
+
+ @Test
+ public void defaultCollectionStorageOptions() {
+ Bson bson = mongoDBConfig.getCollectionStorageOptions(Collections.emptyMap());
+ BsonDocument bsonDocument = bson.toBsonDocument(BasicDBObject.class, MongoClient.getDefaultCodecRegistry());
+ String configuredCompressor = bsonDocument.getDocument(MongoDBConfig.STORAGE_ENGINE).getString(MongoDBConfig.STORAGE_CONFIG).getValue();
+ assertTrue(configuredCompressor.indexOf(CollectionCompressor.SNAPPY.getCompressionType()) > 0);
+
+ }
+
+ @Test (expected = IllegalArgumentException.class)
+ public void invalidCollectionStorageOptions() throws Exception {
+ mongoDBConfig.getCollectionStorageOptions(Collections.singletonMap(MongoDBConfig.COLLECTION_COMPRESSION_TYPE, "Invalid"));
+ }
+
+ @Test
+ public void overrideDefaultCollectionStorageOptions() {
+ Bson bson = mongoDBConfig.getCollectionStorageOptions(Collections.singletonMap(MongoDBConfig.COLLECTION_COMPRESSION_TYPE, "zstd"));
+ BsonDocument bsonDocument = bson.toBsonDocument(BasicDBObject.class, MongoClient.getDefaultCodecRegistry());
+ String configuredCompressor = bsonDocument.getDocument(MongoDBConfig.STORAGE_ENGINE).getString(MongoDBConfig.STORAGE_CONFIG).getValue();
+
+ assertTrue(configuredCompressor.indexOf(CollectionCompressor.ZSTD.getCompressionType()) > 0);
Review Comment:
same as above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] mreutegg commented on pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
mreutegg commented on PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#issuecomment-1238206000
@stefan-egli and @Joscorbe, could you have a look at this PR when you have some time?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] rishabhdaim commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
rishabhdaim commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r966631406
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreService.java:
##########
@@ -81,7 +81,6 @@
import org.apache.jackrabbit.oak.plugins.blob.datastore.SharedDataStoreUtils;
import org.apache.jackrabbit.oak.plugins.document.persistentCache.PersistentCacheStats;
import org.apache.jackrabbit.oak.plugins.document.util.MongoConnection;
-import org.apache.jackrabbit.oak.plugins.document.util.SystemPropertySupplier;
Review Comment:
I think you forgot to address this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] rishabhdaim commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
rishabhdaim commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r965684512
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDBConfig.java:
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jackrabbit.oak.plugins.document.mongo;
+
+import com.google.common.collect.ImmutableMap;
+import org.apache.jackrabbit.oak.commons.json.JsonObject;
+import org.bson.BsonDocument;
+import org.bson.conversions.Bson;
+
+import java.util.Map;
+
+public class MongoDBConfig {
+ public static final String COLLECTION_COMPRESSION_TYPE = "collectionCompressionType";
+ public static final String STORAGE_ENGINE = "wiredTiger";
+ public static final String STORAGE_CONFIG = "configString";
+
+ enum CollectionCompressor {
+ SNAPPY("snappy"), ZLIB("zlib"), ZSTD("zstd");
+
+ private final String compressionType;
+
+ private static final Map<String, CollectionCompressor> supportedCompressionTypes = ImmutableMap.of(
+ "snappy", CollectionCompressor.SNAPPY, "zlib",
+ CollectionCompressor.ZLIB, "zstd", CollectionCompressor.ZSTD);
+
+ CollectionCompressor(String compressionType) {
+ this.compressionType = compressionType;
+ }
+
+ public String getCompressionType() {
+ return compressionType;
+ }
+
+ public static boolean isSupportedCompressor(String compressionType) {
+ return supportedCompressionTypes.containsKey(compressionType);
Review Comment:
```suggestion
return COLLECTION_COMPRESSOR_MAP.containsKey(compressionType.toLowerCase());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] gnaresh19 commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
gnaresh19 commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r967201197
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStoreService.java:
##########
@@ -81,7 +81,6 @@
import org.apache.jackrabbit.oak.plugins.blob.datastore.SharedDataStoreUtils;
import org.apache.jackrabbit.oak.plugins.document.persistentCache.PersistentCacheStats;
import org.apache.jackrabbit.oak.plugins.document.util.MongoConnection;
-import org.apache.jackrabbit.oak.plugins.document.util.SystemPropertySupplier;
Review Comment:
Yeah, this was ununsed import, and since I made some other changes in this file, the unused import was removed too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] Joscorbe merged pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
Joscorbe merged PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] gnaresh19 commented on pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
gnaresh19 commented on PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#issuecomment-1239927704
> It looks good to me. I have run all the IT using a Mongo 4.2 to see if something fails, as by default the pipeline is still using Mongo 3.6.
Thanks for the review.
Please do let me know if anything fails with Mongo4.2 in the pipeline, I will look at it.
Also, how do I get access to jenkins if I need to configure/update pipelines for the features I will be working on?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] rishabhdaim commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
rishabhdaim commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r965682682
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDBConfig.java:
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jackrabbit.oak.plugins.document.mongo;
+
+import com.google.common.collect.ImmutableMap;
+import org.apache.jackrabbit.oak.commons.json.JsonObject;
+import org.bson.BsonDocument;
+import org.bson.conversions.Bson;
+
+import java.util.Map;
+
+public class MongoDBConfig {
+ public static final String COLLECTION_COMPRESSION_TYPE = "collectionCompressionType";
+ public static final String STORAGE_ENGINE = "wiredTiger";
+ public static final String STORAGE_CONFIG = "configString";
+
+ enum CollectionCompressor {
+ SNAPPY("snappy"), ZLIB("zlib"), ZSTD("zstd");
+
+ private final String compressionType;
+
+ private static final Map<String, CollectionCompressor> supportedCompressionTypes = ImmutableMap.of(
Review Comment:
```suggestion
private static final Map<String, CollectionCompressor> COLLECTION_COMPRESSOR_MAP;
static {
ImmutableMap.Builder<String, CollectionCompressor> builder = new ImmutableMap.Builder<>();
for (CollectionCompressor value : CollectionCompressor.values()) {
builder.put(value.getName().toLowerCase(), value);
}
COLLECTION_COMPRESSOR_MAP = builder.build();
}
```
Using lowerCase as a key, avoids unnecessary errors because of the case, and using `Enum.values()` would help in avoiding missing a new compressor type (if added).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] rishabhdaim commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
rishabhdaim commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r965774219
##########
oak-store-document/src/test/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDBConfigTest.java:
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jackrabbit.oak.plugins.document.mongo;
+
+import com.mongodb.BasicDBObject;
+import com.mongodb.MongoClient;
+import org.apache.jackrabbit.oak.plugins.document.mongo.MongoDBConfig.CollectionCompressor;
+import org.bson.BsonDocument;
+import org.bson.conversions.Bson;
+import org.junit.Test;
+
+import java.util.Collections;
+import static org.junit.Assert.assertTrue;
+
+
+public class MongoDBConfigTest {
+
+ private MongoDBConfig mongoDBConfig;
+
+ @Test
+ public void defaultCollectionStorageOptions() {
+ Bson bson = mongoDBConfig.getCollectionStorageOptions(Collections.emptyMap());
+ BsonDocument bsonDocument = bson.toBsonDocument(BasicDBObject.class, MongoClient.getDefaultCodecRegistry());
+ String configuredCompressor = bsonDocument.getDocument(MongoDBConfig.STORAGE_ENGINE).getString(MongoDBConfig.STORAGE_CONFIG).getValue();
+ assertTrue(configuredCompressor.indexOf(CollectionCompressor.SNAPPY.getCompressionType()) > 0);
+
+ }
+
+ @Test (expected = IllegalArgumentException.class)
+ public void invalidCollectionStorageOptions() throws Exception {
+ mongoDBConfig.getCollectionStorageOptions(Collections.singletonMap(MongoDBConfig.COLLECTION_COMPRESSION_TYPE, "Invalid"));
+ }
Review Comment:
Please make this exception explicit by adding `Assert.fail("Shouldn't reach here")`.
It would make it explicit that we are not expecting this code to reach here.
##########
oak-store-document/src/test/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDBConfigTest.java:
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jackrabbit.oak.plugins.document.mongo;
+
+import com.mongodb.BasicDBObject;
+import com.mongodb.MongoClient;
+import org.apache.jackrabbit.oak.plugins.document.mongo.MongoDBConfig.CollectionCompressor;
+import org.bson.BsonDocument;
+import org.bson.conversions.Bson;
+import org.junit.Test;
+
+import java.util.Collections;
+import static org.junit.Assert.assertTrue;
+
+
+public class MongoDBConfigTest {
+
+ private MongoDBConfig mongoDBConfig;
+
+ @Test
+ public void defaultCollectionStorageOptions() {
+ Bson bson = mongoDBConfig.getCollectionStorageOptions(Collections.emptyMap());
+ BsonDocument bsonDocument = bson.toBsonDocument(BasicDBObject.class, MongoClient.getDefaultCodecRegistry());
+ String configuredCompressor = bsonDocument.getDocument(MongoDBConfig.STORAGE_ENGINE).getString(MongoDBConfig.STORAGE_CONFIG).getValue();
+ assertTrue(configuredCompressor.indexOf(CollectionCompressor.SNAPPY.getCompressionType()) > 0);
+
+ }
+
+ @Test (expected = IllegalArgumentException.class)
+ public void invalidCollectionStorageOptions() throws Exception {
+ mongoDBConfig.getCollectionStorageOptions(Collections.singletonMap(MongoDBConfig.COLLECTION_COMPRESSION_TYPE, "Invalid"));
+ }
+
+ @Test
+ public void overrideDefaultCollectionStorageOptions() {
+ Bson bson = mongoDBConfig.getCollectionStorageOptions(Collections.singletonMap(MongoDBConfig.COLLECTION_COMPRESSION_TYPE, "zstd"));
+ BsonDocument bsonDocument = bson.toBsonDocument(BasicDBObject.class, MongoClient.getDefaultCodecRegistry());
+ String configuredCompressor = bsonDocument.getDocument(MongoDBConfig.STORAGE_ENGINE).getString(MongoDBConfig.STORAGE_CONFIG).getValue();
Review Comment:
We can use static imports to make this code more concise and readable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] rishabhdaim commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
rishabhdaim commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r965695309
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDBConfig.java:
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jackrabbit.oak.plugins.document.mongo;
+
+import com.google.common.collect.ImmutableMap;
+import org.apache.jackrabbit.oak.commons.json.JsonObject;
+import org.bson.BsonDocument;
+import org.bson.conversions.Bson;
+
+import java.util.Map;
+
+public class MongoDBConfig {
+ public static final String COLLECTION_COMPRESSION_TYPE = "collectionCompressionType";
+ public static final String STORAGE_ENGINE = "wiredTiger";
+ public static final String STORAGE_CONFIG = "configString";
Review Comment:
Same as above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] gnaresh19 commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
gnaresh19 commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r963796623
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDocumentStore.java:
##########
@@ -383,16 +393,17 @@ private MongoDBConnection getOrCreateClusterNodesConnection(@NotNull MongoDocume
return mc;
}
- private void ensureIndexes(@NotNull MongoStatus mongoStatus) {
+ private void ensureIndexes(MongoDatabase db, @NotNull MongoStatus mongoStatus) {
// reading documents in the nodes collection and checking
// existing indexes is performed against the MongoDB primary
// this ensures the information is up-to-date and accurate
boolean emptyNodesCollection = execute(session -> MongoUtils.isCollectionEmpty(nodes, session), Collection.NODES);
-
+ createCollection(db, Collection.NODES.toString(), mongoStatus);
Review Comment:
This method performs the following, only for mongo v4.2 and above:
- if collection exists (empty or with data), skips creating new collection
- else, creates collection per collection config
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] gnaresh19 commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
gnaresh19 commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r963877911
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDocumentStore.java:
##########
@@ -242,7 +244,7 @@ enum DocumentReadPreference {
* How many times should be the bulk update request retries in case of
* a conflict.
* <p>
- * Default is 0 (no retries).
+ * Default is 0 (no retries).≠≠
Review Comment:
Addressed it.
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDocumentStore.java:
##########
@@ -383,16 +393,17 @@ private MongoDBConnection getOrCreateClusterNodesConnection(@NotNull MongoDocume
return mc;
}
- private void ensureIndexes(@NotNull MongoStatus mongoStatus) {
+ private void ensureIndexes(MongoDatabase db, @NotNull MongoStatus mongoStatus) {
Review Comment:
Addressed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] gnaresh19 commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
gnaresh19 commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r963796623
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDocumentStore.java:
##########
@@ -383,16 +393,17 @@ private MongoDBConnection getOrCreateClusterNodesConnection(@NotNull MongoDocume
return mc;
}
- private void ensureIndexes(@NotNull MongoStatus mongoStatus) {
+ private void ensureIndexes(MongoDatabase db, @NotNull MongoStatus mongoStatus) {
// reading documents in the nodes collection and checking
// existing indexes is performed against the MongoDB primary
// this ensures the information is up-to-date and accurate
boolean emptyNodesCollection = execute(session -> MongoUtils.isCollectionEmpty(nodes, session), Collection.NODES);
-
+ createCollection(db, Collection.NODES.toString(), mongoStatus);
Review Comment:
This method performs:
- if collection exists (empty or with data), skips creating new collection
- else, creates collection per collection config
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] rishabhdaim commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
rishabhdaim commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r965690953
##########
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDBConfig.java:
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jackrabbit.oak.plugins.document.mongo;
+
+import com.google.common.collect.ImmutableMap;
+import org.apache.jackrabbit.oak.commons.json.JsonObject;
+import org.bson.BsonDocument;
+import org.bson.conversions.Bson;
+
+import java.util.Map;
+
+public class MongoDBConfig {
+ public static final String COLLECTION_COMPRESSION_TYPE = "collectionCompressionType";
+ public static final String STORAGE_ENGINE = "wiredTiger";
Review Comment:
It should be made default scoped.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] gnaresh19 commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
gnaresh19 commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r966186075
##########
oak-store-document/src/test/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDBConfigTest.java:
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jackrabbit.oak.plugins.document.mongo;
+
+import com.mongodb.BasicDBObject;
+import com.mongodb.MongoClient;
+import org.apache.jackrabbit.oak.plugins.document.mongo.MongoDBConfig.CollectionCompressor;
+import org.bson.BsonDocument;
+import org.bson.conversions.Bson;
+import org.junit.Test;
+
+import java.util.Collections;
+import static org.junit.Assert.assertTrue;
+
+
+public class MongoDBConfigTest {
+
+ private MongoDBConfig mongoDBConfig;
+
+ @Test
+ public void defaultCollectionStorageOptions() {
+ Bson bson = mongoDBConfig.getCollectionStorageOptions(Collections.emptyMap());
+ BsonDocument bsonDocument = bson.toBsonDocument(BasicDBObject.class, MongoClient.getDefaultCodecRegistry());
+ String configuredCompressor = bsonDocument.getDocument(MongoDBConfig.STORAGE_ENGINE).getString(MongoDBConfig.STORAGE_CONFIG).getValue();
+ assertTrue(configuredCompressor.indexOf(CollectionCompressor.SNAPPY.getCompressionType()) > 0);
Review Comment:
I agree with assertTrue() for comparing against expected value, but in this particular test condition, the expected value (17) is not significant, the validation is on whether the requested compressor is present. Pls do let me know your thoughts on this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] gnaresh19 commented on pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
gnaresh19 commented on PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#issuecomment-1241000693
@rishabhdaim : Thanks, addressed the UTC per MongoThrottlerFactoryTest.java#L41-L45
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [jackrabbit-oak] rishabhdaim commented on a diff in pull request #684: [Oak 9919] Add support for zstd, zlib to document store with mongodb
Posted by GitBox <gi...@apache.org>.
rishabhdaim commented on code in PR #684:
URL: https://github.com/apache/jackrabbit-oak/pull/684#discussion_r966193381
##########
oak-store-document/src/test/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDBConfigTest.java:
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.jackrabbit.oak.plugins.document.mongo;
+
+import com.mongodb.BasicDBObject;
+import com.mongodb.MongoClient;
+import org.apache.jackrabbit.oak.plugins.document.mongo.MongoDBConfig.CollectionCompressor;
+import org.bson.BsonDocument;
+import org.bson.conversions.Bson;
+import org.junit.Test;
+
+import java.util.Collections;
+import static org.junit.Assert.assertTrue;
+
+
+public class MongoDBConfigTest {
+
+ private MongoDBConfig mongoDBConfig;
+
+ @Test
+ public void defaultCollectionStorageOptions() {
+ Bson bson = mongoDBConfig.getCollectionStorageOptions(Collections.emptyMap());
+ BsonDocument bsonDocument = bson.toBsonDocument(BasicDBObject.class, MongoClient.getDefaultCodecRegistry());
+ String configuredCompressor = bsonDocument.getDocument(MongoDBConfig.STORAGE_ENGINE).getString(MongoDBConfig.STORAGE_CONFIG).getValue();
+ assertTrue(configuredCompressor.indexOf(CollectionCompressor.SNAPPY.getCompressionType()) > 0);
+
+ }
+
+ @Test (expected = IllegalArgumentException.class)
+ public void invalidCollectionStorageOptions() throws Exception {
+ mongoDBConfig.getCollectionStorageOptions(Collections.singletonMap(MongoDBConfig.COLLECTION_COMPRESSION_TYPE, "Invalid"));
+ }
Review Comment:
I mean writing this test as follows: https://github.com/apache/jackrabbit-oak/blob/trunk/oak-store-document/src/test/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoThrottlerFactoryTest.java#L41-L45
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org