You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by GitBox <gi...@apache.org> on 2022/08/22 16:41:22 UTC
[GitHub] [ignite-3] sashapolo opened a new pull request, #1031: IGNITE-17338 Implement RocksDB hash index storage
sashapolo opened a new pull request, #1031:
URL: https://github.com/apache/ignite-3/pull/1031
https://issues.apache.org/jira/browse/IGNITE-17338
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] ibessonov commented on a diff in pull request #1031: IGNITE-17338 Implement RocksDB hash index storage
Posted by GitBox <gi...@apache.org>.
ibessonov commented on code in PR #1031:
URL: https://github.com/apache/ignite-3/pull/1031#discussion_r952326547
##########
modules/storage-rocksdb/src/main/java/org/apache/ignite/internal/storage/rocksdb/RocksDbTableStorage.java:
##########
@@ -98,9 +101,15 @@ class RocksDbTableStorage implements MvTableStorage {
/** Column Family handle for partition data. */
private volatile ColumnFamily partitionCf;
+ /** Column Family handle for Hash Index data. */
+ private volatile ColumnFamily hashIndexCf;
+
/** Partition storages. */
private volatile AtomicReferenceArray<RocksDbMvPartitionStorage> partitions;
+ /** Hash Index storages by their names. */
+ private final ConcurrentMap<String, HashIndexStorage> hashIndices = new ConcurrentHashMap<>();
Review Comment:
Storing indexed by ids would be more appropriate
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] ibessonov merged pull request #1031: IGNITE-17338 Implement RocksDB hash index storage
Posted by GitBox <gi...@apache.org>.
ibessonov merged PR #1031:
URL: https://github.com/apache/ignite-3/pull/1031
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sashapolo commented on a diff in pull request #1031: IGNITE-17338 Implement RocksDB hash index storage
Posted by GitBox <gi...@apache.org>.
sashapolo commented on code in PR #1031:
URL: https://github.com/apache/ignite-3/pull/1031#discussion_r952647721
##########
modules/storage-rocksdb/src/main/java/org/apache/ignite/internal/storage/rocksdb/index/RocksDbHashIndexStorage.java:
##########
@@ -145,20 +161,46 @@ public void remove(IndexRow row) {
}
}
+ @Override
+ public void destroy() {
+ byte[] rangeEnd = rangeEnd(constantPrefix);
+
+ try (WriteOptions writeOptions = new WriteOptions().setDisableWAL(true)) {
+ // If we can't construct the range end prefix (nearly impossible case), we need to fallback to manually removing the data,
+ // since 'deleteRange' doesn't work without the upper bound.
+ if (rangeEnd == null) {
Review Comment:
> By the way, have you updated the list of column families in RocksDbTableStorage#scheduleFlush
Wow, this is a very nice hint
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] ibessonov commented on a diff in pull request #1031: IGNITE-17338 Implement RocksDB hash index storage
Posted by GitBox <gi...@apache.org>.
ibessonov commented on code in PR #1031:
URL: https://github.com/apache/ignite-3/pull/1031#discussion_r952632725
##########
modules/storage-rocksdb/src/main/java/org/apache/ignite/internal/storage/rocksdb/index/RocksDbHashIndexStorage.java:
##########
@@ -145,20 +161,46 @@ public void remove(IndexRow row) {
}
}
+ @Override
+ public void destroy() {
+ byte[] rangeEnd = rangeEnd(constantPrefix);
+
+ try (WriteOptions writeOptions = new WriteOptions().setDisableWAL(true)) {
+ // If we can't construct the range end prefix (nearly impossible case), we need to fallback to manually removing the data,
+ // since 'deleteRange' doesn't work without the upper bound.
+ if (rangeEnd == null) {
Review Comment:
I'd rather have an assertion that this is impossible. Partition id stops way before 0xffff.
By the way, have you updated the list of column families in `RocksDbTableStorage#scheduleFlush`? It's easy to miss, because I didn't document it :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sashapolo commented on a diff in pull request #1031: IGNITE-17338 Implement RocksDB hash index storage
Posted by GitBox <gi...@apache.org>.
sashapolo commented on code in PR #1031:
URL: https://github.com/apache/ignite-3/pull/1031#discussion_r952486574
##########
modules/storage-rocksdb/src/main/java/org/apache/ignite/internal/storage/rocksdb/RocksDbTableStorage.java:
##########
@@ -98,9 +101,15 @@ class RocksDbTableStorage implements MvTableStorage {
/** Column Family handle for partition data. */
private volatile ColumnFamily partitionCf;
+ /** Column Family handle for Hash Index data. */
+ private volatile ColumnFamily hashIndexCf;
+
/** Partition storages. */
private volatile AtomicReferenceArray<RocksDbMvPartitionStorage> partitions;
+ /** Hash Index storages by their names. */
+ private final ConcurrentMap<String, HashIndexStorage> hashIndices = new ConcurrentHashMap<>();
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org