[GitHub] [incubator-pegasus] neverchanje commented on a change in pull request #698: feat(split): gc useless data after partition split

[GitBox <gi...@apache.org>]

neverchanje commented on a change in pull request #698:
URL: https://github.com/apache/incubator-pegasus/pull/698#discussion_r598510187



##########
File path: src/server/key_ttl_compaction_filter.h
##########
@@ -58,16 +69,28 @@ class KeyWithTTLCompactionFilter : public rocksdb::CompactionFilter
             *value_changed = true;
             return false;
         }
-        return check_if_ts_expired(utils::epoch_now(), expire_ts);
+        return check_if_ts_expired(utils::epoch_now(), expire_ts) || check_if_stale_split_data(key);
     }
 
     const char *Name() const override { return "KeyWithTTLCompactionFilter"; }
 
+    bool check_if_stale_split_data(const rocksdb::Slice &key) const

Review comment:
       ```suggestion
       // Check if the record is stale after partition-split,
       // which splits the partition into two halves.
       // The stale record belongs to the other half.
       bool check_if_stale_split_data(const rocksdb::Slice &key) const
   ```

##########
File path: src/base/pegasus_const.h
##########
@@ -62,4 +62,6 @@ extern const std::string PEGASUS_CLUSTER_SECTION_NAME;
 extern const std::string ROCKSDB_ENV_SLOW_QUERY_THRESHOLD;
 
 extern const std::string ROCKSDB_ITERATION_THRESHOLD_TIME_MS;
+
+extern const std::string SPLIT_VALIDATE_PARTITION_HASH;

Review comment:
       ```suggestion
   extern const std::string SPLIT_CHECK_PARTITION_HASH;
   ```

##########
File path: src/server/pegasus_server_impl.cpp
##########
@@ -2438,6 +2442,25 @@ void pegasus_server_impl::update_rocksdb_iteration_threshold(
     }
 }
 
+void pegasus_server_impl::update_validate_partition_hash(

Review comment:
       ```suggestion
   void pegasus_server_impl::update_check_partition_hash(
   ```
   
   Be consistent with the concept "check_partition_hash", it would be ambiguous.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org