You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kvrocks.apache.org by "ChaseForward (via GitHub)" <gi...@apache.org> on 2026/09/27 08:41:43 UTC

[PR] feat(cf): add CF.INFO command [kvrocks]

ChaseForward opened a new pull request, #3649:
URL: https://github.com/apache/kvrocks/pull/3649

   # Description
   This PR adds the CF.INFO command, bringing RedisBloom compatibility to the cuckoo filter. It returns the information of a cuckoo filter, including size, number of buckets/filters/items, bucket size, expansion rate and max iterations. A specific field can be queried with CF.INFO key <field>.
   
   Return an empty result if the filter does not exist, following the behavior of the CF.INFO implementation in RedisBloom.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(cf): add CF.INFO command [kvrocks]

Posted by "PragmaTwice (via GitHub)" <gi...@apache.org>.
PragmaTwice commented on code in PR #3649:
URL: https://github.com/apache/kvrocks/pull/3649#discussion_r4151878471


##########
src/types/redis_cuckoo_chain.h:
##########
@@ -37,6 +37,28 @@ const uint16_t kCFDefaultMaxIterations = 20;
 const uint16_t kCFDefaultExpansion = 1;
 const uint16_t kCFMaxExpansion = 32768;
 
+enum class CuckooInfoType {
+  kAll,
+  kSize,
+  kBuckets,
+  kFilters,
+  kItems,
+  kBucketSize,
+  kExpansion,
+  kMaxIterations,
+};

Review Comment:
   I think this can be moved to command sources.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(cf): add CF.INFO command [kvrocks]

Posted by "jihuayu (via GitHub) via issues" <is...@kvrocks.apache.org>.
jihuayu commented on PR #3649:
URL: https://github.com/apache/kvrocks/pull/3649#issuecomment-6567483664

   cc @nagisa-kunhah. Can you have a look?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(cf): add CF.INFO command [kvrocks]

Posted by "ChaseForward (via GitHub)" <gi...@apache.org>.
ChaseForward commented on PR #3649:
URL: https://github.com/apache/kvrocks/pull/3649#issuecomment-5882407525

   > Hi @ChaseForward Please follow our guidelines and disclose your use of LLMs in the PR.
   
   Thanks for the reminder. I've added the LLM disclosure in the PR description.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(cf): add CF.INFO command [kvrocks]

Posted by "jihuayu (via GitHub) via issues" <is...@kvrocks.apache.org>.
jihuayu merged PR #3649:
URL: https://github.com/apache/kvrocks/pull/3649


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(cf): add CF.INFO command [kvrocks]

Posted by "ChaseForward (via GitHub)" <gi...@apache.org>.
ChaseForward commented on code in PR #3649:
URL: https://github.com/apache/kvrocks/pull/3649#discussion_r4192738386


##########
src/types/redis_cuckoo_chain.h:
##########
@@ -37,6 +37,28 @@ const uint16_t kCFDefaultMaxIterations = 20;
 const uint16_t kCFDefaultExpansion = 1;
 const uint16_t kCFMaxExpansion = 32768;
 
+enum class CuckooInfoType {
+  kAll,
+  kSize,
+  kBuckets,
+  kFilters,
+  kItems,
+  kBucketSize,
+  kExpansion,
+  kMaxIterations,
+};

Review Comment:
   ok, i followed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(cf): add CF.INFO command [kvrocks]

Posted by "jihuayu (via GitHub)" <gi...@apache.org>.
jihuayu commented on PR #3649:
URL: https://github.com/apache/kvrocks/pull/3649#issuecomment-5854516694

   Hi @ChaseForward Please follow our guidelines and disclose your use of LLMs in the PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(cf): add CF.INFO command [kvrocks]

Posted by "ChaseForward (via GitHub) via issues" <is...@kvrocks.apache.org>.
ChaseForward commented on code in PR #3649:
URL: https://github.com/apache/kvrocks/pull/3649#discussion_r4505278588


##########
src/types/redis_cuckoo_chain.cc:
##########
@@ -170,6 +175,66 @@ rocksdb::Status CuckooChain::Add(engine::Context &ctx, const Slice &user_key, co
   return rocksdb::Status::OK();
 }
 
+rocksdb::Status CuckooChain::calculateTotalBuckets(const CuckooChainMetadata &metadata, uint32_t base_buckets,
+                                                   uint64_t *total_buckets) {
+  *total_buckets = base_buckets;
+
+  if (metadata.expansion == 1) {
+    // Equal-sized expansion: every sub-filter has the same bucket count as the initial one.
+    *total_buckets *= metadata.n_filters;
+    return rocksdb::Status::OK();
+  }
+
+  uint64_t capacity = metadata.base_capacity;
+  for (uint16_t filter_idx = 1; filter_idx < metadata.n_filters; ++filter_idx) {
+    if (metadata.expansion != 0 && capacity > std::numeric_limits<uint64_t>::max() / metadata.expansion) {
+      return rocksdb::Status::Corruption("invalid metadata: filter capacity is too large");
+    }
+    capacity *= metadata.expansion;
+
+    uint32_t num_buckets = 0;
+    auto s = CuckooFilterHelper::CalculateRequiredBuckets(capacity, metadata.bucket_size, &num_buckets);
+    if (!s.ok()) {
+      return rocksdb::Status::Corruption("invalid metadata: filter capacity is too large");
+    }
+    *total_buckets += num_buckets;
+  }
+  return rocksdb::Status::OK();
+}
+
+rocksdb::Status CuckooChain::Info(engine::Context &ctx, const Slice &user_key, CuckooFilterInfo *info) {
+  std::string ns_key = AppendNamespacePrefix(user_key);
+  CuckooChainMetadata metadata(false);
+  auto s = getCuckooChainMetadata(ctx, ns_key, &metadata);
+  if (!s.ok()) {
+    return s;
+  }
+
+  s = validateMetadata(metadata);
+  if (!s.ok()) {
+    return s;
+  }
+
+  uint32_t base_buckets = 0;
+  s = CuckooFilterHelper::GetFilterNumBuckets(metadata.base_capacity, metadata.expansion, metadata.bucket_size, 0,
+                                              &base_buckets);
+  if (!s.ok()) return s;
+
+  uint64_t total_buckets = 0;
+  s = calculateTotalBuckets(metadata, base_buckets, &total_buckets);

Review Comment:
   in my opinion, it is better not to store `total_buckets` in the chain metadata, for two points. First, `cf.info` is not on the hot path, so this statistic is not critical. Second, if we persist this value, we have to update it on every insertion and deletion, which introduces extra overhead.
   
   
   > Minor: Could we store `total_buckets` in the chain metadata and update it on expansion, rather than recomputing it on every `CF.INFO` call? It would be worth measuring the cost of `calculateTotalBuckets()` for large cuckoo filter chains first to see whether this is worthwhile.
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(cf): add CF.INFO command [kvrocks]

Posted by "jihuayu (via GitHub)" <gi...@apache.org>.
jihuayu commented on PR #3649:
URL: https://github.com/apache/kvrocks/pull/3649#issuecomment-6291295155

   Hi @ChaseForward, could you please sync this PR with the latest unstable and resolve the merge conflicts? Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(cf): add CF.INFO command [kvrocks]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #3649:
URL: https://github.com/apache/kvrocks/pull/3649#issuecomment-5854290253

   Hi @ChaseForward,
   
   Thank you for your pull request. Please review our [Contributing Guide](https://kvrocks.apache.org/community/contributing/).
   
   Please make sure you understand your changes and explain your reasoning in this pull request. Low-quality pull requests may be closed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(cf): add CF.INFO command [kvrocks]

Posted by "ChaseForward (via GitHub)" <gi...@apache.org>.
ChaseForward commented on PR #3649:
URL: https://github.com/apache/kvrocks/pull/3649#issuecomment-6135448509

   Hi @PragmaTwice, I've addressed your comments (moved CuckooInfoType to the command layer) and rebased onto develop. Could you take another look when you have time? Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(cf): add CF.INFO command [kvrocks]

Posted by "ChaseForward (via GitHub)" <gi...@apache.org>.
ChaseForward commented on PR #3649:
URL: https://github.com/apache/kvrocks/pull/3649#issuecomment-6408324510

   > Hi @ChaseForward, could you please sync this PR with the latest unstable and resolve the merge conflicts? Thanks!
   
   Could you help approve the workflows to run the CI for this PR? Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(cf): add CF.INFO command [kvrocks]

Posted by "PragmaTwice (via GitHub)" <gi...@apache.org>.
PragmaTwice commented on code in PR #3649:
URL: https://github.com/apache/kvrocks/pull/3649#discussion_r4151883647


##########
src/commands/cmd_cuckoo_filter.cc:
##########


Review Comment:
   What's this?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(cf): add CF.INFO command [kvrocks]

Posted by "ChaseForward (via GitHub) via issues" <is...@kvrocks.apache.org>.
ChaseForward commented on code in PR #3649:
URL: https://github.com/apache/kvrocks/pull/3649#discussion_r4504896737


##########
src/types/redis_cuckoo_chain.cc:
##########
@@ -170,6 +175,66 @@ rocksdb::Status CuckooChain::Add(engine::Context &ctx, const Slice &user_key, co
   return rocksdb::Status::OK();
 }
 
+rocksdb::Status CuckooChain::calculateTotalBuckets(const CuckooChainMetadata &metadata, uint32_t base_buckets,
+                                                   uint64_t *total_buckets) {
+  *total_buckets = base_buckets;
+
+  if (metadata.expansion == 1) {
+    // Equal-sized expansion: every sub-filter has the same bucket count as the initial one.
+    *total_buckets *= metadata.n_filters;
+    return rocksdb::Status::OK();
+  }
+
+  uint64_t capacity = metadata.base_capacity;
+  for (uint16_t filter_idx = 1; filter_idx < metadata.n_filters; ++filter_idx) {
+    if (metadata.expansion != 0 && capacity > std::numeric_limits<uint64_t>::max() / metadata.expansion) {
+      return rocksdb::Status::Corruption("invalid metadata: filter capacity is too large");
+    }
+    capacity *= metadata.expansion;
+
+    uint32_t num_buckets = 0;
+    auto s = CuckooFilterHelper::CalculateRequiredBuckets(capacity, metadata.bucket_size, &num_buckets);
+    if (!s.ok()) {
+      return rocksdb::Status::Corruption("invalid metadata: filter capacity is too large");
+    }
+    *total_buckets += num_buckets;
+  }
+  return rocksdb::Status::OK();
+}
+
+rocksdb::Status CuckooChain::Info(engine::Context &ctx, const Slice &user_key, CuckooFilterInfo *info) {
+  std::string ns_key = AppendNamespacePrefix(user_key);
+  CuckooChainMetadata metadata(false);
+  auto s = getCuckooChainMetadata(ctx, ns_key, &metadata);
+  if (!s.ok()) {
+    return s;
+  }
+
+  s = validateMetadata(metadata);
+  if (!s.ok()) {
+    return s;
+  }
+
+  uint32_t base_buckets = 0;
+  s = CuckooFilterHelper::GetFilterNumBuckets(metadata.base_capacity, metadata.expansion, metadata.bucket_size, 0,
+                                              &base_buckets);
+  if (!s.ok()) return s;
+
+  uint64_t total_buckets = 0;
+  s = calculateTotalBuckets(metadata, base_buckets, &total_buckets);

Review Comment:
   ok, let me have a check.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(cf): add CF.INFO command [kvrocks]

Posted by "nagisa-kunhah (via GitHub) via issues" <is...@kvrocks.apache.org>.
nagisa-kunhah commented on code in PR #3649:
URL: https://github.com/apache/kvrocks/pull/3649#discussion_r4502515367


##########
src/types/redis_cuckoo_chain.cc:
##########
@@ -170,6 +175,66 @@ rocksdb::Status CuckooChain::Add(engine::Context &ctx, const Slice &user_key, co
   return rocksdb::Status::OK();
 }
 
+rocksdb::Status CuckooChain::calculateTotalBuckets(const CuckooChainMetadata &metadata, uint32_t base_buckets,
+                                                   uint64_t *total_buckets) {
+  *total_buckets = base_buckets;
+
+  if (metadata.expansion == 1) {
+    // Equal-sized expansion: every sub-filter has the same bucket count as the initial one.
+    *total_buckets *= metadata.n_filters;
+    return rocksdb::Status::OK();
+  }
+
+  uint64_t capacity = metadata.base_capacity;
+  for (uint16_t filter_idx = 1; filter_idx < metadata.n_filters; ++filter_idx) {
+    if (metadata.expansion != 0 && capacity > std::numeric_limits<uint64_t>::max() / metadata.expansion) {
+      return rocksdb::Status::Corruption("invalid metadata: filter capacity is too large");
+    }
+    capacity *= metadata.expansion;
+
+    uint32_t num_buckets = 0;
+    auto s = CuckooFilterHelper::CalculateRequiredBuckets(capacity, metadata.bucket_size, &num_buckets);
+    if (!s.ok()) {
+      return rocksdb::Status::Corruption("invalid metadata: filter capacity is too large");
+    }
+    *total_buckets += num_buckets;
+  }
+  return rocksdb::Status::OK();
+}
+
+rocksdb::Status CuckooChain::Info(engine::Context &ctx, const Slice &user_key, CuckooFilterInfo *info) {
+  std::string ns_key = AppendNamespacePrefix(user_key);
+  CuckooChainMetadata metadata(false);
+  auto s = getCuckooChainMetadata(ctx, ns_key, &metadata);
+  if (!s.ok()) {
+    return s;
+  }
+
+  s = validateMetadata(metadata);
+  if (!s.ok()) {
+    return s;
+  }
+
+  uint32_t base_buckets = 0;
+  s = CuckooFilterHelper::GetFilterNumBuckets(metadata.base_capacity, metadata.expansion, metadata.bucket_size, 0,
+                                              &base_buckets);
+  if (!s.ok()) return s;
+
+  uint64_t total_buckets = 0;
+  s = calculateTotalBuckets(metadata, base_buckets, &total_buckets);

Review Comment:
   Minor: Could we store `total_buckets ` in the chain metadata and update it on expansion, rather than recomputing it on every `CF.INFO` call? It would be worth measuring the cost of `calculateTotalBuckets()` for large cuckoo filter chains first to see whether this is worthwhile.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(cf): add CF.INFO command [kvrocks]

Posted by "nagisa-kunhah (via GitHub) via issues" <is...@kvrocks.apache.org>.
nagisa-kunhah commented on code in PR #3649:
URL: https://github.com/apache/kvrocks/pull/3649#discussion_r4502515367


##########
src/types/redis_cuckoo_chain.cc:
##########
@@ -170,6 +175,66 @@ rocksdb::Status CuckooChain::Add(engine::Context &ctx, const Slice &user_key, co
   return rocksdb::Status::OK();
 }
 
+rocksdb::Status CuckooChain::calculateTotalBuckets(const CuckooChainMetadata &metadata, uint32_t base_buckets,
+                                                   uint64_t *total_buckets) {
+  *total_buckets = base_buckets;
+
+  if (metadata.expansion == 1) {
+    // Equal-sized expansion: every sub-filter has the same bucket count as the initial one.
+    *total_buckets *= metadata.n_filters;
+    return rocksdb::Status::OK();
+  }
+
+  uint64_t capacity = metadata.base_capacity;
+  for (uint16_t filter_idx = 1; filter_idx < metadata.n_filters; ++filter_idx) {
+    if (metadata.expansion != 0 && capacity > std::numeric_limits<uint64_t>::max() / metadata.expansion) {
+      return rocksdb::Status::Corruption("invalid metadata: filter capacity is too large");
+    }
+    capacity *= metadata.expansion;
+
+    uint32_t num_buckets = 0;
+    auto s = CuckooFilterHelper::CalculateRequiredBuckets(capacity, metadata.bucket_size, &num_buckets);
+    if (!s.ok()) {
+      return rocksdb::Status::Corruption("invalid metadata: filter capacity is too large");
+    }
+    *total_buckets += num_buckets;
+  }
+  return rocksdb::Status::OK();
+}
+
+rocksdb::Status CuckooChain::Info(engine::Context &ctx, const Slice &user_key, CuckooFilterInfo *info) {
+  std::string ns_key = AppendNamespacePrefix(user_key);
+  CuckooChainMetadata metadata(false);
+  auto s = getCuckooChainMetadata(ctx, ns_key, &metadata);
+  if (!s.ok()) {
+    return s;
+  }
+
+  s = validateMetadata(metadata);
+  if (!s.ok()) {
+    return s;
+  }
+
+  uint32_t base_buckets = 0;
+  s = CuckooFilterHelper::GetFilterNumBuckets(metadata.base_capacity, metadata.expansion, metadata.bucket_size, 0,
+                                              &base_buckets);
+  if (!s.ok()) return s;
+
+  uint64_t total_buckets = 0;
+  s = calculateTotalBuckets(metadata, base_buckets, &total_buckets);

Review Comment:
   Minor: Could we store `total_buckets` in the chain metadata and update it on expansion, rather than recomputing it on every `CF.INFO` call? It would be worth measuring the cost of `calculateTotalBuckets()` for large cuckoo filter chains first to see whether this is worthwhile.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org