[I] [Enhancement] Selective double-write in CombineConsumeQueueStore [rocketmq]

["imzs (via GitHub)" <gi...@apache.org>]

imzs opened a new issue, #10450:
URL: https://github.com/apache/rocketmq/issues/10450

   ### Before Creating the Enhancement Request
   
   - [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature.
   
   
   ### Summary
   
   CombineConsumeQueueStore was originally designed as a transitional state in which every topic is double-written to both the file-based ConsumeQueue and the RocksDB ConsumeQueue. As it has gradually become a long-running state in practice, we observed that not all topics actually need double-write. In particular, LITE topics need it more than ordinary topics, because LMQ is required to run on top of the RocksDB ConsumeQueue. 
   
   This is the original motivation for introducing selective double-write.
   To avoid an incompatible breaking change, we introduce a new opt-in switch `rocksdbCQSelectiveDoubleWriteEnable` as an additional control rather than altering the existing default behavior. Existing deployments are therefore unaffected unless they explicitly opt in.
   
   ### Motivation
   
   .
   
   ### Describe the Solution You'd Like
   
   .
   
   ### Describe Alternatives You've Considered
   
   .
   
   ### Additional Context
   
   .


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Enhancement] Selective double-write in CombineConsumeQueueStore [rocketmq]

["oss-sentinel-ai (via GitHub)" <gi...@apache.org>]

oss-sentinel-ai commented on issue #10450:
URL: https://github.com/apache/rocketmq/issues/10450#issuecomment-4657555585

   ## Assessment by github-manager-bot
   
   **Classification:** 🔧 Enhancement
   
   ### Summary
   The issue proposes introducing selective double-write in `CombineConsumeQueueStore`. Currently, every topic is double-written to both file-based ConsumeQueue and RocksDB ConsumeQueue. The proposal is to make this opt-in via a new switch `rocksdbCQSelectiveDoubleWriteEnable`, since not all topics need double-write (only LITE topics require it for LMQ).
   
   ### Analysis
   This is a well-motivated performance optimization:
   - **Problem**: Unnecessary double-write overhead for topics that don't need RocksDB ConsumeQueue
   - **Solution**: Introduce a configurable switch to control double-write behavior per topic type
   - **Backward compatibility**: The switch is opt-in, avoiding breaking changes
   
   ### Recommendation
   ✅ **Valid enhancement request.** The proposal is technically sound and addresses a real performance concern. The opt-in approach ensures backward compatibility. This requires maintainer review to assess the design and implementation approach.
   
   ---
   *Automated assessment by github-manager-bot*


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Enhancement] Selective double-write in CombineConsumeQueueStore [rocketmq]

["RockteMQ-AI (via GitHub)" <gi...@apache.org>]

RockteMQ-AI commented on issue #10450:
URL: https://github.com/apache/rocketmq/issues/10450#issuecomment-4689420456

   **Issue Evaluation**
   
   Category: `type/enhancement` | Status: **Evaluated**
   
   **Feasibility:** Feasible — well-motivated by observed production behavior.
   **Scope:** `store` module — `CombineConsumeQueueStore`
   **Compatibility:** Internal optimization; no API/protocol changes.
   
   Selective double-write is a sensible evolution. Not all topics need dual persistence — LITE topics have a stronger requirement for RocksDB ConsumeQueue than ordinary topics. The proposed approach of making double-write configurable per-topic aligns with the original transitional design intent.
   
   **Consideration:** Ensure the transition path (full double-write → selective) is safe for in-flight topics that may have existing data in both stores.
   
   ---
   *Automated evaluation by RockteMQ-AI*


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org