[GitHub] [rocketmq-streams] comecny opened a new issue, #279: multi-instance applications consumers problem

["comecny (via GitHub)" <gi...@apache.org>]

comecny opened a new issue, #279:
URL: https://github.com/apache/rocketmq-streams/issues/279

   Can multiple consumers on the same topic join or window?  I want to use it in multi-instance applications 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] multi-instance applications consumers problem [rocketmq-streams]

["RockteMQ-AI (via GitHub)" <gi...@apache.org>]

RockteMQ-AI commented on issue #279:
URL: https://github.com/apache/rocketmq-streams/issues/279#issuecomment-5215664169

   **Issue Evaluation**
   
   Category: `question`
   
   In RocketMQ Streams, when multiple consumer instances consume the same topic:
   
   1. **Join operations**: Each instance processes its assigned queues independently. Join operations (e.g., `join()`) require data from both sides to be co-located on the same instance. If the two topics being joined have different queue assignments, the join may not produce correct results unless you use a **keyBy** to ensure data affinity.
   
   2. **Window operations**: Window computations are per-instance based on queue assignment. Each instance maintains its own window state for its assigned queues. Windows will work correctly as long as each instance processes its own partition's data within the window timeframe.
   
   3. **Scaling**: Adding more instances triggers a rebalance — queues are redistributed. During rebalance, some windows may be incomplete. Use `WindowStream.fireBeforeClose()` or similar mechanisms to handle late data.
   
   For multi-instance deployments, ensure:
   - Use `keyBy()` for operations that require data co-location
   - Configure appropriate watermark strategies for window correctness
   - Monitor rebalance frequency to minimize window disruption
   
   For more details, refer to the project documentation and source code in the `stream` module.
   
   ---
   *Automated evaluation by RocketMQ AI Assistant*


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq-streams] ni-ze closed issue #279: multi-instance applications consumers problem

["ni-ze (via GitHub)" <gi...@apache.org>]

ni-ze closed issue #279: multi-instance applications consumers  problem 
URL: https://github.com/apache/rocketmq-streams/issues/279


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq-streams] ni-ze commented on issue #279: multi-instance applications consumers problem

["ni-ze (via GitHub)" <gi...@apache.org>]

ni-ze commented on issue #279:
URL: https://github.com/apache/rocketmq-streams/issues/279#issuecomment-1482135072

   It is not support join on the same topic, because you are not know the queue will be consumed be which consumer.
   multi-instance applications is the as the single-instance example, you can start up two example which is exactly same to consume common topic.
   BTW, what is the problem do you want to solve?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org