You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "zhbdesign (via GitHub)" <gi...@apache.org> on 2023/03/25 00:58:15 UTC
[GitHub] [rocketmq] zhbdesign opened a new issue, #6471: Does DefaultLitePullConsumer support batch saving mode
zhbdesign opened a new issue, #6471:
URL: https://github.com/apache/rocketmq/issues/6471
The issue tracker is used for bug reporting purposes **ONLY** whereas feature request needs to follow the [RIP process](https://github.com/apache/rocketmq/wiki/RocketMQ-Improvement-Proposal). To avoid unnecessary duplication, please check whether there is a previous issue before filing a new one.
It is recommended to start a discussion thread in the [mailing lists](http://rocketmq.apache.org/about/contact/) or [github discussions](https://github.com/apache/rocketmq/discussions) in cases of discussing your deployment plan, API clarification, and other non-bug-reporting issues.
We welcome any friendly suggestions, bug fixes, collaboration, and other improvements.
Please ensure that your bug report is clear and self-contained. Otherwise, it would take additional rounds of communication, thus more time, to understand the problem itself.
Generally, fixing an issue goes through the following steps:
1. Understand the issue reported;
1. Reproduce the unexpected behavior locally;
1. Perform root cause analysis to identify the underlying problem;
1. Create test cases to cover the identified problem;
1. Work out a solution to rectify the behavior and make the newly created test cases pass;
1. Make a pull request and go through peer review;
As a result, it would be very helpful yet challenging if you could provide an isolated project reproducing your reported issue. Anyway, please ensure your issue report is informative enough for the community to pick up. At a minimum, include the following hints:
**BUG REPORT**
1. Please describe the issue you observed:
- What did you do (The steps to reproduce)?
- What is expected to see?
- What did you see instead?
2. Please tell us about your environment:
3. Other information (e.g. detailed explanation, logs, related issues, suggestions on how to fix, etc):
**FEATURE REQUEST**
1. Please describe the feature you are requesting.
Does DefaultLitePullConsumer support batch saving mode, such as 10 seconds and 1000 pieces? If yes, what is the parameter configuration? Only batchSize is set when checking demo.
DefaultLitePullConsumer支持攒批模式吗，比如10秒和1000条，满足一个条件就批量拉取入库，如果支持，参数怎么配置，查看demo都是只配置了batchSize。
2. Provide any additional detail on your proposed use case for this feature.
3. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?
4. If there are some sub-tasks involved, use -[] for each sub-task and create a corresponding issue to map to the sub-task:
- [sub-task1-issue-number](example_sub_issue1_link_here): sub-task1 description here,
- [sub-task2-issue-number](example_sub_issue2_link_here): sub-task2 description here,
- ...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] miles-ton commented on issue #6471: Does DefaultLitePullConsumer support batch saving mode
Posted by "miles-ton (via GitHub)" <gi...@apache.org>.
miles-ton commented on issue #6471:
URL: https://github.com/apache/rocketmq/issues/6471#issuecomment-1483795454
hello @zhbdesign as far as i known there are no way to support accumulated pulling mode in rocketmq, when pull requests arrives at broker(means consumer sent a consume request to broker), the broker will process these pull requests every 5s(long polling) or 1s(short polling). Regardless of the number of messages held by broker, broker will respond to messages for consumer even if the number of messages is less than the batch size.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Does DefaultLitePullConsumer support batch saving mode [rocketmq]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #6471:
URL: https://github.com/apache/rocketmq/issues/6471#issuecomment-2028958865
This issue is stale because it has been open for 365 days with no activity. It will be closed in 3 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Does DefaultLitePullConsumer support batch saving mode [rocketmq]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #6471:
URL: https://github.com/apache/rocketmq/issues/6471#issuecomment-2038478512
This issue was closed because it has been inactive for 3 days since being marked as stale.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] zhbdesign commented on issue #6471: Does DefaultLitePullConsumer support batch saving mode
Posted by "zhbdesign (via GitHub)" <gi...@apache.org>.
zhbdesign commented on issue #6471:
URL: https://github.com/apache/rocketmq/issues/6471#issuecomment-1483804209
> hello @zhbdesign as far as i known there are no way to support accumulated pulling mode in rocketmq, when pull requests arrives at broker(means consumer sent a consume request to broker), the broker will process these pull requests every 5s(long polling) or 1s(short polling). Regardless of the number of messages held by broker, broker will respond to messages for consumer even if the number of messages is less than the batch size.
Is there any other way to solve this problem? kafka supports it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] changyoutianxia commented on issue #6471: Does DefaultLitePullConsumer support batch saving mode
Posted by "changyoutianxia (via GitHub)" <gi...@apache.org>.
changyoutianxia commented on issue #6471:
URL: https://github.com/apache/rocketmq/issues/6471#issuecomment-1492958857
SimpleConsumer simpleConsumer = provider.newSimpleConsumerBuilder()
.setClientConfiguration(configuration)
.setSubscriptionExpressions(subscriptionMap)
.setConsumerGroup(mqConsumerConfig.getConsumerGroup()).build();
while (true) {
try {
List<MessageView> messageViews = simpleConsumer.receive(30, Duration.ofSeconds(10));
if (Objects.isNull(messageViews) || messageViews.isEmpty()) {
continue;
}
List<SendMessage> mqMessageList = messageViews.stream().map(v -> JSONUtils.toObject(new String(v.getBody().array(), StandardCharsets.UTF_8), SendMessage.class)).collect(Collectors.toList());
List<FullMessage> fullMessages = FullMessageConvert.INSTANCE.mqToPo(mqMessageList);
fullMessageService.saveBatch(fullMessages);
for (MessageView messageView : messageViews) {
simpleConsumer.ack(messageView);
}
} catch (Exception e) {
log.error("consumer message fail,error:{}", e.getMessage());
}
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Does DefaultLitePullConsumer support batch saving mode [rocketmq]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #6471: Does DefaultLitePullConsumer support batch saving mode
URL: https://github.com/apache/rocketmq/issues/6471
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org