You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2021/02/03 03:52:52 UTC
[GitHub] [rocketmq] xxd763795151 opened a new issue #2656: dledger should logging asynchronously
xxd763795151 opened a new issue #2656:
URL: https://github.com/apache/rocketmq/issues/2656
The issue tracker is **ONLY** used for bug report(feature request need to follow [RIP process](https://github.com/apache/rocketmq/wiki/RocketMQ-Improvement-Proposal)). Keep in mind, please check whether there is an existing same report before your raise a new one.
Alternately (especially if your communication is not a bug report), you can send mail to our [mailing lists](http://rocketmq.apache.org/about/contact/). We welcome any friendly suggestions, bug fixes, collaboration and other improvements.
Please ensure that your bug report is clear and that it is complete. Otherwise, we may be unable to understand it or to reproduce it, either of which would prevent us from fixing the bug. We strongly recommend the report(bug report or feature request) could include some hints as the following:
**BUG REPORT**
1. Please describe the issue you observed:
- What did you do (The steps to reproduce)?
io.openmessaging.storage.dledger.DLedgerEntryPusher >>> inner class: QuorumAckChecker#doWork, logging as following:
```
if (DLedgerUtils.elapsed(lastPrintWatermarkTimeMs) > 3000) {
logger.info("[{}][{}] term={} ledgerBegin={} ledgerEnd={} committed={} watermarks={}",
memberState.getSelfId(), memberState.getRole(), memberState.currTerm(), dLedgerStore.getLedgerBeginIndex(), dLedgerStore.getLedgerEndIndex(), dLedgerStore.getCommittedIndex(), JSON.toJSONString(peerWaterMarksByTerm));
lastPrintWatermarkTimeMs = System.currentTimeMillis();
}
```
It is use default appender to log synchronusly. But there are some questions : if log occur rolling and archive, that will cost too time. In my mac , about 400-600ms，different cloud virtual machine：about 600-900ms，physical machine (on our production environment) more than 1000ms. As a result, it may cause the client send timeout(send tps > 1200/broker and master-slave network latency about 0.01 - 0.3 ms in the most time, client`s timeout config is 3s).
The log record for this place is between in the client send request-response, so it logging asynchronously .
- What did you expect to see?
- What did you see instead?
2. Please tell us about your environment:
3. Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] duhenglucky commented on issue #2656: DLedger should logging asynchronously
Posted by GitBox <gi...@apache.org>.
duhenglucky commented on issue #2656:
URL: https://github.com/apache/rocketmq/issues/2656#issuecomment-785507603
> by
would you like to submit a PR to add an async log appender for Deldeger in the logback_broker.xml?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] RongtongJin commented on issue #2656: DLedger should logging asynchronously
Posted by GitBox <gi...@apache.org>.
RongtongJin commented on issue #2656:
URL: https://github.com/apache/rocketmq/issues/2656#issuecomment-778776488
IMO, it would be better to configure asynchronous log in dledger, so could you submit a PR [dledger repo](https://github.com/openmessaging/openmessaging-storage-dledger)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xxd763795151 edited a comment on issue #2656: DLedger should logging asynchronously
Posted by GitBox <gi...@apache.org>.
xxd763795151 edited a comment on issue #2656:
URL: https://github.com/apache/rocketmq/issues/2656#issuecomment-785829007
yeah! I has submit a pr, but there are some questions in my config. https://github.com/apache/rocketmq/pull/2654
my idea as follow:
```
<appender name="DLedger_inner"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${user.home}/logs/rocketmqlogs/dledger.log</file>
<append>true</append>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/dledger.%i.log.gz</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>10</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>100MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p %t - %m%n</pattern>
<charset class="java.nio.charset.Charset">UTF-8</charset>
</encoder>
</appender>
<appender name="DLedgerAppender" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="DLedger_inner"/>
</appender>
<logger name="io.openmessaging.storage.dledger" additivity="false">
<level value="INFO"/>
<appender-ref ref="DLedgerAppender"/>
</logger>
```
@duhenglucky
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] vongosling closed issue #2656: DLedger should logging asynchronously
Posted by GitBox <gi...@apache.org>.
vongosling closed issue #2656:
URL: https://github.com/apache/rocketmq/issues/2656
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] vongosling commented on issue #2656: DLedger should logging asynchronously
Posted by GitBox <gi...@apache.org>.
vongosling commented on issue #2656:
URL: https://github.com/apache/rocketmq/issues/2656#issuecomment-834300723
I will close this issue and wait for the feedback on dledger repo. https://github.com/openmessaging/dledger/issues
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xxd763795151 commented on issue #2656: DLedger should logging asynchronously
Posted by GitBox <gi...@apache.org>.
xxd763795151 commented on issue #2656:
URL: https://github.com/apache/rocketmq/issues/2656#issuecomment-782036557
> IMO, it would be better to configure asynchronous log in dledger, so could you submit a PR [dledger repo](https://github.com/openmessaging/openmessaging-storage-dledger)?
@RongtongJin Why do not think about modify the default appender of rocketmq to asynchronous if configing dledger log separately is not very reasonable in the logback_broker.xml .
DLedger intergrate into the rocketmq, will use the default appender that config by rocketmq(use logback default) to recored log.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] xxd763795151 commented on issue #2656: DLedger should logging asynchronously
Posted by GitBox <gi...@apache.org>.
xxd763795151 commented on issue #2656:
URL: https://github.com/apache/rocketmq/issues/2656#issuecomment-785829007
yeah! I has submit a pr, but there are some questions in my config. https://github.com/apache/rocketmq/issues/2656#issuecomment-785507603
my idea as follow:
```
<appender name="DLedger_inner"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${user.home}/logs/rocketmqlogs/dledger.log</file>
<append>true</append>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${user.home}/logs/rocketmqlogs/otherdays/dledger.%i.log.gz</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>10</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>100MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyy-MM-dd HH:mm:ss,GMT+8} %p %t - %m%n</pattern>
<charset class="java.nio.charset.Charset">UTF-8</charset>
</encoder>
</appender>
<appender name="DLedgerAppender" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="DLedger_inner"/>
</appender>
<logger name="io.openmessaging.storage.dledger" additivity="false">
<level value="INFO"/>
<appender-ref ref="DLedgerAppender"/>
</logger>
```
@duhenglucky
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org