You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tubemq.apache.org by GitBox <gi...@apache.org> on 2020/06/29 12:16:33 UTC
[GitHub] [incubator-tubemq] hystericalhell opened a new pull request #174: [TUBEMQ-126] Increase the unflushed data bytes control
hystericalhell opened a new pull request #174:
URL: https://github.com/apache/incubator-tubemq/pull/174
Implement the unflushSize as a factor to trigger data flush, and the changes consist of the following 3 parts:
**MsgFileStore**: Make use of curUnflushSize and unflushDataHold settings to trigger;
**Metadata**: Reuse unflushDataHold to store the flush strategy;
Details: [TUBE-126](https://issues.apache.org/jira/browse/TUBEMQ-126)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tubemq] aloyszhang commented on pull request #174: [TUBEMQ-126] Increase the unflushed data bytes control
Posted by GitBox <gi...@apache.org>.
aloyszhang commented on pull request #174:
URL: https://github.com/apache/incubator-tubemq/pull/174#issuecomment-651741298
Thanks for your contribution @hystericalhell , LGTM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tubemq] gosonzhang commented on a change in pull request #174: [TUBEMQ-126] Increase the unflushed data bytes control
Posted by GitBox <gi...@apache.org>.
gosonzhang commented on a change in pull request #174:
URL: https://github.com/apache/incubator-tubemq/pull/174#discussion_r447314196
##########
File path: tubemq-server/src/main/java/org/apache/tubemq/server/broker/metadata/TopicMetadata.java
##########
@@ -37,7 +37,7 @@
private int unflushThreshold = 1000;
// data will be flushed to disk when unflushed message count exceed this.
private int unflushInterval = 10000;
- @Deprecated
+ // data will be flushed to disk when unflushed data size reach this threshold.
private int unflushDataHold = 10000;
Review comment:
The initial value of unflushdatahold is set to 0. By default, this function is turned off. When it is online, it is turned on as needed.
##########
File path: tubemq-server/src/main/java/org/apache/tubemq/server/broker/msgstore/disk/MsgFileStore.java
##########
@@ -169,8 +169,10 @@ public void batchAppendMsg(final StringBuilder sb, final int msgCnt,
}
// check whether need to flush to disk.
long currTime = System.currentTimeMillis();
+ int unflushDataHold = this.messageStore.getTopicMetadata().getUnflushDataHold();
Review comment:
The topicMetadata initialized may not be the latest metadata object. It should be safe to construct the unflushdatahold according to the unflushinterval
##########
File path: tubemq-server/src/main/java/org/apache/tubemq/server/broker/metadata/BrokerDefMetadata.java
##########
@@ -32,7 +32,7 @@
private int numPartitions = 1;
// data will be flushed to disk when unflushed message count exceed this.
private int unflushThreshold = 1000;
- @Deprecated
+ // data will be flushed to disk when unflushed data size reaches the threshold.
private int unflushDataHold = 10000;
Review comment:
The initial value of unflushdatahold is set to 0 to turn off this feature by default.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tubemq] aloyszhang merged pull request #174: [TUBEMQ-126] Increase the unflushed data bytes control
Posted by GitBox <gi...@apache.org>.
aloyszhang merged pull request #174:
URL: https://github.com/apache/incubator-tubemq/pull/174
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org