You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by lizhanhui <gi...@git.apache.org> on 2017/08/07 08:22:43 UTC
[GitHub] incubator-rocketmq pull request #143: [ROCKETMQ-256] Clean commit log files ...
GitHub user lizhanhui opened a pull request:
https://github.com/apache/incubator-rocketmq/pull/143
[ROCKETMQ-256] Clean commit log files manually to specified disk usage watermark
Associated JIRA ticket is: https://issues.apache.org/jira/browse/ROCKETMQ-256
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lizhanhui/incubator-rocketmq store_clean
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-rocketmq/pull/143.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #143
----
commit b195d89905ce2ff3c9e24b5c676ebc75ab0537e7
Author: Li Zhanhui <li...@gmail.com>
Date: 2017-08-01T12:57:00Z
develop
commit 9b187f337273ad3f93ee2d923ab55e1cdd46cc9e
Author: Li Zhanhui <li...@gmail.com>
Date: 2017-08-04T05:54:55Z
Add command to clean commit log files to specified watermark
commit 56a1c3587028def12525dd0daed8dd28e128dd64
Author: Li Zhanhui <li...@gmail.com>
Date: 2017-08-07T06:36:09Z
Delete commit log files to meet disk usage goals specified in command line
commit b9d0fafbc7f7e19eeba28a7e4a43c8055587dc41
Author: Li Zhanhui <li...@gmail.com>
Date: 2017-08-07T07:16:05Z
Log detailed activity for manually purging commit log files
commit e6d323b81866386360708ea515734ccdd8cdd72c
Author: Li Zhanhui <li...@gmail.com>
Date: 2017-08-07T07:19:39Z
Fix method throws signature
commit c967102209c27f268fb1c058263c909474d46b00
Author: Li Zhanhui <li...@gmail.com>
Date: 2017-08-07T07:52:45Z
Implement the command
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq pull request #143: [ROCKETMQ-256] Clean commit log files ...
Posted by dongeforever <gi...@git.apache.org>.
Github user dongeforever commented on a diff in the pull request:
https://github.com/apache/incubator-rocketmq/pull/143#discussion_r140139119
--- Diff: broker/src/main/java/org/apache/rocketmq/broker/plugin/AbstractPluginMessageStore.java ---
@@ -246,4 +247,14 @@ public void setConfirmOffset(long phyOffset) {
public ConsumeQueue getConsumeQueue(String topic, int queueId) {
return next.getConsumeQueue(topic, queueId);
}
+
+ @Override
+ public PutMessageResult putMessages(MessageExtBatch messageExtBatch) {
+ return null;
+ }
--- End diff --
why return null but not next.putMessages(messageExtBatch)
---
[GitHub] incubator-rocketmq pull request #143: [ROCKETMQ-256] Clean commit log files ...
Posted by dongeforever <gi...@git.apache.org>.
Github user dongeforever commented on a diff in the pull request:
https://github.com/apache/incubator-rocketmq/pull/143#discussion_r140139906
--- Diff: store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java ---
@@ -1392,6 +1397,14 @@ public void run() {
}, 6, TimeUnit.SECONDS);
}
+ @Override
+ public void purge(int watermark, long consumedPhysicalOffset, boolean force) {
+ cleanCommitLogService.setPurgeForcefullyWhenManual(force);
+ cleanCommitLogService.setDiskSpaceManuallyCleanRatio(watermark / 100.0);
+ cleanCommitLogService.setConsumedPhysicalOffset(consumedPhysicalOffset);
+ cleanCommitLogService.executeDeleteFilesManually();
--- End diff --
This is not thread safe.
How about just delivering the arguments into executeDeleteFilesManually() directly?
---
[GitHub] incubator-rocketmq issue #143: [ROCKETMQ-256] Clean commit log files manuall...
Posted by lizhanhui <gi...@git.apache.org>.
Github user lizhanhui commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/143
@zhouxinyu @dongeforever please help review this PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq issue #143: [ROCKETMQ-256] Clean commit log files manuall...
Posted by dongeforever <gi...@git.apache.org>.
Github user dongeforever commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/143
@lizhanhui This purge is not be safe for scheduled message.
---
[GitHub] incubator-rocketmq issue #143: [ROCKETMQ-256] Clean commit log files manuall...
Posted by lizhanhui <gi...@git.apache.org>.
Github user lizhanhui commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/143
1. "As force delete may cause message lost,how about add a warnning in command help?"--->Added.
2. "And also add a monitor disk usage tool command for check." --> I do not follow this comment, could you please articulate it more specifically?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq pull request #143: [ROCKETMQ-256] Clean commit log files ...
Posted by dongeforever <gi...@git.apache.org>.
Github user dongeforever commented on a diff in the pull request:
https://github.com/apache/incubator-rocketmq/pull/143#discussion_r140139302
--- Diff: common/src/test/java/org/apache/rocketmq/common/UtilAllTest.java ---
@@ -89,6 +89,14 @@ public void testGetDiskPartitionSpaceUsedPercent() {
assertThat(UtilAll.getDiskPartitionSpaceUsedPercent(tmpDir)).isNotCloseTo(-1, within(0.000001));
}
+
+ @Test
+ public void testGetDiskPartitionSpaceUsedPercent4Home() {
+ String tmpDir = System.getProperty("user.home");
+ double percent = UtilAll.getDiskPartitionSpaceUsedPercent(tmpDir);
+ System.out.println(percent);
--- End diff --
What did this unit test mean for?
---
[GitHub] incubator-rocketmq issue #143: [ROCKETMQ-256] Clean commit log files manuall...
Posted by coveralls <gi...@git.apache.org>.
Github user coveralls commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/143
[](https://coveralls.io/builds/12774632)
Coverage increased (+0.03%) to 38.772% when pulling **e767801526ef804a4945270720bba59a490bfc03 on lizhanhui:store_clean** into **98bd032454c9dd01bcaea0f4c92abfa0b1847bac on apache:develop**.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq issue #143: [ROCKETMQ-256] Clean commit log files manuall...
Posted by lindzh <gi...@git.apache.org>.
Github user lindzh commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/143
Thanks for contribution. As force delete may cause message lost,how about add a warnning in command help? And also add a monitor disk usage tool command for check.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq issue #143: [ROCKETMQ-256] Clean commit log files manuall...
Posted by coveralls <gi...@git.apache.org>.
Github user coveralls commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/143
[](https://coveralls.io/builds/12719541)
Coverage decreased (-0.1%) to 38.612% when pulling **c967102209c27f268fb1c058263c909474d46b00 on lizhanhui:store_clean** into **98bd032454c9dd01bcaea0f4c92abfa0b1847bac on apache:develop**.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---