You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pegasus.apache.org by GitBox <gi...@apache.org> on 2021/05/27 03:31:11 UTC
[GitHub] [incubator-pegasus] levy5307 opened a new pull request #738: doc: add rfc about user specified compaction
levy5307 opened a new pull request #738:
URL: https://github.com/apache/incubator-pegasus/pull/738
### What problem does this PR solve? <!--add issue link with summary if exists-->
add rfc about user specified compaction
### Checklist <!--REMOVE the items that are not applicable-->
##### Tests <!-- At least one of them must be included. -->
- No code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] acelyc111 commented on a change in pull request #738: doc: add rfc about user specified compaction
Posted by GitBox <gi...@apache.org>.
acelyc111 commented on a change in pull request #738:
URL: https://github.com/apache/incubator-pegasus/pull/738#discussion_r640305476
##########
File path: rfcs/2021-05-27-user-specified-compaction.md
##########
@@ -0,0 +1,40 @@
+# User Specified Compaction
+
+## Summary
+
+In Pegasus, sometimes we should add user specified compaction policy to reduce disk usage. This RFC proposes a user specified compaction design.
+
+## Design
+
+Add two classes named with `compaction_operation` and `compaction_rule`.
+
+`compaction_filter_rule` represents the compaction rule to filter the keys which are stored in rocksdb.
+There are three types of compaction filter rule:
+- Hashkey rule, which supports prefix match, postfix match and anywhere match.
+2. Sortkey rule. Just like hashkey rule, it also supports prefix match, postfix match and anywhere match.
Review comment:
```suggestion
- Sortkey rule. Just like hashkey rule, it also supports prefix match, postfix match and anywhere match.
```
##########
File path: rfcs/2021-05-27-user-specified-compaction.md
##########
@@ -0,0 +1,40 @@
+# User Specified Compaction
+
+## Summary
+
+In Pegasus, sometimes we should add user specified compaction policy to reduce disk usage. This RFC proposes a user specified compaction design.
+
+## Design
+
+Add two classes named with `compaction_operation` and `compaction_rule`.
+
+`compaction_filter_rule` represents the compaction rule to filter the keys which are stored in rocksdb.
+There are three types of compaction filter rule:
+- Hashkey rule, which supports prefix match, postfix match and anywhere match.
+2. Sortkey rule. Just like hashkey rule, it also supports prefix match, postfix match and anywhere match.
+3. TTL rule. It supports time range match with format [begin_ttl, end_ttl]
+
+`compaction_operation` represents the compaction operation. A compaction operation will be executed when all the corresponding compaction rules are matched.
+There are two types of compaction filter rule:
+1. Delete. It represents that we should delete this key when all the rules are matched.
+2. Update TTL. It represents that we should update TTL when all the rules are matched.
Review comment:
```suggestion
- Delete. It represents that we should delete this key when all the rules are matched.
- Update TTL. It represents that we should update TTL when all the rules are matched.
```
##########
File path: rfcs/2021-05-27-user-specified-compaction.md
##########
@@ -0,0 +1,40 @@
+# User Specified Compaction
+
+## Summary
+
+In Pegasus, sometimes we should add user specified compaction policy to reduce disk usage. This RFC proposes a user specified compaction design.
+
+## Design
+
+Add two classes named with `compaction_operation` and `compaction_rule`.
+
+`compaction_filter_rule` represents the compaction rule to filter the keys which are stored in rocksdb.
+There are three types of compaction filter rule:
+- Hashkey rule, which supports prefix match, postfix match and anywhere match.
+2. Sortkey rule. Just like hashkey rule, it also supports prefix match, postfix match and anywhere match.
+3. TTL rule. It supports time range match with format [begin_ttl, end_ttl]
Review comment:
```suggestion
- TTL rule. It supports time range match with format [begin_ttl, end_ttl]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] acelyc111 commented on a change in pull request #738: doc: add rfc about user specified compaction
Posted by GitBox <gi...@apache.org>.
acelyc111 commented on a change in pull request #738:
URL: https://github.com/apache/incubator-pegasus/pull/738#discussion_r640305209
##########
File path: rfcs/2021-05-27-user-specified-compaction.md
##########
@@ -0,0 +1,40 @@
+# User Specified Compaction
+
+## Summary
+
+In Pegasus, sometimes we should add user specified compaction policy to reduce disk usage. This RFC proposes a user specified compaction design.
+
+## Design
+
+Add two classes named with `compaction_operation` and `compaction_rule`.
+
+`compaction_filter_rule` represents the compaction rule to filter the keys which are stored in rocksdb.
+There are three types of compaction filter rule:
+1. Hashkey rule, which supports prefix match, postfix match and anywhere match.
Review comment:
```suggestion
- Hashkey rule, which supports prefix match, postfix match and anywhere match.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] hycdong merged pull request #738: doc: add rfc about user specified compaction
Posted by GitBox <gi...@apache.org>.
hycdong merged pull request #738:
URL: https://github.com/apache/incubator-pegasus/pull/738
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] hycdong commented on a change in pull request #738: doc: add rfc about user specified compaction
Posted by GitBox <gi...@apache.org>.
hycdong commented on a change in pull request #738:
URL: https://github.com/apache/incubator-pegasus/pull/738#discussion_r640292489
##########
File path: rfcs/2021-05-27-user-specified-compaction.md
##########
@@ -0,0 +1,40 @@
+# User Specified Compaction
+
+## Summary
+
+In pegasus, sometimes we should add user specified compaction policy to reduce disk usage. This RFC proposes a user specified compaction design.
+
+## Design
+
+Add two classes named with `compaction_operation` and `compaction_rule`.
+
+`compaction_filter_rule` represents the compaction rule to filter the keys which are stored in rocksdb.
+There are three types of compaction filter rule:
+1. hashkey rule, which supports prefix match, postfix match and anywhere match.
+2. sortkey rule. Just like hashkey rule, it also supports prefix match, postfix match and anywhere match.
+3. ttl rule. It supports time range match with format [begin_ttl, end_ttl]
+
+`compaction_operation` represents the compaction operation. A compaction operation will be executed when all the corresponding compaction rules are matched.
+There are two types of compaction filter rule:
Review comment:
There are two types of compaction operation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] acelyc111 commented on a change in pull request #738: doc: add rfc about user specified compaction
Posted by GitBox <gi...@apache.org>.
acelyc111 commented on a change in pull request #738:
URL: https://github.com/apache/incubator-pegasus/pull/738#discussion_r640303714
##########
File path: rfcs/2021-05-27-user-specified-compaction.md
##########
@@ -0,0 +1,40 @@
+# User Specified Compaction
+
+## Summary
+
+In pegasus, sometimes we should add user specified compaction policy to reduce disk usage. This RFC proposes a user specified compaction design.
+
+## Design
+
+Add two classes named with `compaction_operation` and `compaction_rule`.
+
+`compaction_filter_rule` represents the compaction rule to filter the keys which are stored in rocksdb.
+There are three types of compaction filter rule:
+1. hashkey rule, which supports prefix match, postfix match and anywhere match.
Review comment:
```suggestion
1. Hashkey rule, which supports prefix match, postfix match and anywhere match.
```
##########
File path: rfcs/2021-05-27-user-specified-compaction.md
##########
@@ -0,0 +1,40 @@
+# User Specified Compaction
+
+## Summary
+
+In pegasus, sometimes we should add user specified compaction policy to reduce disk usage. This RFC proposes a user specified compaction design.
Review comment:
```suggestion
In Pegasus, sometimes we should add user specified compaction policy to reduce disk usage. This RFC proposes a user specified compaction design.
```
##########
File path: rfcs/2021-05-27-user-specified-compaction.md
##########
@@ -0,0 +1,40 @@
+# User Specified Compaction
+
+## Summary
+
+In pegasus, sometimes we should add user specified compaction policy to reduce disk usage. This RFC proposes a user specified compaction design.
+
+## Design
+
+Add two classes named with `compaction_operation` and `compaction_rule`.
+
+`compaction_filter_rule` represents the compaction rule to filter the keys which are stored in rocksdb.
+There are three types of compaction filter rule:
+1. hashkey rule, which supports prefix match, postfix match and anywhere match.
+2. sortkey rule. Just like hashkey rule, it also supports prefix match, postfix match and anywhere match.
+3. ttl rule. It supports time range match with format [begin_ttl, end_ttl]
+
+`compaction_operation` represents the compaction operation. A compaction operation will be executed when all the corresponding compaction rules are matched.
+There are two types of compaction filter rule:
+1. delete. It represents that we should delete this key when all the rules are matched.
Review comment:
```suggestion
1. Delete. It represents that we should delete this key when all the rules are matched.
```
##########
File path: rfcs/2021-05-27-user-specified-compaction.md
##########
@@ -0,0 +1,40 @@
+# User Specified Compaction
+
+## Summary
+
+In pegasus, sometimes we should add user specified compaction policy to reduce disk usage. This RFC proposes a user specified compaction design.
+
+## Design
+
+Add two classes named with `compaction_operation` and `compaction_rule`.
+
+`compaction_filter_rule` represents the compaction rule to filter the keys which are stored in rocksdb.
+There are three types of compaction filter rule:
+1. hashkey rule, which supports prefix match, postfix match and anywhere match.
+2. sortkey rule. Just like hashkey rule, it also supports prefix match, postfix match and anywhere match.
Review comment:
```suggestion
2. Sortkey rule. Just like hashkey rule, it also supports prefix match, postfix match and anywhere match.
```
##########
File path: rfcs/2021-05-27-user-specified-compaction.md
##########
@@ -0,0 +1,40 @@
+# User Specified Compaction
+
+## Summary
+
+In pegasus, sometimes we should add user specified compaction policy to reduce disk usage. This RFC proposes a user specified compaction design.
+
+## Design
+
+Add two classes named with `compaction_operation` and `compaction_rule`.
+
+`compaction_filter_rule` represents the compaction rule to filter the keys which are stored in rocksdb.
+There are three types of compaction filter rule:
+1. hashkey rule, which supports prefix match, postfix match and anywhere match.
+2. sortkey rule. Just like hashkey rule, it also supports prefix match, postfix match and anywhere match.
+3. ttl rule. It supports time range match with format [begin_ttl, end_ttl]
Review comment:
```suggestion
3. TTL rule. It supports time range match with format [begin_ttl, end_ttl]
```
##########
File path: rfcs/2021-05-27-user-specified-compaction.md
##########
@@ -0,0 +1,40 @@
+# User Specified Compaction
+
+## Summary
+
+In pegasus, sometimes we should add user specified compaction policy to reduce disk usage. This RFC proposes a user specified compaction design.
+
+## Design
+
+Add two classes named with `compaction_operation` and `compaction_rule`.
+
+`compaction_filter_rule` represents the compaction rule to filter the keys which are stored in rocksdb.
+There are three types of compaction filter rule:
+1. hashkey rule, which supports prefix match, postfix match and anywhere match.
+2. sortkey rule. Just like hashkey rule, it also supports prefix match, postfix match and anywhere match.
+3. ttl rule. It supports time range match with format [begin_ttl, end_ttl]
+
+`compaction_operation` represents the compaction operation. A compaction operation will be executed when all the corresponding compaction rules are matched.
+There are two types of compaction filter rule:
+1. delete. It represents that we should delete this key when all the rules are matched.
+2. update ttl. It represents that we should update ttl when all the rules are matched.
Review comment:
```suggestion
2. Update TTL. It represents that we should update TTL when all the rules are matched.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org