You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2021/12/30 03:43:08 UTC
[GitHub] [rocketmq] Erik1288 opened a new issue #3679: [feature] Support attributes in TopicConfig
Erik1288 opened a new issue #3679:
URL: https://github.com/apache/rocketmq/issues/3679
The issue tracker is **ONLY** used for bug report(feature request need to follow [RIP process](https://github.com/apache/rocketmq/wiki/RocketMQ-Improvement-Proposal)). Keep in mind, please check whether there is an existing same report before your raise a new one.
Alternately (especially if your communication is not a bug report), you can send mail to our [mailing lists](http://rocketmq.apache.org/about/contact/). We welcome any friendly suggestions, bug fixes, collaboration and other improvements.
Please ensure that your bug report is clear and that it is complete. Otherwise, we may be unable to understand it or to reproduce it, either of which would prevent us from fixing the bug. We strongly recommend the report(bug report or feature request) could include some hints as the following:
**FEATURE REQUEST**
1. Please describe the feature you are requesting.
En: As RocketMQ grows, topics will have more traits. Adding a well-designed Attributes sheet can greatly improve TopicConfig's description capabilities.
For example, whether the ConsumeQueue type of a Topic is BatchCQ or SimpleCQ, whether the Deletion strategy of a Topic is Delete or Compact, and even the topi-level preservation time are all attributes of a Topic. Adding TopicConfig variables endlessly is obviously not an elegant approach.
Ch: 随着RocketMQ的发展，现有的TopicConfig在一定程度上已经没有办法表达清楚一个Topic真实的特质。增加一个设计精良的Attributes属性表可以极大程度提高TopicConfig的描述能力。
比如，一个Topic的ConsumeQueue类型是BatchCQ还是SimpleCQ，一个Topic的Deletion策略是Delete还是Compact，甚至Topic级别的保存时间等，都是属于Topic的一个Attribute，无休止得增加TopicConfig变量显然不是一个优雅的做法。
2. Provide any additional detail on your proposed use case for this feature.
* Structure
```
class TopicConfig {
private Other variables;
private Map<String, String> attributes = new HashMap<>();
}
```
* Persistency
topic.json
```
"new_topic":{
"attributes":{
"queue.type":"bcq"
},
"order":false,
"perm":16,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"SELF_TEST_TOPIC",
"topicSysFlag":0,
"writeQueueNums":16
}
```
* Attribute Type Definition
```
class Attributes {
public static final Attribute queueType = new EnumAttribute(
name: "queue.type",
changeable: false,
universe: newHashSet("cq", "bcq")
defaultValue: "cq");
public static final Attribute deletePolicy = new EnumAttribute(
name: "delete.policy",
changeable: true,
universe: newHashSet("delete", "compact")
defaultValue: "delete");
public static final Attribute retentionMs = new LongRangeAttribute(
name: "retention.ms",
changeable: true,
min: 1000,
max: 1000000000,
defaultValue: 10000000);
}
```
* Validation
```
public abstract class Attribute {
protected String name;
protected boolean changeable;
public abstract void verify(String value);
}
```
* Alteration
"-a" option will be added.
* Add attribute
format: +key1=value1,+key2=value2
```
sh bin/mqadmin \
updateTopic \
-w 16 -r 16 \
-n "127.0.0.1:9876" \
-b "127.0.0.1:10911" \
-t new-topic \
-a "+queue.type=batch,+delete.policy=delete,+retention.ms=1111111"
```
* Update attribute
format: +existKey=anotherVal
* Delete attribute
format:-+existKey
* Mixed
format: +existKey1=anotherVal1,-existKey3,+existKey2=anotherVal2
```
sh bin/mqadmin \
updateTopic \
-w 16 -r 16 \
-n "127.0.0.1:9876" \
-b "127.0.0.1:10911" \
-t new-topic \
-a "+existKey1=anotherVal1,-existKey3,+existKey2=anotherVal2"
```
3. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?
Nice-to-have.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Erik1288 commented on issue #3679: [feature] Support attributes in TopicConfig
Posted by GitBox <gi...@apache.org>.
Erik1288 commented on issue #3679:
URL: https://github.com/apache/rocketmq/issues/3679#issuecomment-1005344292
> It is a very valuable suggestion. For the rocktemq multi-replica group model, the leader / master of each replica group is responsible for the management of the authoritative metadata information of the group. In fact, there are two problems to be solved here.
>
> 1. How to modify the metadata of different brokers consistently. For example, for a topic, modify key1=a, but part of the broker key1=b actually?
1. The create-topic/update-topic will exit with errors if the request partially failed.
2. The name server can check topic attributes consistency [not yet implemented]
> 2. Reverse query topic label cannot be performed. This is similar to how to obtain match resources in K8s
The Map<Topic, Map<K, V>> will be added.
Map<K, Set<Topic>> will not be added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] lizhimins commented on issue #3679: [feature] Support attributes in TopicConfig
Posted by GitBox <gi...@apache.org>.
lizhimins commented on issue #3679:
URL: https://github.com/apache/rocketmq/issues/3679#issuecomment-1003287084
It is a very valuable suggestion. For the rocktemq multi-replica group model, the leader / master of each replica group is responsible for the management of the authoritative metadata information of the group. In fact, there are two problems to be solved here.
1. How to modify the metadata of different brokers consistently. For example, for a topic, modify key1=a, but part of the broker key1=b actually?
2. Reverse query topic label cannot be performed. This is similar to how to obtain match resources in K8s
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] RongtongJin closed issue #3679: [feature] Support attributes in TopicConfig
Posted by GitBox <gi...@apache.org>.
RongtongJin closed issue #3679:
URL: https://github.com/apache/rocketmq/issues/3679
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] dongeforever commented on issue #3679: [feature] Support attributes in TopicConfig
Posted by GitBox <gi...@apache.org>.
dongeforever commented on issue #3679:
URL: https://github.com/apache/rocketmq/issues/3679#issuecomment-1002888431
Great.
Pay attention to that the group needs attributes too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org