You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2022/04/15 09:30:22 UTC
[GitHub] [rocketmq-mqtt] pingww opened a new issue, #65: It is necessary to define a subscription storage interface
pingww opened a new issue, #65:
URL: https://github.com/apache/rocketmq-mqtt/issues/65
According to the [MQTT protocol specifications](http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.pdf) , Session stores not only messages but also subscriptions,
which actually is a persist KV. Currently RocketMQ does not have KV persistence, so we need to define an external storage interface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-mqtt] pingww commented on issue #65: It is necessary to define a subscription storage interface
Posted by GitBox <gi...@apache.org>.
pingww commented on issue #65:
URL: https://github.com/apache/rocketmq-mqtt/issues/65#issuecomment-1100993294
>
The subscription structure is about <clientid, subscription> just like a KV Query but must be durable. Topic built-in RocketMQ may be or not realized. Rocksdb is just a single local persistence, the upper layer also needs to realize the global KV query capability, such as sharding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-mqtt] hill007299 commented on issue #65: It is necessary to define a subscription storage interface
Posted by GitBox <gi...@apache.org>.
hill007299 commented on issue #65:
URL: https://github.com/apache/rocketmq-mqtt/issues/65#issuecomment-1100818774
Maybe we could build a Rocksdb into the rocketmq store and enable KV
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-mqtt] pingww commented on issue #65: It is necessary to define a subscription storage interface
Posted by GitBox <gi...@apache.org>.
pingww commented on issue #65:
URL: https://github.com/apache/rocketmq-mqtt/issues/65#issuecomment-1100983979
>
The subscription structure is about <clientid, subscription> just like a KV Query but must be durable. Topic built-in RocketMQ may be or not realized. Rocksdb is just a single local persistence, the upper layer also needs to realize the global KV query capability, such as sharding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-mqtt] pingww commented on issue #65: It is necessary to define a subscription storage interface
Posted by GitBox <gi...@apache.org>.
pingww commented on issue #65:
URL: https://github.com/apache/rocketmq-mqtt/issues/65#issuecomment-1100828184
> Maybe we could build a Rocksdb into the rocketmq store and enable KV
Many scenarios, not just MQTT scenarios, require RocketMQ to have a built-in persistence metadata(kv) storage capability
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-mqtt] azhsmesos commented on issue #65: It is necessary to define a subscription storage interface
Posted by GitBox <gi...@apache.org>.
azhsmesos commented on issue #65:
URL: https://github.com/apache/rocketmq-mqtt/issues/65#issuecomment-1179528716
So the question is should we introduce the rocksDB engine directly to RocketMQ or should we write RocketMQ own kv storage tool
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-mqtt] azhsmesos commented on issue #65: It is necessary to define a subscription storage interface
Posted by GitBox <gi...@apache.org>.
azhsmesos commented on issue #65:
URL: https://github.com/apache/rocketmq-mqtt/issues/65#issuecomment-1179530382
I have an idea that I want to implement it in this way. Referring to the LSM tree model, write data in sequence and build an index structure. The index only stores the offset of the key on the disk, and an extra byte is added after the key to store the offset of the corresponding value on the disk. Consider deleting and compaction operations, such as the garbage debris that results from a remove operation and how to compaction a memory block。 is a simple idea
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-mqtt] YxAc commented on issue #65: It is necessary to define a subscription storage interface
Posted by GitBox <gi...@apache.org>.
YxAc commented on issue #65:
URL: https://github.com/apache/rocketmq-mqtt/issues/65#issuecomment-1100880753
> > Maybe we could build a Rocksdb into the rocketmq store and enable KV
>
> Many scenarios, not just MQTT scenarios, require RocketMQ to have a built-in persistence metadata(kv) storage capability
+1, but,
what is the location of the KV storage here? Is it positioned as local persistence or remote durable storage?
Take the subscription to discuss, maybe the Topic built-in RocketMQ is enough or convenient to use? also because the topic can be a centralized storage compares to Rocksdb.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-mqtt] tianliuliu commented on issue #65: It is necessary to define a subscription storage interface
Posted by GitBox <gi...@apache.org>.
tianliuliu commented on issue #65:
URL: https://github.com/apache/rocketmq-mqtt/issues/65#issuecomment-1101044038
RocketMQ has a built-in persistence metadata(kv) storage capability which is the best way and does not introduce external dependencies
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-mqtt] YxAc commented on issue #65: It is necessary to define a subscription storage interface
Posted by GitBox <gi...@apache.org>.
YxAc commented on issue #65:
URL: https://github.com/apache/rocketmq-mqtt/issues/65#issuecomment-1101014067
yes, the global KV query capability is necessary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-mqtt] pingww closed issue #65: It is necessary to define a subscription storage interface
Posted by GitBox <gi...@apache.org>.
pingww closed issue #65: It is necessary to define a subscription storage interface
URL: https://github.com/apache/rocketmq-mqtt/issues/65
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org