You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "Mortom123 (via GitHub)" <gi...@apache.org> on 2024/01/29 15:29:24 UTC
[I] Pulsar Manager Persistence in Zookeeper [pulsar-helm-chart]
Mortom123 opened a new issue, #450:
URL: https://github.com/apache/pulsar-helm-chart/issues/450
**Is your feature request related to a problem? Please describe.**
Currently the Pulsar Manager is no persisted outside of the container it is running in.
This means reconditioning of it is necessary should the helm chart be reinstalled. This is not a favorable solution.
**Describe the solution you'd like**
We can either provide a custom jdbc connectionstring to an external storage medium or use the zookeeper to store this data.
I think storing in the zookeeper should be the preferred solution.
The default value in `values.yaml` for the key: `pulsar_manager.configData.URL` (as well as others that need tuning) should be set according to: https://github.com/apache/pulsar-manager#default-test-database-herddb
**Describe alternatives you've considered**
External DB: keeping data in Zookeeper puts Zookeeper as a central point of data and should therefore be favored.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Pulsar Manager Persistence in HerdDB [pulsar-helm-chart]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #450:
URL: https://github.com/apache/pulsar-helm-chart/issues/450#issuecomment-1945696215
#343 has been merged. That provides persistence using a PVC and Postgres Db. I guess that might be sufficient for many use cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Pulsar Manager Persistence in HerdDB [pulsar-helm-chart]
Posted by "Mortom123 (via GitHub)" <gi...@apache.org>.
Mortom123 commented on issue #450:
URL: https://github.com/apache/pulsar-helm-chart/issues/450#issuecomment-1988662411
@eolivelli - I also thought that.
Optimally, I would like to persist all of the data needed to run the pulsar manager in a single point in the cluster. I stumbled upon this setting [here](https://github.com/apache/pulsar-manager/blob/0c87834197cd9d23e360c6b1c5a471f56eabadc3/src/main/resources/application.properties#L43-L44):
```
# HerdDB - start embedded server 'diskless-cluster' mode, WAL and Data persisted on Bookies, Metadata on ZooKeeper in '/herd', listening on localhost:7000
#spring.datasource.url=jdbc:herddb:zookeeper:localhost:2181?server.start=true&server.base.dir=dbdata&server.mode=diskless-cluster&server.node.id=localhost
```
And thought that the environment variable of the Pulsar Manager `SPRING_DATASOURCE_URL` could be set to something like: `jdbc:herddb:zookeeper:zookeeper.service:2181?server.start=true&server.base.dir=dbdata&server.mode=diskless-cluster&server.node.id=(randAlpha10)` to enable this.
This would mean the Pulsar Manager container itself does not need extra volume(-mounts) and data is at a central point in the cluster.
Does this work?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Pulsar Manager Persistence in HerdDB [pulsar-helm-chart]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #450:
URL: https://github.com/apache/pulsar-helm-chart/issues/450#issuecomment-1945523306
I renamed Zookeeper -> HerdDB in the title. @Mortom123 I guess that's what you meant with persistence in Zookeeper? HerdDb uses both Zookeeper and Bookkeeper.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Pulsar Manager Persistence in HerdDB [pulsar-helm-chart]
Posted by "eolivelli (via GitHub)" <gi...@apache.org>.
eolivelli commented on issue #450:
URL: https://github.com/apache/pulsar-helm-chart/issues/450#issuecomment-1975796165
If it is possible to condogurr the jdbc url then storing data on BookKeeper using HerdDB is easy. HerdDB uses Zookeeper the very same way as Pulsar BookKeeper.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org