You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2019/07/24 09:11:04 UTC
Slack digest for #general - 2019-07-24
2019-07-23 09:12:32 UTC - Sijie Guo: @divyasree I see. I would recommend stopping the function workers (if you co-run function worker with broker, disable it and restart the brokers). Then you can delete those topics and fix the permissions issue. Then re-enable the functions worker.
----
2019-07-23 09:49:56 UTC - divyasree: oh ok..
----
2019-07-23 10:20:48 UTC - divyasree: @Sijie Guo thank you.. working as expected :slightly_smiling_face:
----
2019-07-23 10:21:08 UTC - Sijie Guo: @divyasree cool :+1:
----
2019-07-23 10:21:33 UTC - divyasree: Any updates on the above query?
----
2019-07-23 10:58:32 UTC - Alexandre DUVAL: My 3 nodes bookkeeper cluster got 20GB used on each nodes with quorum 2, while the sum of storageSize of topics is 500MB, there is a way to "sync" bookkeeper?
----
2019-07-23 10:59:45 UTC - Alexandre DUVAL: + my tieredStorage got 3GB used. The storage real infos don't match the topics storages infos.
----
2019-07-23 10:59:54 UTC - Alexandre DUVAL: I'm running v2.4.0
----
2019-07-23 11:00:55 UTC - Lari Hotari: @Lari Hotari has joined the channel
----
2019-07-23 11:09:25 UTC - Tilden: @Tilden has joined the channel
----
2019-07-23 11:13:12 UTC - Alexandre DUVAL: I rolling restarted each bookie and the disk usage have been reduced a bit like 3-4GB per node, but still huge usage compare to topics stats.
----
2019-07-23 11:26:02 UTC - Sijie Guo: @divyasree sorry I missed your question.
so `superUserRole` is the super roles can create tenants and grant permissions to tenant admins.
the `--admin-roles` is used for specifying the admins for a given tenant. the admin roles are able to administrate a tenant (e.g. creating namespace, delete namespace).
the `--role` specified when granting permissions are the client roles. `adminRoles` are able to grant produce/consume/functions performissions to client roles.
Hope this explain
----
2019-07-23 11:38:29 UTC - Yuvaraj Loganathan: may be you are hit by this bug <https://github.com/apache/bookkeeper/pull/2119>
----
2019-07-23 11:41:33 UTC - Alexandre DUVAL: oh thanks @Yuvaraj Loganathan will update my bk with this patch
----
2019-07-23 12:00:41 UTC - Alexandre DUVAL: done, the disk usage is now equilavent on each node :slightly_smiling_face:. higher than i thought, let's wait a bit for bookies to manage it
+1 : Yuvaraj Loganathan
----
2019-07-23 12:29:01 UTC - fguo: @fguo has joined the channel
----
2019-07-23 16:06:22 UTC - Matteo Merli: There’s a setting for how much time to wait before deleting ledgers after they’ve been pushed to tiered storage
----
2019-07-23 18:03:03 UTC - Ryan Samo: Hey guys,
Do you have any recommendations for sizing the JVM per node type in pulsar?
Brokers
Bookies
Zookeepers
Functions Workers
Proxies
I know that having a JVM memory that is too large can have adverse affects, as well as, one that is too small. Just trying to see if there is a common formula or ratio that the community is using/seeing. This pertains to having dedicated hardware with large amounts of available RAM. How much of the physical RAM should I allocate for the JVM and how much should I leave for other work within Pulsar?
----
2019-07-23 18:09:03 UTC - Ali Ahmed: @Ryan Samo This depends on the expected ? Do you know the throughput expected ?
----
2019-07-23 18:09:51 UTC - Ryan Samo: Throughput in messages per second or bytes?
----
2019-07-23 18:10:08 UTC - Ali Ahmed: both
----
2019-07-23 18:11:03 UTC - Ryan Samo: 250k per second
----
2019-07-23 18:11:23 UTC - Ryan Samo: ~100G per hour
----
2019-07-23 18:12:05 UTC - Ryan Samo: Theoretically
----
2019-07-23 18:12:20 UTC - Ali Ahmed: is that max or average ?
----
2019-07-23 18:12:34 UTC - Ryan Samo: Avg
----
2019-07-23 18:17:11 UTC - Ali Ahmed: I would use the configs in the kubernetes spec here ```/deployment/kubernetes```, it’s targeted towards a small to mid size cluster. It should be good starting point for you needs
+1 : Ryan Samo
----
2019-07-23 18:22:08 UTC - Matteo Merli: @Ryan Samo please take a look at configs more geared at high throughput at : <https://github.com/openmessaging/openmessaging-benchmark/tree/master/driver-pulsar/deploy/templates>
----
2019-07-23 18:22:37 UTC - Ryan Samo: @Matteo Merli cool thanks!
----
2019-07-24 05:50:29 UTC - divyasree: ya.. it helped...
----
2019-07-24 05:51:21 UTC - divyasree: For geo replication configuration, u have guided me with <https://gist.github.com/sijie/79364497eaa349bf58d9fb760561f930#2-create-tenants-and-namespace>
----
2019-07-24 05:53:13 UTC - divyasree: i have a query on this.. In the above mentioned link, it was explained as ``` Since there is no global configuration store for all three clusters ``` we are creating clusters on all clusters / we are creating tenants / namespaces in all clusters...
----
2019-07-24 05:54:47 UTC - divyasree: Can you help me on how to setup global configuration store? I assume that, if the global configuration store has been setup, we dont have to create clusters / tenants/ namespace in all the region (DC)... Automatically, it will get replicated.. Correct me if i am wrong...
----
2019-07-24 06:02:04 UTC - Yuvaraj Loganathan: @Alexandre DUVAL did the patch help in disk space reduction ? is the disk usage at the expected level ?
----
2019-07-24 07:53:08 UTC - Ali Ahmed: am starting work on a nix pkg
<https://github.com/NixOS/nixpkgs/pull/65324/files> ,
if anyone has experience please suggest improvements
----
2019-07-24 09:07:53 UTC - Matthieu Nicolas: @Matthieu Nicolas has joined the channel
----