You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2019/05/24 09:11:04 UTC
Slack digest for #general - 2019-05-24
2019-05-23 13:58:13 UTC - Addison Higham: Hey all... We are starting a pulsar rollout but one of the things we are missing is a ruby implementation of the client, so I started <#CJLPGAZBM|dev-ruby> to start a discussion about the efforts to make that happen
+1 : Karthik Ramasamy, Jerry Peng
----
2019-05-23 14:55:11 UTC - Fredrick P Eisele: I am using the recently released ZK v3.5.5
Does running the `initialize-cluster-metadata` command only make changes to ZK? I have been looking at the information in ZK (local, global) and it all seems to be there. I am just making sure there no changes made to bookies, files, etc.
----
2019-05-23 14:55:57 UTC - Fredrick P Eisele: No particular problem, I am just working on learning how pulsar works.
----
2019-05-23 15:11:21 UTC - Jacob Fugal: @Jacob Fugal has joined the channel
----
2019-05-23 17:35:36 UTC - Devin G. Bost: For those who missed my meetup presentation on Pulsar last night, you can access my slide deck here: <https://www.slideshare.net/DevinBost/apache-pulsar-realtime-data-flows-drive-core-business-processes>
+1 : Matteo Merli, David Kjerrumgaard, Ezequiel Lovelle, Ali Ahmed, Jerry Peng, Karthik Ramasamy, Sijie Guo
----
2019-05-23 17:36:07 UTC - Devin G. Bost: Thanks @Jon Bock for coming out for it!
----
2019-05-23 17:49:56 UTC - Jon Bock: Thanks @Devin G. Bost for taking the time to prepare and present!
----
2019-05-23 18:27:14 UTC - Devin G. Bost: You're welcome!
----
2019-05-23 18:27:33 UTC - Grant Wu: Where are logs written to in the pulsar-standalone docker image?
----
2019-05-23 18:27:40 UTC - Grant Wu: It’s not written to stdout, that’s for sure…
----
2019-05-23 18:28:34 UTC - Grant Wu: My application keeps on logging that HTTP 500s are coming from Pulsar’s websocket endpoit, and I can’t figure out why
----
2019-05-23 18:30:10 UTC - David Kjerrumgaard: `docker logs <container name>`
-1 : Grant Wu
----
2019-05-23 18:30:12 UTC - David Kjerrumgaard: <https://docs.docker.com/engine/reference/commandline/logs/>
----
2019-05-23 18:31:18 UTC - Matteo Merli: @Grant Wu that image is using supervisor to start multiple stuff (for the pulsar dashboard). logs would be on /var/log/supervisor/…
----
2019-05-23 18:32:28 UTC - Grant Wu: Thanks a ton!
----
2019-05-23 18:32:49 UTC - Grant Wu: I was confused because `/pulsar/logs/pulsar-standalone.log` also exists in the image
----
2019-05-23 18:32:51 UTC - Grant Wu: But is empty
----
2019-05-23 18:35:20 UTC - Devin G. Bost: Do we have any plans to make the endpoints more consistent with the APIs? For example, like how some of the CLI commands are plural and some are singular (e.g. namespace*s* vs sink)?
It definitely slows down development because it's hard for our engineers to remember all of the subtle differences.
----
2019-05-23 18:54:22 UTC - Ezequiel Lovelle: If you see some inconsistency through the API you could add an issue on github and I will be pleased to fix it
----
2019-05-23 19:20:20 UTC - Fredrick P Eisele: Is all of the pulsar state information either in zookeeper or bookkeeper? I believe the answer is yes. No problem to report just trying to understand how pulsar works.
----
2019-05-23 19:21:08 UTC - Matteo Merli: Yes, all data is in BK — metadata kept in ZK
----
2019-05-23 19:28:35 UTC - Fredrick P Eisele: There are two ZK clusters used by pulsar, the documentation calls them 'local' and 'global. Is 'local' used by BK and 'global' used by pulsar? or is there some other discriminator?
----
2019-05-23 19:30:37 UTC - Matteo Merli: global is just the “configuration store” for namespace policies, ACLs and so on. For a single cluster deployment the 2 connections string will point to the same ZK cluster. In a geo-replication scenario, the global ZK will be shared across all clusters
----
2019-05-23 19:39:17 UTC - Fredrick P Eisele: Thanks @Matteo Merli . I am working on pulsar source connectors and am wondering where to store state information. E.g. where in the source stream the connector left off, what was the message id of the last message written to a topic, etc.
I was thinking to construct a pulsar-meta-topic but now I am thinking maybe I should use ZK directly.
----
2019-05-23 19:40:21 UTC - Matteo Merli: It’s currently not exposed, though it could make sense to use the same mechanism used for functions state
----
2019-05-23 19:49:54 UTC - Jerry Peng: We can do that, I was actually working on that
----
2019-05-23 22:21:13 UTC - Fredrick P Eisele: When running `./bin/pulsar-admin` I frequently, but not always, get `Reason: javax.ws.rs.ProcessingException: Connection refused: localhost/127.0.0.1:8080` . I suspect there is a timeout setting somewhere that is too short. Any ideas?
----
2019-05-23 22:24:59 UTC - Fredrick P Eisele: Also, `./bin/pulsar-admin brokers list-dynamic-config` returns a list of about 20 items but `./bin/pulsar-admin brokers get-all-dynamic-config` returns `{}`. Should not the same number of items be returned?
----
2019-05-23 22:29:52 UTC - Matteo Merli: the `list-dynamic-config` just tells which are the config keys that are possible to dynamically update at runtime (without brokers restarting)
----
2019-05-23 22:30:16 UTC - Matteo Merli: the other command lists which keys are being overridden
----
2019-05-24 05:03:50 UTC - Emerson Argueta: @Emerson Argueta has joined the channel
----
2019-05-24 08:44:20 UTC - David Tinker: I am trying to start the Pulsar proxy (2.3.1) and am getting Uncaught exception in thread main: Failed to start HTTP server on ports [8082]:
----
2019-05-24 08:44:44 UTC - David Tinker:
----
2019-05-24 08:45:35 UTC - David Tinker: I have only set zookeeperServers= and configurationStoreServers= in proxy.conf. No TLS stuff.
----
2019-05-24 08:46:09 UTC - David Tinker: servicePort=6652 webServicePort=8082
----
2019-05-24 08:54:48 UTC - Sijie Guo: @David Tinker add `numHttpServerThreads=` to your conf/proxy.conf and set it to be larger than 4.
+1 : David Tinker
----
2019-05-24 09:00:01 UTC - Lewey: Hi, we are currently trying to improve the performance of our consumers by multithreading, is there any particular settings that anyone would recommend, currently using r4.2xlarge instance on aws
----
2019-05-24 09:00:35 UTC - Darren Hobbs: @Darren Hobbs has joined the channel
----
2019-05-24 09:10:16 UTC - Sijie Guo: are you using async or sync?
----