You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2019/05/10 09:11:04 UTC
Slack digest for #general - 2019-05-10
2019-05-09 09:15:21 UTC - Shivji Kumar Jha: @jia zhai I must mention the low turnaround time in the PRs helps a lot in working with pulsar, a big thank you to the pulsar team :slightly_smiling_face:
beers : jia zhai, Karthik Ramasamy
man-running : Matteo Merli
----
2019-05-09 09:15:58 UTC - Shivji Kumar Jha: bin/pulsar-admin namespaces set-schema-autoupdate-strategy -c none yo/logs
----
2019-05-09 09:16:03 UTC - Shivji Kumar Jha: @Alexandre DUVAL ^
----
2019-05-09 09:16:37 UTC - Alexandre DUVAL: Really ? Lol ok :p
----
2019-05-09 09:16:51 UTC - Shivji Kumar Jha: I authored it so i just remember that one :stuck_out_tongue:
----
2019-05-09 11:10:54 UTC - James Creedy: I think I have 2.3.1 of everything. But something is not aligned correctly. Trying to build C++ client from source now, but other dependancy issues (that initially appear to be Python related). All painful. Any tips on how to remove any of this pain on Mac?
----
2019-05-09 11:47:36 UTC - Brian Doran: Is it possible to run the pulsar standalone with tls enabled?
I have run the following command ensure all ports are available externally:
`docker run --name=pulsar-tls -d -p 6650:6650 -p 8080:8080 -p 8081:8081 -p 6651:6651 -p 8443:8443 -v $PWD/data:/pulsar/data apachepulsar/pulsar:2.3.1 bin/pulsar standalone`
I've edited the standalone.conf file to enable tls:
brokerServicePort=6650
brokerServicePortTls=6651
# Configuration to enable authentication
#authenticationEnabled=true
#authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls
tlsEnabled=true
tlsCertificateFilePath=/certs/broker.cert.pem
tlsKeyFilePath=/certs/broker.key-pk8.pem
tlsTrustCertsFilePath=/certs/ca.cert.pem
tlsAllowInsecureConnection=false
# Port to use to server HTTP request
webServicePort=8080
webServicePortTls=8081`
However it still starts up with TLS disabled.
----
2019-05-09 12:19:18 UTC - Sijie Guo: a) it is possible to run tls on standalone
b)
> However it still starts up with TLS disabled.
which file did you modify?
how did you modify the file in docker?
----
2019-05-09 13:05:07 UTC - Brian Doran: well initially I modified broker.conf and that didn't do it.
So now I have modified broker.conf/standalone.conf and client.conf with tls settings.
`docker cp broker.conf pulsar-tls:/pulsar/conf/broker.conf
docker cp client.conf pulsar-tls:/pulsar/conf/client.conf
docker cp standalone.conf pulsar-tls:/pulsar/conf/standalone.conf'
----
2019-05-09 13:06:02 UTC - Brian Doran: after copying them out via
>docker cp pulsar-tls:/pulsar/conf/broker.conf .
>docker cp pulsar-tls:/pulsar/conf/client.conf .
>docker cp pulsar-tls:/pulsar/conf/standalone.conf
----
2019-05-09 13:12:32 UTC - Brian Doran: I still get `11:44:02.465 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service on port=8080, broker url=<pulsar://030fb47520ee:6650>, cluster=standalone, conf........`
----
2019-05-09 13:24:25 UTC - Brian Doran: @Sijie Guo Does this mean that's it's started up without TLS? I would have expected broker to be `broker url=<pulsar+ssl://030fb47520ee:6651>`
----
2019-05-09 13:32:47 UTC - Brian Doran: Actually .. sorry for wasting your time I changed by URL connect to be pulsar+ssl://<my-pulsar-host>:6651 and I am successfully sending
>2019-05-09 13:29:29.962Z INFO [pulsar-timer-35-1] o.a.p.c.i.ProducerStatsRecorderImpl - [<persistent://public/default/MYTopic>] [<persistent://public/default/TestTopic[Export-Pipeline-Queue-7]>] Pending messages: 17 --- Publish throughput: 4354.03 msg/s --- 68.85 Mbit/s --- Latency: med: 14.692 ms - 95pct: 48.839 ms - 99pct: 135.131 ms - 99.9pct: 10119.376 ms - max: 10195.522 ms --- Ack received rate: 4354.03 ack/s --- Failed messages: 0
>2019-05-09 13:29:29.963Z INFO [pulsar-timer-44-1] o.a.p.c.i.ProducerStatsRecorderImpl - [<persistent://public/default/TestTopic>] [<persistent://public/default/TestTopic[Export-Pipeline-Queue-5]>] Pending messages: 17 --- Publish throughput: 4408.42 msg/s --- 69.71 Mbit/s --- Latency: med: 15.003 ms - 95pct: 52.140 ms - 99pct: 165.113 ms - 99.9pct: 10115.282 ms - max: 10187.674 ms --- Ack received rate: 4408.42 ack/s --- Failed messages: 0
>2019-05-09 13:29:29.964Z INFO [pulsar-timer-38-1] o.a.p.c.i.ProducerStatsRecorderImpl - [<persistent://public/default/TestTopic>] [<persistent://public/default/TestTopic[Export-Pipeline-Queue-12]>] Pending messages: 16 --- Publish throughput: 4331.60 msg/s --- 68.50 Mbit/s --- Latency: med: 15.161 ms - 95pct: 48.321 ms - 99pct: 145.882 ms - 99.9pct: 10179.252 ms - max: 10190.454 ms --- Ack received rate: 4331.60 ack/s --- Failed messages: 0
----
2019-05-09 13:35:45 UTC - Sijie Guo: cool
----
2019-05-09 13:48:10 UTC - Brian Doran: Hi @Sijie Guo Just wonder if there has been any progress on this feature request? Thanks
----
2019-05-09 14:02:13 UTC - Sijie Guo: haven’t started yet unfortunately. I will try to get to the first part tomorrow.
+1 : Brian Doran
----
2019-05-09 14:05:32 UTC - Brian Doran: Thanks @Sijie Guo
----
2019-05-09 14:32:35 UTC - Fredrick P Eisele: I see that the Zookeeper bundled with Pulsar has some differences from the general distribution. I am trying to decide which version to run. The Pulsar documentation on the subject <https://pulsar.apache.org/docs/en/deploy-bare-metal/#deploying-a-zookeeper-cluster> seems ambivalent. What are the arguments either way?
----
2019-05-09 15:00:09 UTC - Marc: @Marc has joined the channel
----
2019-05-09 15:17:07 UTC - Grant Wu: Are you _sure_ you have 2.3.1 of everything?
----
2019-05-09 15:22:56 UTC - Matteo Merli: The only difference we have is that we added metrics on top of regular ZK.
Out of the box, ZK has a very limited and not super-useful set of metrics. We added all the instant rates, latencies, breakdown by request types.
The metrics are exported through Prometheus format.
+1 : Shivji Kumar Jha
----
2019-05-09 15:24:35 UTC - Matteo Merli: Eg. this is the resulting dashboard:
----
2019-05-09 15:24:44 UTC - Matteo Merli:
----
2019-05-09 17:17:12 UTC - Dan Stiner: @Dan Stiner has joined the channel
----
2019-05-09 17:30:37 UTC - Thor Sigurjonsson: Any rough guesstimates as to when we might be seeing version 2.3.2 becoming available?
----
2019-05-09 17:31:37 UTC - Matteo Merli: Waiting on votes for BK 4.9.2 <https://lists.apache.org/thread.html/88c0fcc5959558bb1d678f469a903943b218b83fe4b4fef6ae76f668@%3Cdev.bookkeeper.apache.org%3E>
+1 : Thor Sigurjonsson, Jean-Bernard van Zuylen, Ali Ahmed
----
2019-05-09 19:56:28 UTC - Fredrick P Eisele: I see, thanks. Any effort to get those metrics into upstream ZK?
----
2019-05-09 19:57:26 UTC - Fredrick P Eisele: Is that a Grafana dashboard?
----
2019-05-09 20:17:27 UTC - Matteo Merli: There are other efforts for ZK master 3.6. The stable version is still only in 3.4 though
----
2019-05-09 20:18:00 UTC - Matteo Merli: Even 3.5 is still marked as beta
----
2019-05-09 21:55:57 UTC - Fredrick P Eisele: Are there any issues with running Pulsar with ZK v3.5?
----
2019-05-09 21:56:34 UTC - Fredrick P Eisele: <https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html>
----
2019-05-09 21:59:16 UTC - Matteo Merli: No issues. Just that it’s marked beta
----
2019-05-10 08:26:24 UTC - Kim Christian Gaarder: I have two questions related to partitioned topics in wanting to use them as a transaction log:
1) Do partitioned topics provide consistent read-ordering across consumers/readers, i.e. provide total-ordering?
2) Can partitioned topics be combined with tiered storage?
----
2019-05-10 08:27:28 UTC - Ali Ahmed: @Kim Christian Gaarder
yes for both questions
----
2019-05-10 08:27:44 UTC - Kim Christian Gaarder: Excellent, thank you :slightly_smiling_face:
----