You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2020/06/23 09:11:05 UTC
Slack digest for #general - 2020-06-23
2020-06-22 09:42:13 UTC - jujugrrr: Seems to work @Addison Higham, I had to restart all brokers though in my test though, after a full restore. But it's quite good. I means technically I could lose all bookies and recover from S3 :thumbsup:
----
2020-06-22 09:47:51 UTC - jujugrrr: @Marcio Martins Good morning, have you tried 2.6.0 with s3 ? and OIDC?
----
2020-06-22 09:48:47 UTC - Marcio Martins: Hey, no - I have moved on to other things - got 2.5.1 working stable. Will try 2.6.0 in a couple of weeks.
----
2020-06-22 09:54:45 UTC - jujugrrr: oky, I'll give it a go
----
2020-06-22 09:57:32 UTC - dionjansen: Hi I’m trying to upgrade my v2.5.0 Pulsar cluster on Kubernetes to v2.6.0 but my bookkeeper cluster initialise job is failing with the following error:
```kubectl logs -f job.batch/pulsar-bookie-init
[conf/bookkeeper.conf] Applying config autoRecoveryDaemonEnabled = false
[conf/bookkeeper.conf] Applying config dbStorage_readAheadCacheMaxSizeMb = 32
[conf/bookkeeper.conf] Applying config dbStorage_rocksDB_blockCacheSize = 8388608
[conf/bookkeeper.conf] Applying config dbStorage_rocksDB_writeBufferSizeMB = 8
[conf/bookkeeper.conf] Applying config dbStorage_writeCacheMaxSizeMb = 32
[conf/bookkeeper.conf] Applying config httpServerEnabled = true
[conf/bookkeeper.conf] Applying config httpServerPort = 8000
[conf/bookkeeper.conf] Applying config journalMaxBackups = 0
[conf/bookkeeper.conf] Applying config ledgerDirectories = /pulsar/data/bookkeeper/ledgers
[conf/bookkeeper.conf] Applying config statsProviderClass = org.apache.bookkeeper.stats.prometheus.PrometheusMetricsProvider
[conf/bookkeeper.conf] Applying config useHostNameAsBookieID = true
[conf/bookkeeper.conf] Applying config zkLedgersRootPath = /ledgers
[conf/bookkeeper.conf] Applying config zkServers = pulsar-zookeeper:2181
[conf/bookkeeper.conf] Adding config journalDirectories = /pulsar/data/bookkeeper/journal
JMX enabled by default
Error: Could not find or load main class "
JMX enabled by default
Error: Could not find or load main class "```
Does anybody have any idea what could cause this?
----
2020-06-22 10:06:21 UTC - dionjansen: Not sure if this issue is related: <https://github.com/apache/pulsar/issues/6355> seems like it’s related to JVM env vars
----
2020-06-22 10:17:00 UTC - Madhavan Narayanan: @Madhavan Narayanan has joined the channel
----
2020-06-22 10:27:07 UTC - Marcio Martins: Cool, let me know your findings :slightly_smiling_face:
----
2020-06-22 10:51:26 UTC - Jermaine Oppong: @Jermaine Oppong has joined the channel
----
2020-06-22 13:46:34 UTC - Rahul Vashishth: which pulsar chart should i use among <https://github.com/apache/pulsar-helm-chart> and <https://github.com/streamnative/charts>, while both chart repo says *official helm chart for apache pulsar* :confused:
----
2020-06-22 14:11:22 UTC - Vladimir Shchur: Same issue
<https://github.com/apache/pulsar/issues/7243>
+1 : dionjansen
----
2020-06-22 15:11:54 UTC - Jeff Schneller: Asking again... does anyone have the windows c++ client binaries? Has anyone been able to compile them?
----
2020-06-22 15:52:43 UTC - rwaweber: Hey all! Sorry for pestering, just wanna confirm that this is expected behavior
----
2020-06-22 16:04:49 UTC - Chris Hansen: when this was asked before, the answer was to use the apache one. Both are maintained by the same people.
+1 : Rahul Vashishth, Sijie Guo
----
2020-06-22 16:09:41 UTC - Kevin Barresi: @Kevin Barresi has joined the channel
----
2020-06-22 16:31:23 UTC - Addison Higham: For the same topic? If yes, that would be out of spec for Prometheus
----
2020-06-22 16:31:37 UTC - Addison Higham: Possibly a regression
----
2020-06-22 16:32:14 UTC - Addison Higham: If for different topics then that is expected as it gives consumer per topic
----
2020-06-22 17:03:02 UTC - Devin G. Bost: Has anyone tried using the Debezium connector to connect to Oracle?
----
2020-06-22 18:01:36 UTC - rwaweber: Oh sorry — I meant mostly with respect to the metric format itself being in spec for prometheus, regardless of whether its for a topic or not
E.g. in this metric
```pulsar_consumers_count{cluster="pulsar_bdx",namespace="ns/global",topic="<persistent://ns/global/queue>"} 1.0 1558518241859```
1558518241859 isn’t going to be captured by prometheus, so I’m mostly curious as to what it could represent, and whether its supposed to be there at all
----
2020-06-22 18:25:19 UTC - Rahul Vashishth: @Sijie Guo is it fair to say that streamNative charts are ahead from apache charts. how these two are related in terms of development and release cadence.
----
2020-06-22 19:46:16 UTC - Joshua Decosta: @Joshua Decosta has joined the channel
----
2020-06-22 22:50:06 UTC - Addison Higham: @rwaweber, that is part of the prometheus spec, an optional timestamp as milliseconds since epoch after the value
----
2020-06-22 22:50:31 UTC - Addison Higham: <https://prometheus.io/docs/concepts/data_model/#samples> doesn't provide an example, but does mention it
----
2020-06-22 22:52:12 UTC - jujugrrr: Seems to work well :thumbsup:
----
2020-06-22 22:52:18 UTC - Addison Higham: oh this is the better doc that describes it :slightly_smiling_face:
----
2020-06-22 23:10:57 UTC - Addison Higham: <https://prometheus.io/docs/instrumenting/exposition_formats/>
----
2020-06-22 23:16:36 UTC - rwaweber: Oh cool! I had no idea, thanks for the links! I’ve got some brushing up to do it seems :smile:
Really appreciate it!
----
2020-06-23 02:37:32 UTC - sjmittal: Even I had got same issues
----
2020-06-23 02:46:15 UTC - sjmittal: Folks can anyone tell me difference between
<https://hub.docker.com/r/apachepulsar/pulsar-all>
and
<https://hub.docker.com/r/apachepulsar/pulsar>
docker images
----
2020-06-23 06:07:32 UTC - Curtis Cook: > If you are <https://pulsar.apache.org/docs/en/standalone-docker|running Pulsar in Docker> or deploying Pulsar using a docker image (e.g. <https://pulsar.apache.org/docs/en/deploy-kubernetes|K8S> or <https://pulsar.apache.org/docs/en/deploy-dcos|DCOS>), you can use the `apachepulsar/pulsar-all` image instead of the `apachepulsar/pulsar` image. `apachepulsar/pulsar-all` image has already bundled <https://pulsar.apache.org/docs/en/io-overview#working-with-connectors|all builtin connectors>.
----
2020-06-23 06:07:37 UTC - Curtis Cook: <https://pulsar.apache.org/docs/en/standalone/>
----
2020-06-23 08:21:35 UTC - Marcio Martins: Thanks! That makes me hopeful :smile:
----
2020-06-23 08:21:43 UTC - jujugrrr: :wink:
----
2020-06-23 08:22:12 UTC - Marcio Martins: Was the upgrade smooth or did you try from scratch?
----
2020-06-23 08:22:55 UTC - jujugrrr: couldn't find the exact permissions required for S3 on the documentation. Do you just give more access than required right now?
----
2020-06-23 08:23:42 UTC - jujugrrr: I gave :
``` "s3:CompleteMultipartUpload",
"s3:CreateMultipartUpload",
"s3:DeleteObjects",
"s3:GetObject",
"s3:PutObject",
"s3:UploadPart"```
----
2020-06-23 08:24:27 UTC - jujugrrr: on the objects and:
``` "s3:ListBucket",
"s3:GetBucketLocation"```
On the bucket. That's all I could see from S3 logs
----
2020-06-23 08:24:54 UTC - Marcio Martins: Yeah, I have full access at the moment
----
2020-06-23 08:25:01 UTC - jujugrrr: @Addison Higham Do you know if it's documented somewhere?
----
2020-06-23 08:25:19 UTC - jujugrrr: > Yeah, I have full access at the moment
Oky
----
2020-06-23 08:25:31 UTC - Marcio Martins: Nop, couldn't find it anywhere either... I was meant to investigate that, but forgot about after the OIDC thing not working...
----
2020-06-23 08:26:39 UTC - jujugrrr: cool, let me know if you get back and find missing permissions
----
2020-06-23 08:29:44 UTC - Marcio Martins: sure! thanks
----
2020-06-23 09:10:15 UTC - Galen Suen: @Galen Suen has joined the channel
----