You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2019/10/24 09:11:05 UTC
Slack digest for #general - 2019-10-24
2019-10-23 09:12:17 UTC - Retardust: is there any "batched consumer"
I want to consume N messages, parse them in parallel and then send in order of source topic. Should I write that by myself or there is some pulsar java api tricks?
----
2019-10-23 09:38:00 UTC - wangliqin: @wangliqin has joined the channel
----
2019-10-23 09:45:40 UTC - Vladimir Shchur: @Chris Bartholomew we are also interested in infinite retention support (possibly with S3 offload), will it be possible?
----
2019-10-23 09:56:40 UTC - Chris Bartholomew: @Vladimir Shchur Yes, S3 offload will also be supported for infinite retention.
----
2019-10-23 09:57:10 UTC - Vladimir Shchur: Sounds great
----
2019-10-23 10:16:30 UTC - Alexandre DUVAL: No there is no error showed on broker side.
----
2019-10-23 10:16:36 UTC - Alexandre DUVAL: @xiaolong.ran
----
2019-10-23 10:39:24 UTC - Retardust: does acknowledgmentGroupTime affect on acknowledgeCumulative?
or I still need to send last messageId consumed by myself?
----
2019-10-23 11:00:28 UTC - Gopi Krishna L: @Gopi Krishna L has joined the channel
----
2019-10-23 11:04:21 UTC - Gopi Krishna L: Hi, I am new to Apache Pulsar. Can someone guide me through the steps that have to followed after running "bin/pulsar standalone" or any blog for the initial steps
----
2019-10-23 11:43:55 UTC - Sijie Guo: Oh the default replicas is 2. So when you scaled down to 3, you are losing 2 nodes. The data is lost
----
2019-10-23 11:45:03 UTC - Sijie Guo: If you are using persistent disks, you need to check if the disks are still there
----
2019-10-23 11:53:18 UTC - Alexandre DUVAL: (This occurs on function worker v2.4.0.)
----
2019-10-23 12:01:32 UTC - Nicolas Ha: I thought it would be fine as I went from 3 nodes to 5 nodes to 3 nodes again. Using the daemonset (I think, on my phone right now). So I should try to increase the number of replicas? Loosing data right now is not an issue, but having the nodes not starting up again is a problem
----
2019-10-23 12:07:21 UTC - xiaolong.ran: In the `apache-pulsar-2.4.0/logs/functions` directory, there is specific log information about the function. You can check if there is useful log information inside to facilitate further positioning.
----
2019-10-23 12:14:28 UTC - xiaolong.ran: Or can you further describe the problem you are experiencing, and according to the current information, I have no way to locate the problem. @Alexandre DUVAL
----
2019-10-23 12:18:23 UTC - xiaolong.ran: Welcome, here is the official website of pulsar: <https://pulsar.apache.org/en/>
Here is how to run pulsar in the form of standalone
<https://pulsar.apache.org/docs/en/standalone/>
----
2019-10-23 12:19:04 UTC - xiaolong.ran: What are your problems during use? Please feel free to communicate.
----
2019-10-23 12:21:51 UTC - Sergey Zhemzhitsky: @Sijie Guo Thanks for such a great explanation and answer. You know, I’m just asking because for me Pulsar is a better Kafka with additional great features, but the last one has greater adoption (at least for now), so I’d like Pulsar to have higher market-share. E.g. Twitter has also previously had a opensourced distributed log -like solution - Apache DistributedLog, that was rejected in favour of Kafka (<https://blog.twitter.com/engineering/en_us/topics/insights/2018/twitters-kafka-adoption-story.html>).
Also I’m worried about Splunk, that creates a great products, but has a tendency to stop their business in certain countries (e.g. Russia) in just a single day without even letting know about that to its partners and resellers there (<https://www.zdnet.com/article/splunk-pulls-out-of-russia-with-mysterious-statement/>).
In any way - congrats to the whole streamlio team!
----
2019-10-23 12:27:17 UTC - Gopi Krishna L: Hi, so when after running the command , I am getting the error
----
2019-10-23 12:28:07 UTC - xiaolong.ran: Please send us the error message you encountered.
----
2019-10-23 12:28:13 UTC - Retardust: what does " Subscription is fenced " error from broker means?
----
2019-10-23 12:28:44 UTC - Gopi Krishna L: Error after running
"bin/pulsar standalone"
os: linux
----
2019-10-23 12:29:04 UTC - Retardust: We in Russia hope that it will not affect Pulsar:)
----
2019-10-23 12:29:09 UTC - Joose Helle: @Matteo Merli Thanks for explaining!
----
2019-10-23 12:30:48 UTC - xiaolong.ran: What type of subscription are you using?
----
2019-10-23 12:31:40 UTC - xiaolong.ran: If you are using the `Exclusive ` subscription model, only one consumer subscription is allowed for each topic.
----
2019-10-23 12:33:11 UTC - xiaolong.ran: You can try using the following command:
```
./bin/pulsar standalone -nfw -nss
```
----
2019-10-23 12:33:43 UTC - Gopi Krishna L: what does -nfw -nss specify ?
----
2019-10-23 12:34:27 UTC - xiaolong.ran:
----
2019-10-23 12:34:57 UTC - Gopi Krishna L: thanks a lot for the help @xiaolong.ran
----
2019-10-23 12:35:21 UTC - xiaolong.ran: You’re welcome
----
2019-10-23 12:36:29 UTC - Retardust: Failover
----
2019-10-23 12:36:41 UTC - Retardust: should I use constant consumer name?
----
2019-10-23 12:36:53 UTC - Retardust: between restarts
----
2019-10-23 12:40:44 UTC - xiaolong.ran: yes, you should use same subscription name.
----
2019-10-23 12:41:01 UTC - xiaolong.ran:
----
2019-10-23 12:42:30 UTC - xiaolong.ran: As shown in the figure above, in the subscription mode of `failover`, we need to use the same subscription name to subscribe to the topic for different consumers.
----
2019-10-23 12:42:47 UTC - Retardust: not subscription name, consumer name.
I have constant subscription name
----
2019-10-23 12:43:33 UTC - xiaolong.ran: OK, you should use a different consumer name
----
2019-10-23 12:43:50 UTC - Retardust: it's UUID.random)
----
2019-10-23 12:44:46 UTC - Retardust: sorry, i didn't set any. which is default?
----
2019-10-23 12:45:48 UTC - xiaolong.ran: default value is `Exclusive`
----
2019-10-23 12:46:02 UTC - Retardust: I mean default for consumer name
----
2019-10-23 12:46:59 UTC - Retardust: @xiaolong.ran also maybe you have answer too?)
----
2019-10-23 12:48:30 UTC - Gautam Singh: Hi All , Has anyone been able to get CDC for MySQL up and running, using the following tutorial <https://pulsar.apache.org/docs/en/io-cdc-debezium/#usage> . I keep getting connection refused error, even after following the tutorial exactly. Any help is appreciated
----
2019-10-23 12:51:08 UTC - Gopi Krishna L: @xiaolong.ran I am still facing the issue, pls check the file attached.
----
2019-10-23 12:52:32 UTC - xiaolong.ran: ```
this.consumerName = conf.getConsumerName() == null ? ConsumerName.generateRandomName() : conf.getConsumerName();
```
----
2019-10-23 12:52:58 UTC - Retardust: so I'm still don't know why "fenced" :disappointed:
----
2019-10-23 12:54:17 UTC - Gopi Krishna L: And then it just goes on repeating the following
----
2019-10-23 12:54:57 UTC - Alexandre DUVAL: Sure, I added logs in my function this is how I saw that context.publish stucked.
----
2019-10-23 12:56:07 UTC - Alexandre DUVAL: then I used jstack
----
2019-10-23 12:56:42 UTC - Alexandre DUVAL: But nothing more. The function logs well and then stuck, the last log output is the line before the context.publish.
----
2019-10-23 12:58:21 UTC - Alexandre DUVAL: More, the function continues to output consomption/produce metrics, just the process stucks.
----
2019-10-23 12:58:47 UTC - Alexandre DUVAL: About describe more. I can share you the code in DM.
----
2019-10-23 13:03:31 UTC - xiaolong.ran: Can you describe your operation?
----
2019-10-23 13:06:04 UTC - xiaolong.ran: ```
./bin/pulsar standalone -nfw -nss -a 127.0.0.1
```
----
2019-10-23 13:11:00 UTC - xiaolong.ran: Hello, can you use
```
<O> TypedMessageBuilder<O> newOutputMessage(String topicName, Schema<O> schema)
```
instead of `publish`
----
2019-10-23 13:12:37 UTC - xiaolong.ran: See if you will encounter the same problem
----
2019-10-23 13:20:04 UTC - xiaolong.ran: If you are using `Exclusive` or `Failover` subscription mode, it will not affect
----
2019-10-23 13:25:10 UTC - Nicolas Ha: But thanks that was useful - as usual :grin:
----
2019-10-23 13:29:04 UTC - Sijie Guo: @jia zhai @xiaolong.ran ^
----
2019-10-23 13:31:04 UTC - jia zhai: @Gautam Singh Would you please post the error logs?
----
2019-10-23 13:31:22 UTC - jia zhai: which version are you using?
----
2019-10-23 13:35:26 UTC - Gautam Singh: I am using pulsar 2.4.1 . Will post the logs in sometime
----
2019-10-23 13:36:07 UTC - Naby: I am trying to build pulsar python client from source for Raspberry Pi, but, I can not build c++ client first. I got Errors during `cmake .` call. Here is what I got:
----
2019-10-23 13:38:16 UTC - xiaolong.ran: <https://pulsar.apache.org/docs/en/develop-cpp/>
----
2019-10-23 13:39:10 UTC - xiaolong.ran: Maybe you can use this tutorial to build your cpp client locally.
----
2019-10-23 13:40:15 UTC - Naby: That’s exactly what I am trying to do. I am using this tutorial.
----
2019-10-23 13:42:15 UTC - xiaolong.ran: ```
cmake . -DBUILD_TESTS=OFF
```
----
2019-10-23 13:43:14 UTC - xiaolong.ran: You can disable the test and try to build again.
----
2019-10-23 13:45:06 UTC - Naby: It is a default setting as you can see in the first line. I have tried it though, but, it does not work.
----
2019-10-23 13:46:42 UTC - xiaolong.ran: you can use `./docker-build.sh` in pulsar-client-cpp
----
2019-10-23 13:47:47 UTC - Naby: How? Instead of cmake and make commands?
----
2019-10-23 13:49:21 UTC - xiaolong.ran: You can execute `./docker-build.sh` in the `pulsar-client-cpp` directory. Build using docker
----
2019-10-23 13:49:39 UTC - xiaolong.ran: yes
----
2019-10-23 13:50:12 UTC - Naby: Okay. I’ll try to see if it works for me. Thanks.
----
2019-10-23 14:45:08 UTC - Retardust: two topics. get messages from first, parse and send result to the next one. with async acking
----
2019-10-23 14:47:08 UTC - Retardust: thanks. is it safe to acknowledgeCumulative once per 100ms only the last messageId?
----
2019-10-23 14:56:53 UTC - xiaolong.ran: sure
+1 : Retardust
----
2019-10-23 14:58:00 UTC - Retardust: Im wondering why "acknowledgeWindow" doesn't handle that by itself?
----
2019-10-23 15:44:32 UTC - David Kjerrumgaard: You can either add some simple counter logic , or a fixed sized data structure to your existing code. Or you could use a tumbling windowing pulsar function. <https://streaml.io/blog/eda-event-processing-design-patterns-with-pulsar-functions>
----
2019-10-23 16:12:58 UTC - Addison Higham: be off slack for one day and stuff like this happens! congrats guys!
----
2019-10-23 17:04:13 UTC - Sijie Guo: @Retardust Pulsar will not be affected :slightly_smiling_face:
----
2019-10-23 18:16:49 UTC - Peter Huang: @Peter Huang has joined the channel
----
2019-10-23 18:19:53 UTC - Raman Gupta: I think I'm not understanding something about consumers+subscriptions. I have a shared subscription x on topic y. The backlog for the subscription was 0. I moved the code that processes the subscription to another process, with the same subscription name (but different consumer name). My understanding was that the backlog should remain at 0 elements in this case. However, it appears everything that was already acked in that topic is ~not~ now in the backlog again for the subscription. What am I missing here?
----
2019-10-23 18:26:39 UTC - Nicolas Ha: wait - can a `broker` fail to recover a ledger? I thought ledgers were on `bookies`??
Also one of the brokers is fine (from it’s state and from the logs), but as a whole the service fails to reply (even though the proxy state is “Running”)
----
2019-10-23 18:29:30 UTC - David Kjerrumgaard: @Raman Gupta In a shared subscription, when a consumer disconnects, all the messages that were sent to it and not acknowledged will be rescheduled for sending to the remaining consumers. So It appears that the messages may not have been acked by the first consumer process.
----
2019-10-23 18:29:48 UTC - Raman Gupta: How could the backlog be 0 if they were not acked?
----
2019-10-23 18:30:25 UTC - Nicolas Ha: So actually restarting the brokers worked … should have tried the old turn on/off earlier!
EDIT: actually - back to `CrashLoopBackOff` :confused:
----
2019-10-23 18:31:02 UTC - Raman Gupta: Plus I restarted that consumer many times without this happening. Its only when I moved the subscription code that the "zombies" came back.
----
2019-10-23 18:31:20 UTC - Raman Gupta: I'm trying to create a repro right now to get more insight
----
2019-10-23 18:33:32 UTC - David Kjerrumgaard: Where were you seeing the zero backlog? From the pulsar-admin stats?
----
2019-10-23 18:33:38 UTC - Raman Gupta: Yeah
----
2019-10-23 18:34:35 UTC - David Kjerrumgaard: are you using async acknowledgement on the consumer?
----
2019-10-23 18:36:49 UTC - David Kjerrumgaard: This sounds like a bug, would be interested in seeing if you can reproduce the issue. If so, please file an issue on Github.
----
2019-10-23 18:44:58 UTC - Raman Gupta: Yes it is async ack, but I'm not talking seconds here -- some of the zombies have been acked for hours or days
+1 : David Kjerrumgaard
----
2019-10-23 18:57:28 UTC - Nicolas Ha: ```
If you are using persistent disks, you need to check if the disks are still there
```
How do I check that brokers use/don’t use disks? I was not aware there could be disks for the brokers - I am using the same as <https://github.com/apache/pulsar/blob/master/deployment/kubernetes/generic/k8s-1-9-and-above/broker.yaml> just with more memory
----
2019-10-23 19:07:45 UTC - Jon Bock: As an Apache project, Apache Pulsar and the Apache Pulsar community will continue moving forward without change.
----
2019-10-23 19:09:24 UTC - Jon Bock: There are currently commercial offerings based on Pulsar from multiple companies, and that ecosystem can be expected to grow and evolve over time also.
----
2019-10-23 19:34:03 UTC - Kendall Magesh-Davis: Anyone know a good way to increase pulsar-admin verbosity? I’m getting a 401 on authenticating a particular Tenant JWT but the most detail I’m finding in logs is
```HTTP 401 Unauthorized
Reason: HTTP 401 Unauthorized```
The token validates as a `{sub=thisTenantsAdmin}` as expected, so I figure it must be in the client config but can’t get a better error.
----
2019-10-23 19:54:18 UTC - Nicolas Ha: kubectl describe deployments.apps broker
I get
```
Name: broker
Namespace: default
CreationTimestamp: Wed, 23 Oct 2019 19:44:48 +0100
Labels: app=pulsar
component=broker
Annotations: <http://deployment.kubernetes.io/revision|deployment.kubernetes.io/revision>: 2
<http://kubectl.kubernetes.io/last-applied-configuration|kubectl.kubernetes.io/last-applied-configuration>:
{"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"name":"broker","namespace":"default"},"spec":{"replicas":3,"sele...
Selector: app=pulsar,component=broker
Replicas: 3 desired | 3 updated | 3 total | 1 available | 2 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=pulsar
component=broker
Annotations: <http://prometheus.io/port|prometheus.io/port>: 8080
<http://prometheus.io/scrape|prometheus.io/scrape>: true
Containers:
broker:
Image: apachepulsar/pulsar-all:2.4.1
Ports: 8080/TCP, 6650/TCP
Host Ports: 0/TCP, 0/TCP
Command:
sh
-c
Args:
bin/apply-config-from-env.py conf/broker.conf && bin/apply-config-from-env.py conf/pulsar_env.sh && bin/gen-yml-from-env.py conf/functions_worker.yml && bin/pulsar broker
Environment Variables from:
broker-config ConfigMap Optional: false
Environment:
advertisedAddress: (v1:status.podIP)
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Progressing True NewReplicaSetAvailable
Available False MinimumReplicasUnavailable
OldReplicaSets: <none>
NewReplicaSet: broker-6dc6bd464 (3/3 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 57m deployment-controller Scaled up replica set broker-6dc6bd464 to 1
Normal ScalingReplicaSet 57m deployment-controller Scaled down replica set broker-6995b9c76 to 2
Normal ScalingReplicaSet 57m deployment-controller Scaled up replica set broker-6dc6bd464 to 2
Normal ScalingReplicaSet 57m deployment-controller Scaled down replica set broker-6995b9c76 to 1
Normal ScalingReplicaSet 57m deployment-controller Scaled up replica set broker-6dc6bd464 to 3
Normal ScalingReplicaSet 57m deployment-controller Scaled down replica set broker-6995b9c76 to 0
```
so, no volumes there it seems
----
2019-10-23 19:57:01 UTC - Nicolas Ha: also `kubectl get pv` and `kubectl get pvc` return no volumes.
----
2019-10-23 20:05:04 UTC - Raman Gupta: So weird, I'm unable to reproduce the issue.
thinking_face : David Kjerrumgaard
----
2019-10-23 20:09:18 UTC - Raman Gupta: Subscriptions don't expire right?
----
2019-10-23 20:11:24 UTC - Addison Higham: they can, there is this (not very well documented) configuration option:
```
@FieldContext(
category = CATEGORY_POLICIES,
doc = "How long to delete inactive subscriptions from last consuming."
+ " When it is 0, inactive subscriptions are not deleted automatically"
)
private long subscriptionExpirationTimeMinutes = 0;
```
----
2019-10-23 20:11:33 UTC - Addison Higham: but the default is 0, which is never to expire subscriptions
----
2019-10-23 20:15:41 UTC - Naby: I did not try docker-build.sh, but I was able to fix my problem. I had to make clang as my default compiler and the rest is explained here: <https://github.com/apache/pulsar/issues/4096>
----
2019-10-23 20:35:33 UTC - Raman Gupta: Is there a way to see a "history" of a message id i.e. when it was acked by what subscriptions?
----
2019-10-23 20:44:21 UTC - Raman Gupta: Ok, figured it out -- problem was on the producer side -- due to a bug, the same messages got produced again. My bad.
----
2019-10-23 21:07:14 UTC - Raman Gupta: On a related topic, is there a way to "copy" the state of a subscription from one subscription to another? Analogous to copying Kafka offsets from one consumer group to another?
----
2019-10-23 21:38:52 UTC - Kendall Magesh-Davis: Here’s my steps to reproduce that weird Unauthorized Tenant Admin problem I mentioned above:
Create new tenant `pulsarbug` with admin role `bugadmin`
```
root@pulsar-test-241-bastion-658bc9b859-fx68v:/pulsar# bin/pulsar-admin tenants create pulsarbug --admin-roles bugadmin
```
Verify that `bugadmin` is an admin role for tenant `pulsarbug`
```root@pulsar-test-241-bastion-658bc9b859-fx68v:/pulsar# bin/pulsar-admin tenants get pulsarbug
{
"adminRoles" : [ "bugadmin" ],
"allowedClusters" : [ "pulsar-test-241" ]
}```
Get new token for `bugadmin`
```root@pulsar-test-241-bastion-658bc9b859-fx68v:/pulsar# bin/pulsar tokens create -sk file:///opt/secret.key -s bugadmin
eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJidWdhZG1pbiJ9.HWWJRc_9BEq_4wsOk7YtC2XeqiU1BfhjAPlKpC06MB4```
Validate token for the `bugadmin` role
```root@pulsar-test-241-bastion-658bc9b859-fx68v:/pulsar# bin/pulsar tokens validate -sk file:///opt/secret.key -i eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJidWdhZG1pbiJ9.HWWJRc_9BEq_4wsOk7YtC2XeqiU1BfhjAPlKpC06MB4
{sub=bugadmin}```
As cluster admin, this command executes and returns nothing (as expected, I haven’t setup any namespaces for this Tenant yet)
`root@pulsar-test-241-bastion-658bc9b859-fx68v:/pulsar# bin/pulsar-admin namespaces list pulsarbug`
As tenant admin role, I get a 401.
```root@pulsar-test-241-bastion-658bc9b859-fx68v:/pulsar# bin/pulsar-admin --auth-params token:eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJidWdhZG1pbiJ9.HWWJRc_9BEq_4wsOk7YtC2XeqiU1BfhjAPlKpC06MB4 namespaces list pulsarbug
HTTP 401 Unauthorized
Reason: HTTP 401 Unauthorized```
Any help?
----
2019-10-23 21:45:39 UTC - Matteo Merli: You can create a subscription an reset it to a a particular message id
----
2019-10-23 22:03:46 UTC - Kendall Magesh-Davis: @Matteo Merli :pray:
----
2019-10-23 22:08:41 UTC - Nicolas Ha: from <https://pulsar.apache.org/docs/en/concepts-architecture-overview/#brokers> it seems the brokers are stateless, if so the problem would be in the bookies then? however the bookies seem fine (logs and kubernetes state)
----
2019-10-23 22:42:33 UTC - Addison Higham: hrm... I am running with auth in 2.4.1 and not having any issues. Is this with a proxy and/or multiple brokers?
----
2019-10-23 22:43:04 UTC - Addison Higham: if you are running a proxy, you need to set some configuration options to allow the proxy to properly pass forward auth
----
2019-10-23 22:43:20 UTC - Addison Higham: if you are running multiple brokers, you need to make sure that the brokers are configured with a superuser token to talk to each other
----
2019-10-24 00:01:56 UTC - tuteng: Please configure parameters of client according to this document <http://pulsar.apache.org/docs/en/next/security-jwt/#cli-tools> Parameters authPlugin and authParams need to be added.
----
2019-10-24 01:50:28 UTC - Kendall Magesh-Davis: Thanks for the responses guys. I had it working yesterday. Tore down the cluster and spun it back up from scratch and now it’s not working.
@Addison Higham I have a helm deployment with multiple brokers. They are configured as well as the proxy with a role. Everything is healthy and works when using the “admin” token.
```kubectl get po
NAME READY STATUS RESTARTS AGE
pulsar-test-241-autorecovery-868d8449df-nr58r 1/1 Running 1 7h
pulsar-test-241-bastion-658bc9b859-fx68v 1/1 Running 0 7h
pulsar-test-241-bookkeeper-0 1/1 Running 0 7h
pulsar-test-241-bookkeeper-1 1/1 Running 0 7h
pulsar-test-241-bookkeeper-2 1/1 Running 0 7h
pulsar-test-241-broker-595995759-mgzgm 1/1 Running 0 7h
pulsar-test-241-broker-595995759-qqmcv 1/1 Running 0 7h
pulsar-test-241-proxy-7f474cc5f5-6ldb4 1/1 Running 0 7h
pulsar-test-241-zookeeper-0 1/1 Running 0 7h
pulsar-test-241-zookeeper-1 1/1 Running 0 7h
pulsar-test-241-zookeeper-2 1/1 Running 0 7h
pulsar-test-241-zookeeper-metadata-65w9k 0/1 Completed 0 7h```
@tuteng I followed that very doc. The client config for the bastion host (where i’m executing pulsar-admin) is:
```webServiceUrl=<http://pulsar-test-241-broker:8080/>
brokerServiceUrl=<pulsar://pulsar-test-241-broker:6650/>
#tlsAllowInsecureConnection=false
#tlsEnableHostnameVerification=false
authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationToken
authParams=token:redactedadmin.token.fromhelmconfig```
----
2019-10-24 01:53:22 UTC - Kendall Magesh-Davis: Just to be pedantic - broker config:
```configData:
PULSAR_MEM: "\"-Xms128m -Xmx256m -XX:MaxDirectMemorySize=128m -Dio.netty.leakDetectionLevel=disabled -Dio.netty.recycler.linkCapacity=1024 -XX:+ParallelRefProcEnabled -XX:+UnlockExperimentalVMOptions -XX:+AggressiveOpts -XX:+DoEscapeAnalysis -XX:ParallelGCThreads=32 -XX:ConcGCThreads=32 -XX:G1NewSizePercent=50 -XX:+DisableExplicitGC -XX:-ResizePLAB -XX:+ExitOnOutOfMemoryError -XX:+PerfDisableSharedMem\""
PULSAR_GC: "\"-XX:+UseG1GC -XX:MaxGCPauseMillis=10\""
managedLedgerDefaultEnsembleSize: "2"
managedLedgerDefaultWriteQuorum: "2"
managedLedgerDefaultAckQuorum: "2"
deduplicationEnabled: "false"
exposeTopicLevelMetricsInPrometheus: "true"
authenticationEnabled: "true"
authorizationEnabled: "true"
authenticationProviders: "org.apache.pulsar.broker.authentication.AuthenticationProviderToken"
tokenSecretKey: "data:base64,redactedBrokerSecret"
superUserRoles: "admin"
authPlugin: "org.apache.pulsar.client.impl.auth.AuthenticationToken"
brokerClientAuthenticationPlugin: "org.apache.pulsar.client.impl.auth.AuthenticationToken"
brokerClientAuthenticationParameters: "redactedadmin.token.fromhelmconfig"```
proxy config:
```configData:
PULSAR_MEM: "\"-Xms64m -Xmx128m -XX:MaxDirectMemorySize=64m\""
authenticationEnabled: "true"
authorizationEnabled: "true"
authenticationProviders: "org.apache.pulsar.broker.authentication.AuthenticationProviderToken"
tokenSecretKey: "redactedBrokerSecret"
superUserRoles: "admin"
authPlugin: "org.apache.pulsar.client.impl.auth.AuthenticationToken"
brokerClientAuthenticationPlugin: "org.apache.pulsar.client.impl.auth.AuthenticationToken"
brokerClientAuthenticationParameters: "token:redactedadmin.token.fromhelmconfig"```
----
2019-10-24 02:43:42 UTC - Raman Gupta: As I understand it that won't work if the origin subscription has acked particular messages. e.g.:
```
m1 (acked), m2 (not acked), m3 (acked)
```
How would I tell my new subscription that only m2 needs to be processed?
----
2019-10-24 03:03:35 UTC - Anand Sinha: @Anand Sinha has joined the channel
----
2019-10-24 03:08:44 UTC - Anand Sinha: I had a question about `ConsumerEventListener`.
The following method <http://pulsar.apache.org/api/client/org/apache/pulsar/client/api/ConsumerConfiguration.html#setConsumerEventListener-org.apache.pulsar.client.api.ConsumerEventListener->
has been marked `deprecated` (and `experimental subject to 2.0`).
Is there an alternative to this method? Essentially, I would like to get notified when a consumer in Failover group "dies".
----
2019-10-24 03:22:59 UTC - Anand Sinha: Another question about Java Client API's...
Is there a mechanism to add more `ConsumerInterceptor` and `ProducerInterceptor` instances after Consumers and Producers have been created? Or, can this be done only at the construction of ConsumerBuilder and ProducerBuilder.
----
2019-10-24 04:12:35 UTC - Matteo Merli: The `ConsumerConfiguration` class was deprecated, in favor of `ConsumerBuilder`:
<http://pulsar.apache.org/api/client/org/apache/pulsar/client/api/ConsumerBuilder.html#consumerEventListener-org.apache.pulsar.client.api.ConsumerEventListener->
----
2019-10-24 04:13:02 UTC - Matteo Merli: That was when we switched to a typed API in 2.0
----
2019-10-24 04:16:34 UTC - Matteo Merli: It can only be done at the construction. Though you could add your own meta-interceptor that handles that logic
----
2019-10-24 04:52:11 UTC - Anand Sinha: Sure, thanks, Matteo @Matteo Merli.
I too was thinking in a similar direction (meta-interceptor) in case we cannot add Interceptor instances after a producer/consumer is created.
----
2019-10-24 04:52:58 UTC - Matteo Merli: we try to treat the config passed on the builders as “immutable” to simplify the implementation
----
2019-10-24 04:53:18 UTC - Anand Sinha: :+1:
----
2019-10-24 04:54:26 UTC - Anand Sinha: Thanks. I overlooked that fact.
Quick follow-up question...
----
2019-10-24 04:58:39 UTC - Anand Sinha: If I have a group of 3 consumers for a fail-over subscription; and I would like to handle the scenario of node-failovers, can I use the `ConsumerEventListener` to do it. When node 1 dies (and hence, Consumer 1 becomes inactive), is it expected that the listeners on node 2 and 3 would know of this fact? (Thereby, allowing me to take an action of kicking off another node with a similar consumer).
If this is not the right listener for this, is there an alternative?
----
2019-10-24 05:02:56 UTC - Matteo Merli: the listener will only notify the consumer that will become active/inactive
----
2019-10-24 05:03:31 UTC - Matteo Merli: if a consumer remains in same state (eg: another “inactive” consumer crashes), then it wont be notified
----
2019-10-24 05:04:14 UTC - Matteo Merli: same if a particular consumer is already “inactive” and another consumer is selected to become active
----
2019-10-24 05:31:39 UTC - Anand Sinha: Hmmm. Is there an alternative EventListener (or a system queue with appropriate messages) that helps me achieve the goal that I described earlier?
----
2019-10-24 05:35:19 UTC - Matteo Merli: > When node 1 dies (and hence, Consumer 1 becomes inactive), is it expected that the listeners on node 2 and 3 would know of this fact? (Thereby, allowing me to take an action of kicking off another node with a similar consumer).
When node-1 goes down, either node-2 or node-3 will become active and will receive the notification event. If I understand the problem statement correctly, I think that should be sufficient
----
2019-10-24 05:40:24 UTC - Anand Sinha: When node-1 and node-2 die, node-3 is the only active consumer. When node-1 and node-2 died, I would have expected node-4 and node-5 to have kicked off a consumer process. Else, when node-3 also dies, the subscription has no consumers.
----
2019-10-24 06:31:05 UTC - Gopi Krishna L: Nothing new same thing just happens and happens
----
2019-10-24 06:37:13 UTC - Anand Sinha: If there was a way wherein every consumer could know who are the other consumers under the same subscription (in an efficient way), I could have a way to solve this problem in alternate ways.
(Are admin API's to list consumers for subscription an overkill?)
----
2019-10-24 06:38:58 UTC - Matteo Merli: It’s not particularly heavy to call, as long as it’s not 100s of times/s
----
2019-10-24 06:40:08 UTC - Anand Sinha: Even if each subscription calls once a second, there could be tens of thousands of calls per second; if we have that many subscriptions.
----
2019-10-24 06:40:57 UTC - Anand Sinha: Maybe I throttle it to once a min. But, even then it would be a large number.
----
2019-10-24 06:41:16 UTC - Matteo Merli: that’s true
----
2019-10-24 06:41:40 UTC - Matteo Merli: the other option is to get the topics stats in bulk, from each broker
----
2019-10-24 06:42:07 UTC - Matteo Merli: one single call will return a json with all the topic stats
----
2019-10-24 06:42:21 UTC - Anand Sinha: If I know the names of subscriptions, does that make life any simpler?
----
2019-10-24 06:43:12 UTC - Matteo Merli: check `pulsar-admin broker-stats topics -i`
----
2019-10-24 06:43:22 UTC - Matteo Merli: it goes against the local broker
----
2019-10-24 06:45:59 UTC - Anand Sinha: I need to run this on each of the brokers?
----
2019-10-24 06:46:57 UTC - Anand Sinha: Do we have a "system" queue in Pulsar which puts the messages like "xyz consumer has died"?
----
2019-10-24 06:48:00 UTC - Matteo Merli: It is in the planning to have system topics for all the events
----
2019-10-24 06:48:13 UTC - Matteo Merli: Though it’s not there yet
----
2019-10-24 06:48:51 UTC - Anand Sinha: ok. Another possibility is to add a hook in the component that orchestrates the failovers from one consumer to another.
----
2019-10-24 06:50:21 UTC - Anand Sinha: Or, the component that detects that a node/consumer is down.
----
2019-10-24 06:50:45 UTC - Anand Sinha: Which are these components? And can we add hooks to them?
----