You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2020/10/23 09:11:06 UTC
Slack digest for #general - 2020-10-23
2020-10-22 09:20:42 UTC - Johannes Wienke: What is the issue with SubRecords in Python? Can't get this code working:
```from enum import Enum
import pulsar
from pulsar.schema import AvroSchema, String, Record
class TypeEnum(Enum):
TYPE_1 = "type_1"
TYPE_2 = "type_2"
class TestType(Record):
value = String()
class Composite(Record):
the_type = TypeEnum
first = TestType()
client = pulsar.Client("<pulsar://localhost:6650>")
producer = client.create_producer("schema-test-2", schema=AvroSchema(Composite))
for i in range(10):
producer.send(Composite(the_type=TypeEnum.TYPE_1, first=TestType(value="test")))
client.close()```
This always gives
> ValueError: <__main__.TestType object at 0x7efc7b09eeb0> (type <class '__main__.TestType'>) do not match ['null', {'type': 'record', 'name': 'TestType', 'fields': [{'name': 'value', 'type': ['null', 'string']}]}] on field first
----
2020-10-22 11:45:59 UTC - VanderChen: Thank you very much. I'll have a try.
----
2020-10-22 12:15:08 UTC - Joshua Decosta: I appreciate the response, no worries as i know this isn’t your main responsibility. I’d be down for a call. I’ve been looking at the source code to see what’s going on. I noticed in September there was an issue that seems related to what I’m seeing. It was closed with a recent PR. Perhaps that’s not in the current version I’m using.
----
2020-10-22 12:16:43 UTC - Joshua Decosta: Did you configure Prometheus to use the same auth? If not it can’t get the metrics unless you’ve turned off the auth on the metrics endpoint
----
2020-10-22 16:37:19 UTC - Aaron Stockton: Am I reading this correctly, that with batching enabled there’s no way to get a count of unconsumed messages, only unconsumed batches? <https://pulsar.apache.org/docs/en/concepts-messaging/#batching>
----
2020-10-22 16:40:30 UTC - Lari Hotari: this is a real gotcha in Pulsar. I now realize that I have this wrong for the namespace where the 100% CPU bug occured. I checked the docs and it's only in the javadocs where it's documented clearly.
<https://pulsar.apache.org/docs/en/2.6.1/cookbooks-retention-expiry/#set-retention-policy> doesn't explicitly document the meaning of 0 and -1. There's an example, but no explicit documentation like there is in the Javadoc at <https://github.com/apache/pulsar/blob/22b8923a251c3356401090dacb8ff97fd37d14c1/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedLedgerConfig.java#L415-L417> . I could create a Github issue for documenting this in a better way.
I think that setting the retention policy to something where only either one of the values is non zero doesn't make sense at all. That should fail already in validation. I think that deserves it's own Github issue. WDYT?
----
2020-10-22 16:43:28 UTC - Lari Hotari: I take my claim back. I now see a sentence "It is also possible to set _unlimited_ retention time or size by setting `-1` for either time or size retention." <https://pulsar.apache.org/docs/en/2.6.1/cookbooks-retention-expiry/#retention-policies> . However it's really easy to miss that and I think that a config where one is non zero and other is zero doesn't make sense and should be rejected to prevent mistakes in configuration.
----
2020-10-22 16:44:08 UTC - Lari Hotari: Well that sentence is a bit misleading. since it says "by setting -1 for either time or size retention".
----
2020-10-22 16:55:11 UTC - Sijie Guo: Nice catch. Are you going to fix the documentation? Or shall I loop @Jennifer Huang on this?
----
2020-10-22 17:01:10 UTC - Lari Hotari: I'll create a GH issue now and I can possibly send a PR. I'm currently filing 2 GH issues, one for the docs and the other for the validation of the policy when it's set.
----
2020-10-22 17:05:09 UTC - otman: Hi i face this error in my client that try to connect to remote broker: - 2020-10-22 16:51:57.091 [pulsar-client-io-923-1] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0x106d4fd0, L:/10.42.10.2:57588 - R:pulsar.testenv.local/127.0.0.158:30590] Received error from server: Namespace bundle for topic (<persistent://tenant/ns/action-7015e80f-60dc-485f-9c6f-b092bf5644d9>) not served by this instance. Please redo the lookup. Request is denied: namespace=tenant/ns. - Also i have several broker running in server side and 50000 clients that try to connect to broker. Each client have 4 topics. When i try to list all available non partitioned topics from CLI, i face 500 internal error and when i did a lookup on a specific topic, it return to me a specific broker (for ex broker2). Is it possible that a topics is handle by a specific instance ?
----
2020-10-22 17:12:03 UTC - Lari Hotari: <https://github.com/apache/pulsar/issues/8345> for the documentation issue
+1 : Jennifer Huang
----
2020-10-22 17:14:51 UTC - Addison Higham: I agree with having either one of those values be zero is going to cause issues and we should validate against that, bit of a truth table on all the combinations:
both greater than zero: valid
both -1: valid
both 0: valid
one greater than, one -1: valid
one 0, the other greater than 0 or -1: invalid
I think that makes sense?
+1 : Lari Hotari
----
2020-10-22 17:20:08 UTC - Lari Hotari: yes, makes sense. I created a separate issue for the validation <https://github.com/apache/pulsar/issues/8346>
----
2020-10-22 17:20:56 UTC - Lari Hotari: @Addison Higham please add your truth table suggestion on that issue
----
2020-10-22 17:22:08 UTC - Addison Higham: done
+1 : Lari Hotari
----
2020-10-22 17:40:26 UTC - Guillaume: I don't think so. Where and how can I configure Prometheus to use the same auth? Thank you.
----
2020-10-22 17:41:52 UTC - Joshua Decosta: You would need to google Prometheus authentication. (Sorry I’m on my phone) there are some docs floating around that will show you what you need to do.
----
2020-10-22 17:42:40 UTC - Joshua Decosta: I know there is a token setup floating around somewhere if that’s what you’re using
----
2020-10-22 18:27:36 UTC - Milos Matijasevic: here is added async producer send, (<https://gist.github.com/milos-matijasevic/e29d00e279b5d1a540552864c7ce1321|new-code>) but i have problem that for example 10k messages are processed and acked but in new topic only 10 messages are inserted and for solving this, i put `DisableBatching` to true in producersOptions, but i don't understand why this is problem ?
----
2020-10-22 21:47:23 UTC - Milos Matijasevic: i understood, all messages are there just they are batched inside entries thats why there is 10 entries for 10k messages because 1 batch contains 1k messages
----
2020-10-22 21:57:41 UTC - Kenny Sargent: Hey Pulsar friends, any idea why RedeliveryCount not being incremented in Exclusive and Failover subscriptions? When we nack a message and it retries (a minute later) we see RedeliveryCount stay at zero. For Shared and KeyShared subscription types (with or without dead letter queue) we see the RedeliveryCount incrementing. Is this by design, something we’re doing wrong, or a pulsar bug? Any thoughts?
----
2020-10-23 00:58:27 UTC - Jennifer Huang: :+1:
----
2020-10-23 02:28:12 UTC - テテナン: @テテナン has joined the channel
----
2020-10-23 02:29:50 UTC - テテナン: Hi, I'm using Python with Pulsar-Client, but cannot make it through the company proxy. Any idea ?
----
2020-10-23 05:07:30 UTC - Sankararao Routhu: Hi, is it possible to have two bookies(part of the Ensemble) down at the same time with out impacting consumer with Ensemble=3, Qw=2, Qa=2..It is causing the consumers to stall until I bring back one of the bookies back..I tried resetting the cursor to a recent messageId and then consumer is able to resume with recent messageId. But consumer is missing the old unconsumed messages. If I bring back the bookies and reset the cursor back to old msgId then consumer started receiving old messages along with recent messages(which are already consumed and acknowledged). Is there any better to handle this case?
----
2020-10-23 05:13:47 UTC - Addison Higham: I believe some metrics properly break it down, but in general that is correct
----
2020-10-23 05:16:53 UTC - Addison Higham: interesting... not sure why that would be happening, but are you trying to use a dead letter queue?
----
2020-10-23 05:22:08 UTC - Addison Higham: is the consumer caught up when it stalls? because when Qa=2 and you have two brokers down, producers won't be able to produce.
----
2020-10-23 05:39:06 UTC - Sankararao Routhu: @Addison Higham I brought down 2 bookies, and topic owner broker is still up. Producer is still able to publish when two bookies down because we have 5 bookies in a region and Ensemble(3) formed with two other bookies
----
2020-10-23 05:39:36 UTC - Sankararao Routhu: Only consumer is stalled because it is waiting for a message which is in down bookies
----
2020-10-23 05:45:59 UTC - Addison Higham: I see, consumers should be able to read from any copy of the bookie, @Sijie Guo you have any idea on that one
----
2020-10-23 05:47:23 UTC - Sijie Guo: Because Qw is 2
----
2020-10-23 05:47:34 UTC - Sijie Guo: So when you lose 2 bookies, there is no copy to read
----
2020-10-23 05:48:15 UTC - Addison Higham: oops! didn't notice that
----
2020-10-23 05:48:52 UTC - Sankararao Routhu: yes 2 copies might be in the down bookies
----
2020-10-23 05:49:45 UTC - Sankararao Routhu: is there a way to recover from this immediately without impacting the consumer until we bring back the bookies?
----
2020-10-23 05:50:00 UTC - Addison Higham: yes, so the benefit of having more members in ensemble that in write quorum is striping but it does come at cost of reads, so usually you would want 3/3/2
----
2020-10-23 05:50:42 UTC - Sankararao Routhu: you mean write quorum should be same as Ensemble?
----
2020-10-23 05:51:33 UTC - Addison Higham: but if you are sure you want just 2 copies, then you can use `autoSkipNonRecoverableData`
----
2020-10-23 05:51:58 UTC - Sankararao Routhu: 2 copies are enough for us
----
2020-10-23 05:52:50 UTC - David Tinker: Hi Guys. I have a simple pulsar setup with 3 machines each running zk, bookie and broker. I want to use a separate zk cluster for the configuration store in preparation for adding another pulsar cluster. How can do this while keeping everything live? Tx.
----
2020-10-23 05:52:54 UTC - Sankararao Routhu: will `autoSkipNonRecoverableData` create any message loss to consumer?
----
2020-10-23 05:53:28 UTC - Sankararao Routhu: I mean what ever messages that were skipped, can I go back and consume when bookies are brought back?
----
2020-10-23 05:56:33 UTC - Addison Higham: if the ledger can't currently `be read, then once the bookeis come back up, you would need to rewind the consumer
----
2020-10-23 05:57:08 UTC - Addison Higham: but in answer to you question, usually keeping your ensemble size and quorum at the same value is preferred
----
2020-10-23 06:03:29 UTC - Sankararao Routhu: thanks @Addison Higham I will test with `autoSkipNonRecoverableData` enabled. I have used Ensemble=3 just to achieve striping for high performance, if E and Qw should be same then I will set E=2, Qw=2
----
2020-10-23 06:04:13 UTC - Sankararao Routhu: can I set Qa=1 in this case? will it cause any data loss if one bookie goes down with E=2, Qw=2,Qa=1?
----
2020-10-23 08:03:33 UTC - Lari Hotari: I pushed the PR to improve the documentation of retention policy: <https://github.com/apache/pulsar/pull/8356> . Please review
----