Slack digest for #general - 2018-03-25

[Apache Pulsar Slack <ap...@gmail.com>]

2018-03-24 09:40:56 UTC - Igor Zubchenok: When I use several publishers to the same topic, every publisher starts sequence number from 0. How can I get message among all publishers sequence number?
----
2018-03-24 09:52:36 UTC - Igor Zubchenok: getEntryId looks like a such sequence number, however it is not exposed to MessageId interface.
----
2018-03-24 15:35:01 UTC - Piotr: Ivan wrote this earlier:
----
2018-03-24 15:35:03 UTC - Piotr: Piotr [9:50 AM]
I see - and are the pulsar ids sequential? I tried to find docs about that - and couldnt but then I just debugged them and they seem to look like 1:15:2 etc

ivan [9:50 AM]
you're basically externalizing the database wal
pulsar ids are sequential yes. you can read it as <segment>:<entryinsegment>:<partition>:<positioninbatch>

Piotr [9:52 AM]
Ok - so they are not sequential really?
I mean, how would you get the latest messageId from a set of those?

ivan [9:54 AM]
they are. there are gaps, but they are monotonically increasing

Piotr [9:56 AM]
so you would order those first by segment, then by entryinsegment, then by partition and then by positioninbatch?
you can never get a messageId with partition 2 and then the next one with partition 1?

ivan [9:58 AM]
well, there's no ordering between partitions, so you shouldn't try to
but yes, first by segment, then entry, then batch
----
2018-03-24 15:42:31 UTC - Piotr: MessageId extends Comparable. Not sure if that logic applies there tho
----
2018-03-24 15:53:22 UTC - Piotr: it seems to compare by ledgerId, entryId, partitionId - hmm...
----
2018-03-24 15:54:37 UTC - Piotr: Does that mean we can´t get the latest messageId by using compare?
----
2018-03-24 15:55:28 UTC - Matteo Merli: What is the intention?
----
2018-03-24 16:01:35 UTC - Piotr: to get the latest streamed message id :slightly_smiling_face:
----
2018-03-24 16:03:35 UTC - Piotr: I know insertion order can´t be guaranteed - but the order of all messages for the consumers is the same right? And from a set of messages I can use the id to see which is the last streamed one?
----
2018-03-24 16:04:07 UTC - Matteo Merli: if you’re consuming from multiple partitions, even the consumption order is not guaranteed
----
2018-03-24 16:04:45 UTC - Matteo Merli: it’s like you getting data from multiple queues and multiplexing into a single queue
----
2018-03-24 16:05:10 UTC - Matteo Merli: if you crash and come back, you can have a different order (across partitions)
----
2018-03-24 16:05:38 UTC - Piotr: ok - but single-partition consumption is safe then?
----
2018-03-24 16:05:56 UTC - Matteo Merli: yes, that where you get total ordering
----
2018-03-24 17:29:13 UTC - Venkat: @Venkat has joined the channel
----
2018-03-24 18:05:14 UTC - Karthik Palanivelu: @Matteo Merli can you please let me know how to get super user roles added and used for standalone cluster creation or point me to any docs? All I cud find is create cluster needs super user privilege.
----
2018-03-24 18:09:58 UTC - Matteo Merli: Hi @Karthikeyan Palanivelu, the super-user roles are only used when authn/authz are enabled. These are the admin roles that are granted permission to work on every namespace/topic
----
2018-03-24 18:10:14 UTC - Matteo Merli: I think I missed to answer your previous message
----
2018-03-24 18:11:47 UTC - Matteo Merli: > @Matteo Merli I tried with url and broker-url options and both all returning connection refused. I can access these instances from my local env, 3 standalone are in AWS

> @Matteo Merli is this because of super user roles is empty in my standalone conf?

No, if you don’t have auth, then that config is not relevant
----
2018-03-24 18:12:04 UTC - Matteo Merli: Can you show the exact error you’re getting ?
----
2018-03-25 00:58:08 UTC - Karthik Palanivelu: I am getting connection refused; then let me check the security group.
----
2018-03-25 00:58:33 UTC - Karthik Palanivelu: Thanks for your time and response. Appreciate it.
----