Slack digest for #general - 2018-11-14

[Apache Pulsar Slack <ap...@gmail.com>]

2018-11-13 10:16:13 UTC - Christophe Bornet: The doc states that messages for which all subscription have acked are deleted. How does it work with geo-replication ? If I don't have any subscription on the backup topic and no retention policy, will it be deleted right away ?
----
2018-11-13 10:23:26 UTC - Ivan Kelly: i think georeplication itself holds a subscription
----
2018-11-13 11:09:38 UTC - Jeroen van der Wal: @Jeroen van der Wal has joined the channel
----
2018-11-13 11:23:06 UTC - Christophe Bornet: Geo-replication holds a subscription on the master, not on the replica, isn't it ?
----
2018-11-13 11:24:31 UTC - Ivan Kelly: oh, that i don't know
----
2018-11-13 11:36:30 UTC - Kyle: @Kyle has joined the channel
----
2018-11-13 17:01:08 UTC - Christophe Bornet: @Matteo Merli what would be needed to do a replicated subscription ? That would be a killer feature compared to the competition ! If you give some guidance we could help
----
2018-11-13 17:31:43 UTC - Matteo Merli: @Christophe Bornet is actually a bit tricky. We have a general idea on how to implement this but it was not flushed into a complete design yet. We’ll probably come back to it once 2.3 is released.

The main thing is that, while supporting basic scenario is (relatively) easy (eg: active-passive failover with 2 DCs), it can become very complex once taking into picture many DCs with producers in each of them
----
2018-11-13 17:58:15 UTC - Christophe Bornet: Active-passive with 2 DCs is precisely the use-case I'm interested in. Is it something that can I can implement by myself client-side or would it need things to be done in the brokers ?
----
2018-11-13 18:06:38 UTC - Matteo Merli: From client side, you could publish timestamps of last message acknowledged in a different topic. Then use that at failover time to set reposition the subscription in the 2nd DC to a particular time. It’s not “difficult” but it’s not automated either.. .
----
2018-11-13 18:07:00 UTC - Thomas Remmert: @Thomas Remmert has joined the channel
----
2018-11-13 18:13:33 UTC - Thomas Remmert: Evening from London everyone.. question about Pulsar partitions if I may. Am looking at alternative to kafka..  With Kafka we are making use of consumer groups on a partitioned topic, ensuring that all messages with a certain ID (used as an id to hash) end up with  the same consumer in a group of consumers...  (the consumer has state).... Is that something I can do with Pulsar in some way.... ?
----
2018-11-13 18:15:32 UTC - Matteo Merli: @Thomas Remmert Yes, there are different subscription types in Pulsar. Using “Failover” type, you get a similar behavior as a Kafka consumer group: available partitions are assigned across available consumers. Each partition will only have 1 active consumer at a given point in time.
----
2018-11-13 18:17:20 UTC - Thomas Remmert: Thank you... and can a consumer have more than one partition ? ... e.g. We got 100 partitions and we scale up and down consumers at the moment with Kafka?
----
2018-11-13 18:18:38 UTC - Matteo Merli: Yes, each partitition will be assigned to one consumer
----
2018-11-13 20:21:35 UTC - Thomas Remmert: Thank you very much. :slightly_smiling_face:
----
2018-11-14 03:39:45 UTC - wolf: @wolf has joined the channel
----
2018-11-14 03:54:44 UTC - rjhym: @rjhym has joined the channel
----
2018-11-14 06:55:00 UTC - Strong Liu: @Strong Liu has joined the channel
----
2018-11-14 07:47:40 UTC - Lintao Fan: @Lintao Fan has joined the channel
----
2018-11-14 08:07:25 UTC - zhoukun: @zhoukun has joined the channel
----