Slack digest for #general - 2018-08-19

[Apache Pulsar Slack <ap...@gmail.com>]

2018-08-18 21:44:39 UTC - Georg Heiler: @Georg Heiler has joined the channel
----
2018-08-18 21:50:38 UTC - Georg Heiler: Hi, does Pulsar have the nice scale out / scale down properties which <http://www.pravega.io> tries to provide as well? Regarding Apache Beam integration - as Pulsar offers some Kafka API is this how it should be used?
----
2018-08-19 01:34:48 UTC - Ali Ahmed: there is no request yet for a beam api support , in terms of elasticity we can transparently add and remove nodes for managing serving and storage capacity
----
2018-08-19 02:01:52 UTC - Sijie Guo: @Georg Heiler regarding beam integration, I am not aware of any efforts on providing beam io connector. Pulsar has kafka api, so the existing kafka connector should probably work. However I think pulsar is a combination of kafka and rabbitmq via different subscription type. So it might be making sense to do a deep integration between beam and pulsar. We are not beam experts, but would love to collaborate with beam :)
+1 : Abhishek Sharma
----
2018-08-19 02:06:02 UTC - Sijie Guo: @Georg Heiler regarding scale up and down, currently pulsar doesnt provide automatically scale up and down partitions, but it provide rest api for people to change partitions. Creating partitions on pulsar is a cheap operation. It can be hook up with stats/monitoring systems to do so. Pulsar chooses to provide flexibility to users for now. But if the community has the need for auto scaling up and down, we can prioritize that :) 
----
2018-08-19 05:41:17 UTC - Georg Heiler: @Sijie Guo @Ali Ahmed thanks for the answers. Is Pulsar currently better integrated into Spark or Flink?
----
2018-08-19 05:44:57 UTC - Ali Ahmed: there is spark connector available
<https://pulsar.incubator.apache.org/docs/latest/adaptors/PulsarSpark/>
----
2018-08-19 05:45:17 UTC - Ali Ahmed: flink has pr open <https://github.com/apache/flink/pull/5845/>
----
2018-08-19 05:45:43 UTC - Georg Heiler: thanks
----