You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2018/05/28 09:11:02 UTC
Slack digest for #general - 2018-05-28
2018-05-27 15:28:22 UTC - JD Suarez: Is it possible to deploy a Java Pulsar Function which uses third party libraries, e.g. Spring? If so, how would you go about doing it? Would you need to build a "fat" jar?
----
2018-05-27 15:29:28 UTC - Matteo Merli: correct, it would need to be a fat jar
----
2018-05-27 18:00:39 UTC - Idan: We were trying to POC pulsar. trying to implement the client. followed the guide on your official website but seems like pulsar-client:2.0.0-incubating not available via maven repo. any idea?
----
2018-05-27 18:01:15 UTC - Idan: > Could not find org.apache.pulsar:pulsar-client:2.0.0-incubating.
Searched in the following locations:
<https://repo1.maven.org/maven2/org/apache/pulsar/pulsar-client/2.0.0-incubating/pulsar-client-2.0.0-incubating.pom>
<https://repo1.maven.org/maven2/org/apache/pulsar/pulsar-client/2.0.0-incubating/pulsar-client-2.0.0-incubating.jar>
file:/Users/idanfridman/.m2/repository/org/apache/pulsar/pulsar-client/2.0.0-incubating/pulsar-client-2.0.0-incubating.pom
file:/Users/idanfridman/.m2/repository/org/apache/pulsar/pulsar-client/2.0.0-incubating/pulsar-client-2.0.0-incubating.jar
<https://jcenter.bintray.com/org/apache/pulsar/pulsar-client/2.0.0-incubating/pulsar-client-2.0.0-incubating.pom>
<https://jcenter.bintray.com/org/apache/pulsar/pulsar-client/2.0.0-incubating/pulsar-client-2.0.0-incubating.jar>
----
2018-05-27 18:01:55 UTC - Matteo Merli: It should be available by now already since the release was finally approved. Note that the release version is `2.0.0-rc1-incubating`
----
2018-05-27 18:02:18 UTC - Idan: not by your site
----
2018-05-27 18:02:47 UTC - Idan: @Idan uploaded a file: <https://apache-pulsar.slack.com/files/UALJD8929/FAWEAJJRF/screen_shot_2018-05-27_at_21.02.23.png|Screen Shot 2018-05-27 at 21.02.23.png>
----
2018-05-27 18:04:25 UTC - Matteo Merli: I know, figured out yesterday that version was incorrect and also it was published before the jars were on maven. There's a PR already open to fix the website. <https://github.com/apache/incubator-pulsar/pull/1847>
----
2018-05-27 18:07:41 UTC - Idan: when final version is planned to be launched ?
----
2018-05-27 18:09:06 UTC - Matteo Merli: It's already "approved" and ready. It will be announced early this week
----
2018-05-27 18:09:46 UTC - Idan: thanks for info
----
2018-05-27 18:11:55 UTC - Idan: @Matteo Merli I think you have another issue on docs:
----
2018-05-27 18:12:14 UTC - Idan: @Idan uploaded a file: <https://apache-pulsar.slack.com/files/UALJD8929/FAY2BGSUF/screen_shot_2018-05-27_at_21.11.43.png|Screen Shot 2018-05-27 at 21.11.43.png>
----
2018-05-27 18:12:30 UTC - Idan: @Idan uploaded a file: <https://apache-pulsar.slack.com/files/UALJD8929/FAY2BHW7R/screen_shot_2018-05-27_at_21.12.16.png|Screen Shot 2018-05-27 at 21.12.16.png>
----
2018-05-27 18:12:36 UTC - Idan: last screenshoot from my iDE
----
2018-05-27 18:17:29 UTC - Idan: anyway generics in java are written before the variable (left-side)
----
2018-05-27 18:18:16 UTC - Matteo Merli: Yes, that was a typo :)
----
2018-05-27 18:24:21 UTC - Idan: :slightly_smiling_face:
----
2018-05-27 18:25:22 UTC - Idan: this doc is not updated:(
----
2018-05-27 18:25:31 UTC - Idan: MessageBuilder<byte[]> msgBuilder = MessageBuilder.create();
----
2018-05-27 18:25:36 UTC - Idan: is deprecated on rc1
----
2018-05-27 18:25:56 UTC - Idan: @Idan uploaded a file: <https://apache-pulsar.slack.com/files/UALJD8929/FAW9Y23H6/screen_shot_2018-05-27_at_21.25.43.png|Screen Shot 2018-05-27 at 21.25.43.png>
----
2018-05-27 18:27:33 UTC - Matteo Merli: Yes, the preferred is to get the builder from `producer.newMessage()...` since it already has the type info. Thanks for pointing out. We'll fix asap
----
2018-05-27 18:31:35 UTC - Idan: ok. Please let me know from where I can get updated manual
----
2018-05-27 21:20:07 UTC - Luc Perkins: @Idan In the future, please don’t hesitate to file issues on the GitHub repo
----
2018-05-27 21:26:21 UTC - Jerry Peng: @JD Suarez Yes that is totally possible. We expect users to do that. You just need to build a fat jar
----
2018-05-27 21:26:51 UTC - Jerry Peng: with all your external dependencies
----
2018-05-28 06:40:29 UTC - Idan: will do. thanks.
----
2018-05-28 06:43:22 UTC - Idan: although iam still not sure how to create new message with the new API that @Matteo Merli was mentioned
----
2018-05-28 06:44:01 UTC - Idan: the deprecated info on code wasnt mentioning the new api
----
2018-05-28 06:59:35 UTC - Matteo Merli: @Idan there's one example in the javadoc: <https://pulsar.incubator.apache.org/api/client/org/apache/pulsar/client/api/Producer.html#newMessage-->
----
2018-05-28 07:17:25 UTC - Idan: ok great. thank you.
+1 : Matteo Merli
----
2018-05-28 08:07:44 UTC - Idan: @Matteo Merli our app main flow is realtime and requires very low latency. we have few services where we plan to add pulsar queue’s to communicate between them to ensure low-latency message communication. Which approach would you recommend to procude the messages. sync or async? We using vert.x framework which allow us non blocking thread IO. so we must not block our thread(so we probablly can wait for the broker to respond - the sync way) . I read about the async-producer which allow us to queue messages. so that will be great with our non-blocking framework but wont it harm the latency? as I read that those messages wont be deliver to the address right away (only after interval or when X messages will be queued up). what do you recommend?
----
2018-05-28 08:22:59 UTC - Ali Ahmed: @Idan Can you be more specific about your latency requirements ?
----
2018-05-28 08:23:17 UTC - Ali Ahmed: do you have any hard constraints ?
----
2018-05-28 08:24:14 UTC - Ali Ahmed: <http://vertx.io|vertx.io> is a event loop framework it’s non blocking but I won’t consider it a low latency system
----
2018-05-28 08:47:49 UTC - Idan: @Ali Ahmed our latency requirements would be 300ms top for the whole flow
----
2018-05-28 08:48:23 UTC - Idan: we have 4 services in our real-time flow. each one of them consumes avg 20-40ms. all the rest should be “wasted” on communications
----
2018-05-28 08:49:14 UTC - Idan: so 4 services * 40ms = 160ms. if we have 3 queues between them that leaves us with around 50ms latency as Max latency per queue
----
2018-05-28 09:10:17 UTC - Ali Ahmed: that should not a problem pulsar can achieve publish latency of 5 ms and less , I would recommend going with async producer client with small batch size or size even of 1.
----