Slack digest for #general - 2018-07-03

[Apache Pulsar Slack <ap...@gmail.com>]

2018-07-02 20:18:46 UTC - Beast in Black: @Beast in Black has joined the channel
----
2018-07-02 20:27:11 UTC - Beast in Black: Hi all. I was checking out the "functions" feature of pulsar 2.0, and wanted to programmatically register a function using the Admin REST API. I see in the docs ( <https://pulsar.incubator.apache.org/docs/latest/reference/RestApi/> ) that there is a POST method for creating a function, but the only parameters listed in the doc are "tenant", "namespace" and "functionName", whereas the rest of the documentation on Functions (which use the pulsar-admin CLI tool) allow specification of a source and sink topic via "--inputs" and "--output". My question was: Is there a way to specify the source and sink topics via the Admin REST API?
----
2018-07-02 20:34:47 UTC - Sijie Guo: @Beast in Black 

&gt; whereas the rest of the documentation on Functions (which use the pulsar-admin CLI tool) allow specification of a source and sink topic via “--inputs” and “--output”

sorry I think the functions rest api is not well documented at this moment. we have a working item to improve it this week. 

for now, you can check this file for some of the details: <https://github.com/apache/incubator-pulsar/blob/master/pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/FunctionsImpl.java>

if you are programming in java, you can use pulsar-client-admin to submit functions.

back to your question:

you need to construct a functionDetails json object and attach functionsDetails json and functions jar to multi-part body to submit.
----
2018-07-02 20:35:59 UTC - Beast in Black: @Sijie Guo thank you. I am using python, actually. But I'll take a look at the Java code and see if I can extrapolate from it.
----
2018-07-02 20:36:28 UTC - Sijie Guo: and this is the structure: `functionsDetails` - <https://github.com/apache/incubator-pulsar/blob/master/pulsar-functions/proto/src/main/proto/Function.proto#L43>
----
2018-07-02 20:37:58 UTC - Beast in Black: @Sijie Guo that last link seems to use SerDe for specifying the source and sink. Would this work in python as well?
----
2018-07-02 20:39:47 UTC - Beast in Black: @Sijie Guo I also took a look at <https://github.com/apache/incubator-pulsar/blob/master/pulsar-functions/python-examples> but those don't have any examples on using the Admin REST API to create the function in the cluster.
----
2018-07-02 20:42:59 UTC - Sijie Guo: @Beast in Black 

&gt; that last link seems to use SerDe for specifying the source and sink. Would this work in python as well?

yes SerDe is working in python as well. you can see the available SerDe for python here: <https://pulsar.incubator.apache.org/docs/latest/functions/api/#PythonSerDe-dpjav9>

&gt; <https://github.com/apache/incubator-pulsar/blob/master/pulsar-functions/python-examples>

this was intended to be used for keeping example functions. so when people download the pulsar distribution, they can quickly start using pulsar-admin tool to submit functions.

we didn’t realize the demand for programming functions submission in python. we are improving that part now.
----
2018-07-02 20:47:05 UTC - Beast in Black: @Sijie Guo I read that link, thank you. The Python section of the documentation at <https://pulsar.incubator.apache.org/docs/latest/functions/api/> mentions the Python context object and the associated methods but there does not seem to be any way to set things - just get them (at least as per the docs)
----
2018-07-02 20:49:22 UTC - Sijie Guo: context - is a *immutable* object, which provides functions an access point for it to access resources in pulsar. the object is set and provided by pulsar python runtime. you are not supposed to modify it in python functions.
----
2018-07-02 20:49:53 UTC - Beast in Black: @Sijie Guo aah ok
----
2018-07-02 20:50:29 UTC - Beast in Black: @Sijie Guo thank you for your help, much appreciated! I may just end up coding stuff in Java and give up on Python for now :slightly_smiling_face:
----
2018-07-02 20:51:36 UTC - Sijie Guo: :slightly_smiling_face: we are improving python. that’s probably most easiest way for people to write a function and play with pulsar functions
----
2018-07-02 21:20:12 UTC - Matteo Merli: <!here> Reminder regarding meetup event on Tue July 10th hosted by Oath/Yahoo - <https://www.meetup.com/SF-Bay-Area-Apache-Pulsar-Meetup/events/252312389/>
+1 : Sijie Guo
bananadance : Sijie Guo
star-struck : Sijie Guo
passenger_ship : Sijie Guo
----
2018-07-02 21:20:22 UTC - Matteo Merli: @Matteo Merli pinned a message to this channel.
----
2018-07-03 08:06:22 UTC - Matti-Pekka Laaksonen: When using consumer's receiveAsync, is it preferable to use an object that implements MessageListener interface and handle the message with the received() method? If we only use one consumer, is the ordering of messages guaranteed when using async receive and MessageListener?
----
2018-07-03 08:26:20 UTC - Sijie Guo: @Matti-Pekka Laaksonen 

both receiveAsync &amp; MessageListener guaranteed messages ordering when using non-shared subscription type.

MessageListener is probably easy to startup if your application is purely just acting on messages.

receiveAsync gives you more controls.

under the hood, they are executing same code path.
----
2018-07-03 09:04:21 UTC - Matti-Pekka Laaksonen: Alright, thank you!'
----