Re: [discuss] provide array result

[Matt Welke <ma...@gmail.com>]

Sean,

Sorry for the late reply. I had my OpenWhisk mailing list emails going into
a separate folder in my email. Didn't see this stuff until now.

I have nothing to contribute regarding the new array functionality, but I
think your idea to have a library that allows the OW user to take more
control over serialization and deserialization makes a lot of sense.

A while back, I was creating a few custom runtimes to explore some ideas,
and I think I came to the same conclusion as you with both of your points.
That:

a. It would be nice if the user's function could receive as a parameter and
return native dictionary data types instead of types belonging to
particular JSON libraries that they might not want to use. For example,
Gson in Java.
b. It would be nice if, for programming languages that involve a compiler,
the user could add a dependency on an official library that would give them
a nice developer experience. For example, extending a class in Java.

When I implemented A and tried it out, with .NET and Java as test
languages, I was happy that I didn't have those dependencies on Newtonsoft
and Gson anymore in my code. But then, I realized that, from the
perspective of the OW user, I was at the mercy of the runtime to
deserialize the function input into the native dictionary type for me. If I
didn't like how it did that, I had no recourse. It was nice to get to try
my idea in practice to see this flaw with it.

When I implemented B to try it out, with my Java 18 runtime (
https://github.com/mattwelke/apache-openwhisk-runtime-java-18), I was
really happy with the result. No complaints in retrospect. It was nice to
be able to use a capable IDE like IntelliJ IDEA to start typing "class
MyAction extends ..." and have the IDE fill in the rest. If my code
compiled, I was confident it was compatible programmatically with the Java
18 runtime. You can see an example of this use of the library and extending
a class approach in
https://github.com/mattwelke/packt-book-bot/blob/main/title-fetcher/src/main/java/com/mattwelke/packtbookbot/TitleFetcherAction.java
.

I think it'd be a good idea to combine those concepts together in the
future for strongly typed programming languages. An OW user could extend a
class (or whichever other way is most appropriate for the language) and
have their tooling help them create their function. And the user could
control serialization and deserialization while they do that, by being
given a chance to provide the code for serialization and deserialization
hooks. Perhaps users could be given a choice, by that library, of which
type of function they want to create, where they have the option of
accepting default choices for JSON serialization and deserialization.

I'm happy to share my thoughts more on this if we want to move in this
direction in the future.

Matt

On Wed, May 25, 2022 at 12:06 AM Shawn Black <en...@yahoo.com.invalid>
wrote:

> All,
>
> I'm glad this discussion is happening because I've been wanting to
> implement some sort of automatic (de)serialization framework on the .NET
> runtime, which requires additional libraries, etc., that provide the bare
> minimum requirements to allow this.
>
> Speaking as generic as I can, what I would implement, at a minimum, are
> two libraries ...
>
> One that provides a contract between the OW runtime and the function on
> how serde should happen. (In .NET this could be an interface or abstract
> class that defined two method signatures, one for serialization and one for
> deserialization. One contract implementation defined per function
> deployable object - the .DLL file or .JAR file, etc.).
>
> One would implement the contract. (In .NET this would be something that
> used another library that provides JSON support, like Newtonsoft.Json or
> System.Text.Json).
>
> This would require the libraries that support this to be published to
> their respective public package repository. (.NET: NuGet)
>
> The code for the function would have some standard metadata that refers to
> the appropriate contract implementation.
>
> The runtime would determine which contract implementation to use, inferred
> from the standard metadata of the function.
>
> I think this approach can be applied to other runtimes as well and I
> believe it would continue to honor the internal OW contracts (data
> transferred in JSON, etc.) but also provide an easier developer experience
> by allowing plain old {insert language} objects to be passed into and out
> of the function being written and called, which I think covers the use case
> here.
>
> Thoughts?
>
> Thanks,
> Shawn
>
>
> On May 24, 2022 10:13:18 PM CDT, "甯尤刚" <ni...@navercorp.com> wrote:
> >Hi, Rabbah, good question.
> >
> >1. Why stop at array, why not any other type?
> >dictionary and array are very common data type in all languages.
> >It is natural to think of array.
> >
> >2. composition work in that case?
> >Since the action is wrote by user, he can control it?
> >If applied this feature, he can judge the return data by like
> `isinstanceof`.
> >
> >-----Original Message-----
> >From: "Rodric Rabbah"<ro...@gmail.com>
> >To: <de...@openwhisk.apache.org>;
> >Cc:
> >Sent: 2022/5/25周三 09:01 (GMT+08:00)
> >Subject: Re: [discuss] provide array result
> >
> >Thanks for the proposal - my immediate reaction is that this changes the
> >programming model in a potentially disruptive way. The function signature
> >now is dictionary (string->json) in and same type out. This allows
> >functions to be composed in a sequence because all functions have a
> uniform
> >signature. If you admit a different kind of return, the signature of the
> >function changes. Why stop at array, why not any other type? And how does
> >composition work in that case?
> >
> >I'm not saying we shouldn't consider this but rather that it's a bit more
> >involved.
> >
> >
> >On Tue, May 24, 2022 at 8:50 PM 甯尤刚 <ni...@navercorp.com> wrote:
> >
> >> Hi, guys
> >>
> >> Currently, openwhisk supports return json object only,
> >> I wrote a POEM in openwhisk upstream that allows user to write their own
> >> action which supports array result. e.g. already implmeneted in my local
> >> env for python, worked well
> >> ```
> >> def main(args):
> >>
> >>    try:
> >>
> >>        name = args.get("name", "World")
> >>
> >>        place  = args.get("place", "Naver")
> >>
> >>        return ["a", "b"]
> >>
> >>    except Exception as e:
> >>
> >>        # please handle exception here
> >>
> >>        # you can return or hide an error message
> >>
> >>        return {"error": str(e)}
> >> ```
> >>
> >> So the result will support object and array both in future.
> >> refer to: https://github.com/apache/openwhisk/pull/5244
> >>
> >> Welcome to give suggestion
> >>
> >>
> >>
> >> 此电子邮件及其包含的信息将仅发送给上面列出的收件人，必须加以保护，并且可能包含法律或其他原因禁止披露的信息。
> >> 如果您不是此电子邮件的预期收件人，未经许可，您不得存储、复制、发送、分发或披露它。 禁止存储、复制、发送、分发或披露电子邮件的任何部分。
> >> 如果此电子邮件发送不正确，请立即联系 NAVER Security(dl_naversecurity@navercorp.com
> >> )。然后删除所有原件、副本和附件。谢谢您的合作。
> >> ​
> >> This email and the information contained in this email are intended
> solely
> >> for the recipient(s) addressed above and may contain information that is
> >> confidential and/or privileged or whose disclosure is prohibited by law
> or
> >> other reasons.
> >> If you are not the intended recipient of this email, please be advised
> >> that any unauthorized storage, duplication, dissemination, distribution
> or
> >> disclosure of all or part of this email is strictly prohibited.
> >> If you received this email in error, please immediately contact NAVER
> >> Security (dl_naversecurity@navercorp.com) and delete this email and any
> >> copies and attachments from your system. Thank you for your
> cooperation.​
> >>
>

Re: [discuss] provide array result

[Matt Welke <ma...@gmail.com>]

Shawn*

Sorry about that.

On Sat, Sep 24, 2022 at 11:12 PM Matt Welke <ma...@gmail.com> wrote:

> Sean,
>
> Sorry for the late reply. I had my OpenWhisk mailing list emails going
> into a separate folder in my email. Didn't see this stuff until now.
>
> I have nothing to contribute regarding the new array functionality, but I
> think your idea to have a library that allows the OW user to take more
> control over serialization and deserialization makes a lot of sense.
>
> A while back, I was creating a few custom runtimes to explore some ideas,
> and I think I came to the same conclusion as you with both of your points.
> That:
>
> a. It would be nice if the user's function could receive as a parameter
> and return native dictionary data types instead of types belonging to
> particular JSON libraries that they might not want to use. For example,
> Gson in Java.
> b. It would be nice if, for programming languages that involve a compiler,
> the user could add a dependency on an official library that would give them
> a nice developer experience. For example, extending a class in Java.
>
> When I implemented A and tried it out, with .NET and Java as test
> languages, I was happy that I didn't have those dependencies on Newtonsoft
> and Gson anymore in my code. But then, I realized that, from the
> perspective of the OW user, I was at the mercy of the runtime to
> deserialize the function input into the native dictionary type for me. If I
> didn't like how it did that, I had no recourse. It was nice to get to try
> my idea in practice to see this flaw with it.
>
> When I implemented B to try it out, with my Java 18 runtime (
> https://github.com/mattwelke/apache-openwhisk-runtime-java-18), I was
> really happy with the result. No complaints in retrospect. It was nice to
> be able to use a capable IDE like IntelliJ IDEA to start typing "class
> MyAction extends ..." and have the IDE fill in the rest. If my code
> compiled, I was confident it was compatible programmatically with the Java
> 18 runtime. You can see an example of this use of the library and extending
> a class approach in
> https://github.com/mattwelke/packt-book-bot/blob/main/title-fetcher/src/main/java/com/mattwelke/packtbookbot/TitleFetcherAction.java
> .
>
> I think it'd be a good idea to combine those concepts together in the
> future for strongly typed programming languages. An OW user could extend a
> class (or whichever other way is most appropriate for the language) and
> have their tooling help them create their function. And the user could
> control serialization and deserialization while they do that, by being
> given a chance to provide the code for serialization and deserialization
> hooks. Perhaps users could be given a choice, by that library, of which
> type of function they want to create, where they have the option of
> accepting default choices for JSON serialization and deserialization.
>
> I'm happy to share my thoughts more on this if we want to move in this
> direction in the future.
>
> Matt
>
> On Wed, May 25, 2022 at 12:06 AM Shawn Black <en...@yahoo.com.invalid>
> wrote:
>
>> All,
>>
>> I'm glad this discussion is happening because I've been wanting to
>> implement some sort of automatic (de)serialization framework on the .NET
>> runtime, which requires additional libraries, etc., that provide the bare
>> minimum requirements to allow this.
>>
>> Speaking as generic as I can, what I would implement, at a minimum, are
>> two libraries ...
>>
>> One that provides a contract between the OW runtime and the function on
>> how serde should happen. (In .NET this could be an interface or abstract
>> class that defined two method signatures, one for serialization and one for
>> deserialization. One contract implementation defined per function
>> deployable object - the .DLL file or .JAR file, etc.).
>>
>> One would implement the contract. (In .NET this would be something that
>> used another library that provides JSON support, like Newtonsoft.Json or
>> System.Text.Json).
>>
>> This would require the libraries that support this to be published to
>> their respective public package repository. (.NET: NuGet)
>>
>> The code for the function would have some standard metadata that refers
>> to the appropriate contract implementation.
>>
>> The runtime would determine which contract implementation to use,
>> inferred from the standard metadata of the function.
>>
>> I think this approach can be applied to other runtimes as well and I
>> believe it would continue to honor the internal OW contracts (data
>> transferred in JSON, etc.) but also provide an easier developer experience
>> by allowing plain old {insert language} objects to be passed into and out
>> of the function being written and called, which I think covers the use case
>> here.
>>
>> Thoughts?
>>
>> Thanks,
>> Shawn
>>
>>
>> On May 24, 2022 10:13:18 PM CDT, "甯尤刚" <ni...@navercorp.com>
>> wrote:
>> >Hi, Rabbah, good question.
>> >
>> >1. Why stop at array, why not any other type?
>> >dictionary and array are very common data type in all languages.
>> >It is natural to think of array.
>> >
>> >2. composition work in that case?
>> >Since the action is wrote by user, he can control it?
>> >If applied this feature, he can judge the return data by like
>> `isinstanceof`.
>> >
>> >-----Original Message-----
>> >From: "Rodric Rabbah"<ro...@gmail.com>
>> >To: <de...@openwhisk.apache.org>;
>> >Cc:
>> >Sent: 2022/5/25周三 09:01 (GMT+08:00)
>> >Subject: Re: [discuss] provide array result
>> >
>> >Thanks for the proposal - my immediate reaction is that this changes the
>> >programming model in a potentially disruptive way. The function signature
>> >now is dictionary (string->json) in and same type out. This allows
>> >functions to be composed in a sequence because all functions have a
>> uniform
>> >signature. If you admit a different kind of return, the signature of the
>> >function changes. Why stop at array, why not any other type? And how does
>> >composition work in that case?
>> >
>> >I'm not saying we shouldn't consider this but rather that it's a bit more
>> >involved.
>> >
>> >
>> >On Tue, May 24, 2022 at 8:50 PM 甯尤刚 <ni...@navercorp.com> wrote:
>> >
>> >> Hi, guys
>> >>
>> >> Currently, openwhisk supports return json object only,
>> >> I wrote a POEM in openwhisk upstream that allows user to write their
>> own
>> >> action which supports array result. e.g. already implmeneted in my
>> local
>> >> env for python, worked well
>> >> ```
>> >> def main(args):
>> >>
>> >>    try:
>> >>
>> >>        name = args.get("name", "World")
>> >>
>> >>        place  = args.get("place", "Naver")
>> >>
>> >>        return ["a", "b"]
>> >>
>> >>    except Exception as e:
>> >>
>> >>        # please handle exception here
>> >>
>> >>        # you can return or hide an error message
>> >>
>> >>        return {"error": str(e)}
>> >> ```
>> >>
>> >> So the result will support object and array both in future.
>> >> refer to: https://github.com/apache/openwhisk/pull/5244
>> >>
>> >> Welcome to give suggestion
>> >>
>> >>
>> >>
>> >> 此电子邮件及其包含的信息将仅发送给上面列出的收件人，必须加以保护，并且可能包含法律或其他原因禁止披露的信息。
>> >> 如果您不是此电子邮件的预期收件人，未经许可，您不得存储、复制、发送、分发或披露它。 禁止存储、复制、发送、分发或披露电子邮件的任何部分。
>> >> 如果此电子邮件发送不正确，请立即联系 NAVER Security(dl_naversecurity@navercorp.com
>> >> )。然后删除所有原件、副本和附件。谢谢您的合作。
>> >> ​
>> >> This email and the information contained in this email are intended
>> solely
>> >> for the recipient(s) addressed above and may contain information that
>> is
>> >> confidential and/or privileged or whose disclosure is prohibited by
>> law or
>> >> other reasons.
>> >> If you are not the intended recipient of this email, please be advised
>> >> that any unauthorized storage, duplication, dissemination,
>> distribution or
>> >> disclosure of all or part of this email is strictly prohibited.
>> >> If you received this email in error, please immediately contact NAVER
>> >> Security (dl_naversecurity@navercorp.com) and delete this email and
>> any
>> >> copies and attachments from your system. Thank you for your
>> cooperation.​
>> >>
>>
>