You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openwhisk.apache.org by OpenWhisk Team Slack <ra...@apache.org> on 2019/11/09 09:20:35 UTC
[slack-digest] [2019-11-08] #general
2019-11-08 00:39:59 UTC - Rodric Rabbah: Maybe restart your deployment
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573173599217700?thread_ts=1573156915.217400&cid=C3TPCAQG1
----
2019-11-08 00:40:05 UTC - Rodric Rabbah: Did you have a restart?
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573173605218100?thread_ts=1573156915.217400&cid=C3TPCAQG1
----
2019-11-08 00:40:25 UTC - Rodric Rabbah: Is the engine doing this orchestration external?
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573173625218600?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 00:40:39 UTC - Rodric Rabbah: @Olivier Tardieu might have some thoughts
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573173639219100?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 00:40:50 UTC - Rodric Rabbah: Welcome to the community.
pray : Houman Kamali
grin : Houman Kamali
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573173650219500?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 00:46:04 UTC - Olivier Tardieu: for hour-long intervals I would use triggers
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573173964219800?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 00:47:25 UTC - Olivier Tardieu: then you either need to create lots of triggers or you need some periodic job + an external store to keep track of what went wrong
+1 : Houman Kamali
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573174045220000?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 00:48:18 UTC - Olivier Tardieu: to capture the parameters of failed runs and post them to an external service you can use conductor actions or composer to avoid having to modify to code of the action itself
+1 : Houman Kamali
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573174098220200?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 00:50:35 UTC - Olivier Tardieu: be design the input params of an action are not logged or stored in the activation record as these may be sensitive
+1 : Houman Kamali
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573174235220400?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:23:51 UTC - Houman Kamali: @Olivier Tardieu Thanks for the info, interesting about the reason for not storing the activation record.
For capturing the parameters using conductor, what happens if the conductor action fails? I'm asking cause we used to have some errors where the function wouldn't even be executed (it would just log an unexpected error happened); we thought possibly it was because we (we as in OpenWhisk Package Kafka) was sending too many messages and it was overshooting some kind of limit, perhaps message size or whatever; wouldn't such an issue just cascade from the composed action to the sequence action?
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573176231220600?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:25:20 UTC - Olivier Tardieu: sequence and conductor actions are throttled like everything else
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573176320221200?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:25:27 UTC - Olivier Tardieu: there is no execution guarantee
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573176327221400?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:25:52 UTC - Houman Kamali: So there's a possibility that some errors might be silently lost?
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573176352221600?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:26:01 UTC - Olivier Tardieu: yes
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573176361221800?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:26:45 UTC - Houman Kamali: I was thinking, since OpenWhisk internally has persistence (couchDB) and persistence queueing (Kafka), wouldn't it be possible to implement a reliable retry mechanism inside the OpenWhisk itself?
(I'm assuming Kafka and DB in an OpenWhisk will be persistent say, as stateful deployment)
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573176405222000?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:28:18 UTC - Olivier Tardieu: other pieces of openwhisk are not persistent
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573176498222300?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:29:00 UTC - Olivier Tardieu: for instance once an invocation is read from kafka, it may fail execute and be lost
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573176540222500?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:29:49 UTC - Olivier Tardieu: in principle yes openwhisk could be switched from an at-most-once semantics to an at-least-once semantics but not without a significant effort/reengineering
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573176589222700?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:31:34 UTC - Houman Kamali: How much effort? I have a project for which I need to implement a rather sophisticated retry mechanism using kafka with a DLQ (basically for implementing the architecture you described for storing the events and their corresponding logs) anyways
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573176694222900?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:33:41 UTC - Olivier Tardieu: I would say a major effort… but @Rodric Rabbah and others may know better
pray : Houman Kamali
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573176821223300?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:33:54 UTC - Houman Kamali: It is true that a retry policy is too general a concept to be a core feature for OpenWhisk, but I can't think of a possible implementation of a fail proof retry mechanism where messages are tried at least once without modifying how the invoker moves its kafka cursor... (Hm... Maybe an external service that tracks all the messages that are sent to OpenWhisk and then clears them only after it receives a response from each action/sequence?)
I was thinking the policies could be defined at the composer level and OpenWhisk just provides the very basic building blocks, say for creating queues and an internal scheduler
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573176834223500?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:38:56 UTC - Houman Kamali: Thanks a lot for your input @Olivier Tardieu, definitely clears up the possible actions we can take!
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573177136224100?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:47:58 UTC - Waqar Muhammad: Hi
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573177678224600
----
2019-11-08 01:48:19 UTC - Waqar Muhammad: How can i create docker action with already compiled docker image
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573177699225300
----
2019-11-08 01:50:10 UTC - Olivier Tardieu: yes right now I think you would need an external store, then a periodic trigger could be used to do the retry/garbage collection
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573177810225400?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:50:56 UTC - Olivier Tardieu: while there is no guarantee that the periodic trigger will always work, almost always is good enough in this situation
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573177856226200?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:51:25 UTC - Rodric Rabbah: at most once was a bad idea.
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573177885226500?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:51:56 UTC - Olivier Tardieu: one obvious limitation of this scheme is scalability…
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573177916226700?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:52:52 UTC - Olivier Tardieu: + it is not as serverless (someone has to pay for this store)
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573177972227300?thread_ts=1573153626.214900&cid=C3TPCAQG1
----
2019-11-08 01:53:50 UTC - Jiang PengCheng: you can refer to <https://github.com/apache/openwhisk/blob/master/docs/actions-docker.md>, and the image needs to implement the [action-interface](<https://github.com/apache/openwhisk/blob/master/docs/actions-new.md#action-interface>)
+1 : Dominic Kim, Waqar Muhammad
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573178030228100
----
2019-11-08 01:54:21 UTC - Waqar Muhammad: Thanks got it
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573178061228500
----
2019-11-08 03:45:28 UTC - Ali Tariq: we exchanged direct msges, she was able to get it running.
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573184728228700?thread_ts=1573156915.217400&cid=C3TPCAQG1
----
2019-11-08 08:56:01 UTC - Dominic Kim: Are there any references about how many tests(unit, system, performance, ...) do we have now?
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573203361229600
----
2019-11-08 09:20:20 UTC - Bill Zong: Not so far
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573204820230200
----
2019-11-08 09:21:55 UTC - chetanm: You can get some data from Gradle scan reports like this <https://scans.gradle.com/s/3vnprrs35czbo/tests> … which are logged after each gradle run. So you would need to collect for all jobs
Current build also uploads the junit report which has stats but looks like log upload is not working since long time
+1 : Dominic Kim
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573204915231400
----
2019-11-08 10:21:24 UTC - Rodric Rabbah: We can also do a count using ci tool for tests that run in Travis
+1 : Dominic Kim
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573208484232600
----
2019-11-08 11:04:38 UTC - Dominic Kim: Hm i tried to find it via Travis log but could not find it.
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573211078233300
----
2019-11-08 11:04:56 UTC - Dominic Kim: Anyway I figured it out via test result report.
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573211096233900
----
2019-11-08 11:05:03 UTC - Dominic Kim: Thanks
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573211103234200
----
2019-11-08 11:07:44 UTC - chetanm: So how many tests are there now
ear : Rodric Rabbah
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1573211264234800
----