Updating Package and Language versions for a kind

[Rob Allen <ro...@akrabat.com>]

Hi all,

On the PHP PR, @rr [commented] [1]:

> The built in packages are convenient - less zip files for the initial ramp up. But it creates a maintenance issue: when do you pick up updates to the packages (minor/patch level only?) and not break existing actions using the "kind". That is: is the kind itself semantically versioned?

This applies to all kinds and so probably should be discussed project level and ideally we should document how this is handled.

There are two things here:

1. The language runtimes release patch updates for minor versions. e.g. PHP `7.1.7` will become `7.1.8` next month with a small number of bug fixes including crashers and possibly security fixes.

2. Each kind bindles a number of packages via the language's standard package management system: Swift Package Manager for Swift, NPM for NodeJs, etc. The projects that produce these packages update them with new versions minor and patch versions.

The tension is obviously between keeping updated for fixes vs the risk of breaks due to a project's inability to keep BC between patch versions. e.g. the NodeJS kind comes with the `async v2.1.4` package. However `v2.1.5` of that package fixes a stack overflow issue. Should our actions have that fix? Closer to home, the NodeJS kind ships with `OpenWhisk v3.3.2`, but the latest is `v3.6.0` which is needed for non-experimental API Gateway support… 

Some questions:

1. Should we update the language runtime for a kind for a patch level change (e.g. update the current NodeJS:6 kind from `6.9.1` to the latest `6.9.5`?
2. Should we ever update the language runtime for a kind for a minor level change (e.g. update the current NodeJS:6 kind from `6.9.1` to the latest `6.11.1`?
3. Should we ever update the packages in a kind to the latest patch level or minor level?
4. What's our policy when a security issue is published for a language or a package that we ship in a non-deprecated kind?

Whatever the answers are, I think we should document them clearly somewhere.


Also, I've started this conversation as a mailing list topic as it's a "policy" thing. Given my previous comments on mailing lists, should I also create a GitHub issue prefixed with "Discussion" to provide more visibility in order to garner wider community input?


Regards,

Rob...

[1]: https://github.com/apache/incubator-openwhisk/pull/2415#issuecomment-314716101 <https://github.com/apache/incubator-openwhisk/pull/2415#issuecomment-314716101>

Re: Updating Package and Language versions for a kind

[Rob Allen <ro...@akrabat.com>]

Yeah, this is my thinking too.

Regards,

Rob...

> On 27 Jul 2017, at 16:20, Carlos Santana <cs...@gmail.com> wrote:
> 
> Being thinking more about this, and having a core set of packages included
> by default is very useful and I think critical for open source integration
> and continues integration in the open.
> Also keeping large actions away from the system for folks that just want to
> get started and kick the tires using the embedded packages
> 
> I think we could have a mid way alternative:
> 
> What about if we create new kinds every time we want to pickup an update
> for new language update but at the minor (y) level (version x.y.z).
> Also include a very small set of core packages, for example request for
> nodejs very stable and popular package
> 
> We create a new kind to pick up a new minor update and at that time will
> pickup the latest version of the packages.
> For patch updates, at least for security updates this are done at the patch
> level, will update the current kind to update the patch level of the
> language.
> 
> So I think this aligns with what Rob mentioned about trying to keep kinds
> versions at the minor level.
> 
> Taking nodejs as an example:
> we have nodejs:6 (this is set to 6.9)
> let's create a new kind for node nodejs:6.11
> This will include latest version of 6.11 and latest versions of packages at
> the time of creating of the new kind nodejs:6.11
> If there are nodejs patch updates to pickup for 6.11 we update without
> creating a new kind will be kept at nodejs:6.11 with the packages pinned to
> the versions that they shipped.
> I think nodejs moves a bit faster than the other languages so we could
> pickup  a new minor nodejs:6.12 or 6.14 when ever the community thinks
> there is a minor update from nodejs worth the update to create a new kind.
> 
> There might be a gap, I think it's OK for those corner cases that an user
> wants to use the latest nodejs version but use old version of the packages,
> or use the latest version of the package but an old version of nodejs. I
> think for this cases user would need to pinned and include the exact
> version of the modules to include and that's OK, I think this will be a
> normal thing for people going into production and pinning and controlling
> dependencies.
> 
> I think it's hard to come up with silver bullet for all languages, as every
> language updates at a different speed, and dependencies are more slowest
> pace than others.
> 
> 
> 
> 
> 
> 
> 
>> On Wed, Jul 26, 2017 at 8:30 PM Carlos Santana <cs...@gmail.com> wrote:
>> 
>> One thought I have being debating is that the open source project doesn't
>> include any packages going forward.
>> The project will have an image that is small and common denominator that
>> the operators deploying decide which packages to add on top.
>> 
>> For example for nodejs 8, we create the kind nodejs:8 (at LTS Oct/2017),
>> 
>> Will keep the version of language nodejs 8 up to date to the latest as
>> official releases come out from nodejs
>> Operators like IBM and Adobe, RH, others, can base their custom images for
>> their offerings on the same base image, adding packages that they want base
>> on their offering.
>> 
>> Same for swift new kind swift:4, and so on.
>> 
>> This will be a general policy, as every language has their peculiarity and
>> probably we need our runtimes SMEs to talk about the risk on updating or
>> not updating per language.
>> 
>> dealing with user level packages is always a pickle for example today we
>> have nodejs:6 and we have a set of users that benefit from updating the
>> packages to the latest latest, and could have users that they depend on a
>> current version of the package and updating the package on them could cause
>> problems. If I had my way I would say thought luck for using the embedded
>> package and not packaging or doing a bundle build, but serverless is on
>> baby years of adoption and I don't want to be thought yet :-)
>> 
>> So for nodejs8 we say thought luck no embedded packages, but will do
>> something to make it super easy will have the system help you build/compile
>> your action and include your dependencies, these system actions that allow
>> users to provide only their code/git-url, and package.json/package.swift,
>> and these actions will compile/npm install etc..  and install your action
>> including the declared dependencies.
>> 
>> 
>> 
>> 
>> 
>> On Tue, Jul 18, 2017 at 4:08 AM Michael Marth <mm...@adobe.com.invalid>
>> wrote:
>> 
>>> Hi Rob,
>>> 
>>> Re “I wonder how many deployments roll their own action containers?”
>>> 
>>> I spoke too general (implying what most OW deployments would want). What
>>> I can say is that our team wishes to be in control of the libraries that go
>>> into the action containers, that’s why we need to roll our own.
>>> 
>>> Agree with the approach you have outlined on patch and minor releases.
>>> 
>>> The topic of deprecation IMHO is the same discussion as when to upgrade
>>> language runtimes in action containers: at least for us the deprecation
>>> policy is something that we need to be in control of (which is a given as
>>> we roll our own containers).
>>> +1 to the deprecation tooling you hint at regarding not creating actions
>>> for deprecated containers (this is needed regardless of any deprecation
>>> policies). This is not covered by the issues you linked to, is it?
>>> 
>>> Cheers
>>> Michael
>>> 
>>> 
>>> 
>>>> On 14/07/17 19:08, "Rob Allen" <ro...@akrabat.com> wrote:
>>>> 
>>>> Hi Michael, all,
>>>> 
>>>> I wonder how many deployments roll their own action containers? I
>>> suspect not many.
>>>> 
>>>> I agree with you both that it would be nice to keep packages (modules)
>>> up to date on a regular basis within a kind makes the most sense and a
>>> regular schedule would be nice, especially if there's a documented way to
>>> do the update for each kind.
>>>> 
>>>> I think we should also keep the language of the kind up to date at the
>>> *patch* level on the same cadence too. e.g. Node 6.11.0 to Node 6.11.1 -
>>> unless there are language kinds that break semver at the patch level? (In
>>> which case my head may explode!)
>>>> 
>>>> For minor releases, it's a little more wooly at the moment at least. For
>>> NodeJS, [PR 2078  [1] simply updates the `node6` kind from 6.9.1 to 6.11.0
>>> without creating a new new "kind". However, for Swift, [PR] [2120] gave us
>>> a new `swift:3.1.1` kind rather than updating the `swift3` kind which runs
>>> Swift 3.0. I'm not sure that Swift 3.1.2 will need its own kind though, so
>>> maybe it should have been called `swift:3.1`?
>>>> 
>>>> My thoughts on this is that we should create new kinds for minor
>>> language releases on the principle of least surprise for the container
>>> users.
>>>> 
>>>> I would like to fairly aggressively deprecate old ones to stop new
>>> actions being created for out-of-date language versions, but as deprecating
>>> a kind prevents invoking any actions for that kind that's not possible
>>> unless some work is done to allow invoking actions on a deprecated kind,
>>> but not creating or updating them would be necessary for that to be a
>>> possibility.
>>>> 
>>>> I think GitHub Issue [319] [3] is relevant in terms of the tooling
>>> around this.
>>>> 
>>>> 
>>>> Regards,
>>>> 
>>>> Rob...
>>>> 
>>>> [1]: https://github.com/apache/incubator-openwhisk/pull/2078
>>>> [2]: https://github.com/apache/incubator-openwhisk/pull/2120
>>>> [3]: https://github.com/apache/incubator-openwhisk/issues/319
>>>> 
>>>> 
>>>>> On 14 Jul 2017, at 16:27, Michael Marth <mm...@adobe.com.INVALID>
>>> wrote:
>>>>> 
>>>>> Rob, all,
>>>>> 
>>>>> I largely echo what James wrote below (except moving this discussion
>>> off-list :) )
>>>>> 
>>>>> Incidentally, we had a very similar discussion within my team recently
>>> on how to handle this topic for our OW deployment. The bottom line is that
>>> the trade-off between stability and being up-to-date can only be decided in
>>> a given business context of the OW deployment.
>>>>> We (as a project) do not have that context and cannot really make the
>>> decision, IMHO. In that light the only reasonable policy in my view is to
>>> strive to keep everything updated as best as we can, like monthly or every
>>> second month, etc.
>>>>> If we miss a beat (i.e. don’t immediately update to latest release) I
>>> don’t think that’d be critical. Like James said, most OW providers will
>>> have their own runtimes anyway and would (should) take the latest if they
>>> need it.
>>>>> 
>>>>> Cheers
>>>>> Michael
>>>>> 
>>>>> 
>>>>>> On 13/07/17 11:02, "James Thomas" <jt...@gmail.com> wrote:
>>>>>> 
>>>>>> Good ideas Rob. I had a similar issue when looking at the Swift
>>> runtime
>>>>>> recently.
>>>>>> 
>>> https://lists.apache.org/thread.html/fa01baca7d97d08c855abfc69fc17a23e038115fcfc4f2a31d650fa1@%3Cdev.openwhisk.apache.org%3E
>>>>>> 
>>>>>> Would it be possible to have a scheduled upgrade process for installed
>>>>>> modules? Once every four, six or eight weeks? If OpenWhisk did start
>>> to
>>>>>> produce "releases", it could tie in with that.
>>>>>> 
>>>>>> I'd guess that most people using the built-in packages are more
>>> kicking the
>>>>>> tires than building production apps. Once you start being a
>>> production app,
>>>>>> you'll want to explicitly bundle and control your app dependencies.
>>> I'd +1
>>>>>> on being more aggressive with upgrading module versions.
>>>>>> 
>>>>>> I'd like to have a Github issue to follow for this, I find it easier
>>> than
>>>>>> the mailing list.
>>>>>> 
>>>>>>> On 13 July 2017 at 09:33, Rob Allen <ro...@akrabat.com> wrote:
>>>>>>> 
>>>>>>> Hi all,
>>>>>>> 
>>>>>>> On the PHP PR, @rr [commented] [1]:
>>>>>>> 
>>>>>>>> The built in packages are convenient - less zip files for the
>>> initial
>>>>>>> ramp up. But it creates a maintenance issue: when do you pick up
>>> updates to
>>>>>>> the packages (minor/patch level only?) and not break existing
>>> actions using
>>>>>>> the "kind". That is: is the kind itself semantically versioned?
>>>>>>> 
>>>>>>> This applies to all kinds and so probably should be discussed project
>>>>>>> level and ideally we should document how this is handled.
>>>>>>> 
>>>>>>> There are two things here:
>>>>>>> 
>>>>>>> 1. The language runtimes release patch updates for minor versions.
>>> e.g.
>>>>>>> PHP `7.1.7` will become `7.1.8` next month with a small number of
>>> bug fixes
>>>>>>> including crashers and possibly security fixes.
>>>>>>> 
>>>>>>> 2. Each kind bindles a number of packages via the language's standard
>>>>>>> package management system: Swift Package Manager for Swift, NPM for
>>> NodeJs,
>>>>>>> etc. The projects that produce these packages update them with new
>>> versions
>>>>>>> minor and patch versions.
>>>>>>> 
>>>>>>> The tension is obviously between keeping updated for fixes vs the
>>> risk of
>>>>>>> breaks due to a project's inability to keep BC between patch
>>> versions. e.g.
>>>>>>> the NodeJS kind comes with the `async v2.1.4` package. However
>>> `v2.1.5` of
>>>>>>> that package fixes a stack overflow issue. Should our actions have
>>> that
>>>>>>> fix? Closer to home, the NodeJS kind ships with `OpenWhisk v3.3.2`,
>>> but the
>>>>>>> latest is `v3.6.0` which is needed for non-experimental API Gateway
>>> support…
>>>>>>> 
>>>>>>> Some questions:
>>>>>>> 
>>>>>>> 1. Should we update the language runtime for a kind for a patch level
>>>>>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the
>>> latest
>>>>>>> `6.9.5`?
>>>>>>> 2. Should we ever update the language runtime for a kind for a minor
>>> level
>>>>>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the
>>> latest
>>>>>>> `6.11.1`?
>>>>>>> 3. Should we ever update the packages in a kind to the latest patch
>>> level
>>>>>>> or minor level?
>>>>>>> 4. What's our policy when a security issue is published for a
>>> language or
>>>>>>> a package that we ship in a non-deprecated kind?
>>>>>>> 
>>>>>>> Whatever the answers are, I think we should document them clearly
>>>>>>> somewhere.
>>>>>>> 
>>>>>>> 
>>>>>>> Also, I've started this conversation as a mailing list topic as it's
>>> a
>>>>>>> "policy" thing. Given my previous comments on mailing lists, should
>>> I also
>>>>>>> create a GitHub issue prefixed with "Discussion" to provide more
>>> visibility
>>>>>>> in order to garner wider community input?
>>>>>>> 
>>>>>>> 
>>>>>>> Regards,
>>>>>>> 
>>>>>>> Rob...
>>>>>>> 
>>>>>>> [1]: https://github.com/apache/incubator-openwhisk/pull/2415#
>>>>>>> issuecomment-314716101 <https://github.com/apache/
>>>>>>> incubator-openwhisk/pull/2415#issuecomment-314716101>
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Regards,
>>>>>> James Thomas
>>>> 
>>>> --
>>>> Development thoughts at http://akrabat.com
>>>> Daily Jotter for macOS at http://dailyjotter.com
>>>> 
>>> 
>> 

Re: Updating Package and Language versions for a kind

[Carlos Santana <cs...@gmail.com>]

Express is there because of the nodejs proxy needs it to implement the web
server on port 8080 for the /init and /run
https://github.com/apache/incubator-openwhisk/blob/master/core/nodejsActionBase/app.js#L24

express is very useful because allows you to build/re-use express apps
using a web action minus the port listening part. [1], [2], [3]

[1] https://www.npmjs.com/package/expressjs-openwhisk
[2]
https://medium.com/openwhisk/deploying-express-js-apps-to-openwhisk-part-1-9133ba5f262c
[3]
https://medium.com/openwhisk/deploying-express-js-apps-to-openwhisk-part-2-dd8fc51e8944




On Fri, Jul 28, 2017 at 12:33 PM James Thomas <jt...@gmail.com> wrote:

> That sounds good. I've been thinking about reviewing the existing packages
> and trying to propose a new list based upon real-world usage and relevance.
> Looking at the current Node.js list:
>
> https://github.com/apache/incubator-openwhisk/blob/master/core/nodejs6Action/Dockerfile#L33
>
> It has packages like "express" amongst others which I can't see a use for?
>
> On 27 July 2017 at 15:20, Carlos Santana <cs...@gmail.com> wrote:
>
> > Being thinking more about this, and having a core set of packages
> included
> > by default is very useful and I think critical for open source
> integration
> > and continues integration in the open.
> > Also keeping large actions away from the system for folks that just want
> to
> > get started and kick the tires using the embedded packages
> >
> > I think we could have a mid way alternative:
> >
> > What about if we create new kinds every time we want to pickup an update
> > for new language update but at the minor (y) level (version x.y.z).
> > Also include a very small set of core packages, for example request for
> > nodejs very stable and popular package
> >
> > We create a new kind to pick up a new minor update and at that time will
> > pickup the latest version of the packages.
> > For patch updates, at least for security updates this are done at the
> patch
> > level, will update the current kind to update the patch level of the
> > language.
> >
> > So I think this aligns with what Rob mentioned about trying to keep kinds
> > versions at the minor level.
> >
> > Taking nodejs as an example:
> > we have nodejs:6 (this is set to 6.9)
> > let's create a new kind for node nodejs:6.11
> > This will include latest version of 6.11 and latest versions of packages
> at
> > the time of creating of the new kind nodejs:6.11
> > If there are nodejs patch updates to pickup for 6.11 we update without
> > creating a new kind will be kept at nodejs:6.11 with the packages pinned
> to
> > the versions that they shipped.
> > I think nodejs moves a bit faster than the other languages so we could
> > pickup  a new minor nodejs:6.12 or 6.14 when ever the community thinks
> > there is a minor update from nodejs worth the update to create a new
> kind.
> >
> > There might be a gap, I think it's OK for those corner cases that an user
> > wants to use the latest nodejs version but use old version of the
> packages,
> > or use the latest version of the package but an old version of nodejs. I
> > think for this cases user would need to pinned and include the exact
> > version of the modules to include and that's OK, I think this will be a
> > normal thing for people going into production and pinning and controlling
> > dependencies.
> >
> > I think it's hard to come up with silver bullet for all languages, as
> every
> > language updates at a different speed, and dependencies are more slowest
> > pace than others.
> >
> >
> >
> >
> >
> >
> >
> > On Wed, Jul 26, 2017 at 8:30 PM Carlos Santana <cs...@gmail.com>
> > wrote:
> >
> > > One thought I have being debating is that the open source project
> doesn't
> > > include any packages going forward.
> > > The project will have an image that is small and common denominator
> that
> > > the operators deploying decide which packages to add on top.
> > >
> > > For example for nodejs 8, we create the kind nodejs:8 (at LTS
> Oct/2017),
> > >
> > > Will keep the version of language nodejs 8 up to date to the latest as
> > > official releases come out from nodejs
> > > Operators like IBM and Adobe, RH, others, can base their custom images
> > for
> > > their offerings on the same base image, adding packages that they want
> > base
> > > on their offering.
> > >
> > > Same for swift new kind swift:4, and so on.
> > >
> > > This will be a general policy, as every language has their peculiarity
> > and
> > > probably we need our runtimes SMEs to talk about the risk on updating
> or
> > > not updating per language.
> > >
> > > dealing with user level packages is always a pickle for example today
> we
> > > have nodejs:6 and we have a set of users that benefit from updating the
> > > packages to the latest latest, and could have users that they depend
> on a
> > > current version of the package and updating the package on them could
> > cause
> > > problems. If I had my way I would say thought luck for using the
> embedded
> > > package and not packaging or doing a bundle build, but serverless is on
> > > baby years of adoption and I don't want to be thought yet :-)
> > >
> > > So for nodejs8 we say thought luck no embedded packages, but will do
> > > something to make it super easy will have the system help you
> > build/compile
> > > your action and include your dependencies, these system actions that
> > allow
> > > users to provide only their code/git-url, and
> package.json/package.swift,
> > > and these actions will compile/npm install etc..  and install your
> action
> > > including the declared dependencies.
> > >
> > >
> > >
> > >
> > >
> > > On Tue, Jul 18, 2017 at 4:08 AM Michael Marth <mmarth@adobe.com.invalid
> >
> > > wrote:
> > >
> > >> Hi Rob,
> > >>
> > >> Re “I wonder how many deployments roll their own action containers?”
> > >>
> > >> I spoke too general (implying what most OW deployments would want).
> What
> > >> I can say is that our team wishes to be in control of the libraries
> > that go
> > >> into the action containers, that’s why we need to roll our own.
> > >>
> > >> Agree with the approach you have outlined on patch and minor releases.
> > >>
> > >> The topic of deprecation IMHO is the same discussion as when to
> upgrade
> > >> language runtimes in action containers: at least for us the
> deprecation
> > >> policy is something that we need to be in control of (which is a given
> > as
> > >> we roll our own containers).
> > >> +1 to the deprecation tooling you hint at regarding not creating
> actions
> > >> for deprecated containers (this is needed regardless of any
> deprecation
> > >> policies). This is not covered by the issues you linked to, is it?
> > >>
> > >> Cheers
> > >> Michael
> > >>
> > >>
> > >>
> > >> On 14/07/17 19:08, "Rob Allen" <ro...@akrabat.com> wrote:
> > >>
> > >> >Hi Michael, all,
> > >> >
> > >> >I wonder how many deployments roll their own action containers? I
> > >> suspect not many.
> > >> >
> > >> >I agree with you both that it would be nice to keep packages
> (modules)
> > >> up to date on a regular basis within a kind makes the most sense and a
> > >> regular schedule would be nice, especially if there's a documented way
> > to
> > >> do the update for each kind.
> > >> >
> > >> >I think we should also keep the language of the kind up to date at
> the
> > >> *patch* level on the same cadence too. e.g. Node 6.11.0 to Node
> 6.11.1 -
> > >> unless there are language kinds that break semver at the patch level?
> > (In
> > >> which case my head may explode!)
> > >> >
> > >> >For minor releases, it's a little more wooly at the moment at least.
> > For
> > >> NodeJS, [PR 2078  [1] simply updates the `node6` kind from 6.9.1 to
> > 6.11.0
> > >> without creating a new new "kind". However, for Swift, [PR] [2120]
> gave
> > us
> > >> a new `swift:3.1.1` kind rather than updating the `swift3` kind which
> > runs
> > >> Swift 3.0. I'm not sure that Swift 3.1.2 will need its own kind
> though,
> > so
> > >> maybe it should have been called `swift:3.1`?
> > >> >
> > >> >My thoughts on this is that we should create new kinds for minor
> > >> language releases on the principle of least surprise for the container
> > >> users.
> > >> >
> > >> >I would like to fairly aggressively deprecate old ones to stop new
> > >> actions being created for out-of-date language versions, but as
> > deprecating
> > >> a kind prevents invoking any actions for that kind that's not possible
> > >> unless some work is done to allow invoking actions on a deprecated
> kind,
> > >> but not creating or updating them would be necessary for that to be a
> > >> possibility.
> > >> >
> > >> >I think GitHub Issue [319] [3] is relevant in terms of the tooling
> > >> around this.
> > >> >
> > >> >
> > >> >Regards,
> > >> >
> > >> >Rob...
> > >> >
> > >> >[1]: https://github.com/apache/incubator-openwhisk/pull/2078
> > >> >[2]: https://github.com/apache/incubator-openwhisk/pull/2120
> > >> >[3]: https://github.com/apache/incubator-openwhisk/issues/319
> > >> >
> > >> >
> > >> >> On 14 Jul 2017, at 16:27, Michael Marth <mm...@adobe.com.INVALID>
> > >> wrote:
> > >> >>
> > >> >> Rob, all,
> > >> >>
> > >> >> I largely echo what James wrote below (except moving this
> discussion
> > >> off-list :) )
> > >> >>
> > >> >> Incidentally, we had a very similar discussion within my team
> > recently
> > >> on how to handle this topic for our OW deployment. The bottom line is
> > that
> > >> the trade-off between stability and being up-to-date can only be
> > decided in
> > >> a given business context of the OW deployment.
> > >> >> We (as a project) do not have that context and cannot really make
> the
> > >> decision, IMHO. In that light the only reasonable policy in my view is
> > to
> > >> strive to keep everything updated as best as we can, like monthly or
> > every
> > >> second month, etc.
> > >> >> If we miss a beat (i.e. don’t immediately update to latest
> release) I
> > >> don’t think that’d be critical. Like James said, most OW providers
> will
> > >> have their own runtimes anyway and would (should) take the latest if
> > they
> > >> need it.
> > >> >>
> > >> >> Cheers
> > >> >> Michael
> > >> >>
> > >> >>
> > >> >> On 13/07/17 11:02, "James Thomas" <jt...@gmail.com> wrote:
> > >> >>
> > >> >>> Good ideas Rob. I had a similar issue when looking at the Swift
> > >> runtime
> > >> >>> recently.
> > >> >>>
> > >> https://lists.apache.org/thread.html/fa01baca7d97d08c855abfc69fc17a
> > 23e038115fcfc4f2a31d650fa1@%3Cdev.openwhisk.apache.org%3E
> > >> >>>
> > >> >>> Would it be possible to have a scheduled upgrade process for
> > installed
> > >> >>> modules? Once every four, six or eight weeks? If OpenWhisk did
> start
> > >> to
> > >> >>> produce "releases", it could tie in with that.
> > >> >>>
> > >> >>> I'd guess that most people using the built-in packages are more
> > >> kicking the
> > >> >>> tires than building production apps. Once you start being a
> > >> production app,
> > >> >>> you'll want to explicitly bundle and control your app
> dependencies.
> > >> I'd +1
> > >> >>> on being more aggressive with upgrading module versions.
> > >> >>>
> > >> >>> I'd like to have a Github issue to follow for this, I find it
> easier
> > >> than
> > >> >>> the mailing list.
> > >> >>>
> > >> >>> On 13 July 2017 at 09:33, Rob Allen <ro...@akrabat.com> wrote:
> > >> >>>
> > >> >>>> Hi all,
> > >> >>>>
> > >> >>>> On the PHP PR, @rr [commented] [1]:
> > >> >>>>
> > >> >>>>> The built in packages are convenient - less zip files for the
> > >> initial
> > >> >>>> ramp up. But it creates a maintenance issue: when do you pick up
> > >> updates to
> > >> >>>> the packages (minor/patch level only?) and not break existing
> > >> actions using
> > >> >>>> the "kind". That is: is the kind itself semantically versioned?
> > >> >>>>
> > >> >>>> This applies to all kinds and so probably should be discussed
> > project
> > >> >>>> level and ideally we should document how this is handled.
> > >> >>>>
> > >> >>>> There are two things here:
> > >> >>>>
> > >> >>>> 1. The language runtimes release patch updates for minor
> versions.
> > >> e.g.
> > >> >>>> PHP `7.1.7` will become `7.1.8` next month with a small number of
> > >> bug fixes
> > >> >>>> including crashers and possibly security fixes.
> > >> >>>>
> > >> >>>> 2. Each kind bindles a number of packages via the language's
> > standard
> > >> >>>> package management system: Swift Package Manager for Swift, NPM
> for
> > >> NodeJs,
> > >> >>>> etc. The projects that produce these packages update them with
> new
> > >> versions
> > >> >>>> minor and patch versions.
> > >> >>>>
> > >> >>>> The tension is obviously between keeping updated for fixes vs the
> > >> risk of
> > >> >>>> breaks due to a project's inability to keep BC between patch
> > >> versions. e.g.
> > >> >>>> the NodeJS kind comes with the `async v2.1.4` package. However
> > >> `v2.1.5` of
> > >> >>>> that package fixes a stack overflow issue. Should our actions
> have
> > >> that
> > >> >>>> fix? Closer to home, the NodeJS kind ships with `OpenWhisk
> v3.3.2`,
> > >> but the
> > >> >>>> latest is `v3.6.0` which is needed for non-experimental API
> Gateway
> > >> support…
> > >> >>>>
> > >> >>>> Some questions:
> > >> >>>>
> > >> >>>> 1. Should we update the language runtime for a kind for a patch
> > level
> > >> >>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the
> > >> latest
> > >> >>>> `6.9.5`?
> > >> >>>> 2. Should we ever update the language runtime for a kind for a
> > minor
> > >> level
> > >> >>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the
> > >> latest
> > >> >>>> `6.11.1`?
> > >> >>>> 3. Should we ever update the packages in a kind to the latest
> patch
> > >> level
> > >> >>>> or minor level?
> > >> >>>> 4. What's our policy when a security issue is published for a
> > >> language or
> > >> >>>> a package that we ship in a non-deprecated kind?
> > >> >>>>
> > >> >>>> Whatever the answers are, I think we should document them clearly
> > >> >>>> somewhere.
> > >> >>>>
> > >> >>>>
> > >> >>>> Also, I've started this conversation as a mailing list topic as
> > it's
> > >> a
> > >> >>>> "policy" thing. Given my previous comments on mailing lists,
> should
> > >> I also
> > >> >>>> create a GitHub issue prefixed with "Discussion" to provide more
> > >> visibility
> > >> >>>> in order to garner wider community input?
> > >> >>>>
> > >> >>>>
> > >> >>>> Regards,
> > >> >>>>
> > >> >>>> Rob...
> > >> >>>>
> > >> >>>> [1]: https://github.com/apache/incubator-openwhisk/pull/2415#
> > >> >>>> issuecomment-314716101 <https://github.com/apache/
> > >> >>>> incubator-openwhisk/pull/2415#issuecomment-314716101>
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >>> --
> > >> >>> Regards,
> > >> >>> James Thomas
> > >> >
> > >> >--
> > >> >Development thoughts at http://akrabat.com
> > >> >Daily Jotter for macOS at http://dailyjotter.com
> > >> >
> > >>
> > >
> >
>
>
>
> --
> Regards,
> James Thomas
>

Re: Updating Package and Language versions for a kind

[James Thomas <jt...@gmail.com>]

That sounds good. I've been thinking about reviewing the existing packages
and trying to propose a new list based upon real-world usage and relevance.
Looking at the current Node.js list:
https://github.com/apache/incubator-openwhisk/blob/master/core/nodejs6Action/Dockerfile#L33

It has packages like "express" amongst others which I can't see a use for?

On 27 July 2017 at 15:20, Carlos Santana <cs...@gmail.com> wrote:

> Being thinking more about this, and having a core set of packages included
> by default is very useful and I think critical for open source integration
> and continues integration in the open.
> Also keeping large actions away from the system for folks that just want to
> get started and kick the tires using the embedded packages
>
> I think we could have a mid way alternative:
>
> What about if we create new kinds every time we want to pickup an update
> for new language update but at the minor (y) level (version x.y.z).
> Also include a very small set of core packages, for example request for
> nodejs very stable and popular package
>
> We create a new kind to pick up a new minor update and at that time will
> pickup the latest version of the packages.
> For patch updates, at least for security updates this are done at the patch
> level, will update the current kind to update the patch level of the
> language.
>
> So I think this aligns with what Rob mentioned about trying to keep kinds
> versions at the minor level.
>
> Taking nodejs as an example:
> we have nodejs:6 (this is set to 6.9)
> let's create a new kind for node nodejs:6.11
> This will include latest version of 6.11 and latest versions of packages at
> the time of creating of the new kind nodejs:6.11
> If there are nodejs patch updates to pickup for 6.11 we update without
> creating a new kind will be kept at nodejs:6.11 with the packages pinned to
> the versions that they shipped.
> I think nodejs moves a bit faster than the other languages so we could
> pickup  a new minor nodejs:6.12 or 6.14 when ever the community thinks
> there is a minor update from nodejs worth the update to create a new kind.
>
> There might be a gap, I think it's OK for those corner cases that an user
> wants to use the latest nodejs version but use old version of the packages,
> or use the latest version of the package but an old version of nodejs. I
> think for this cases user would need to pinned and include the exact
> version of the modules to include and that's OK, I think this will be a
> normal thing for people going into production and pinning and controlling
> dependencies.
>
> I think it's hard to come up with silver bullet for all languages, as every
> language updates at a different speed, and dependencies are more slowest
> pace than others.
>
>
>
>
>
>
>
> On Wed, Jul 26, 2017 at 8:30 PM Carlos Santana <cs...@gmail.com>
> wrote:
>
> > One thought I have being debating is that the open source project doesn't
> > include any packages going forward.
> > The project will have an image that is small and common denominator that
> > the operators deploying decide which packages to add on top.
> >
> > For example for nodejs 8, we create the kind nodejs:8 (at LTS Oct/2017),
> >
> > Will keep the version of language nodejs 8 up to date to the latest as
> > official releases come out from nodejs
> > Operators like IBM and Adobe, RH, others, can base their custom images
> for
> > their offerings on the same base image, adding packages that they want
> base
> > on their offering.
> >
> > Same for swift new kind swift:4, and so on.
> >
> > This will be a general policy, as every language has their peculiarity
> and
> > probably we need our runtimes SMEs to talk about the risk on updating or
> > not updating per language.
> >
> > dealing with user level packages is always a pickle for example today we
> > have nodejs:6 and we have a set of users that benefit from updating the
> > packages to the latest latest, and could have users that they depend on a
> > current version of the package and updating the package on them could
> cause
> > problems. If I had my way I would say thought luck for using the embedded
> > package and not packaging or doing a bundle build, but serverless is on
> > baby years of adoption and I don't want to be thought yet :-)
> >
> > So for nodejs8 we say thought luck no embedded packages, but will do
> > something to make it super easy will have the system help you
> build/compile
> > your action and include your dependencies, these system actions that
> allow
> > users to provide only their code/git-url, and package.json/package.swift,
> > and these actions will compile/npm install etc..  and install your action
> > including the declared dependencies.
> >
> >
> >
> >
> >
> > On Tue, Jul 18, 2017 at 4:08 AM Michael Marth <mm...@adobe.com.invalid>
> > wrote:
> >
> >> Hi Rob,
> >>
> >> Re “I wonder how many deployments roll their own action containers?”
> >>
> >> I spoke too general (implying what most OW deployments would want). What
> >> I can say is that our team wishes to be in control of the libraries
> that go
> >> into the action containers, that’s why we need to roll our own.
> >>
> >> Agree with the approach you have outlined on patch and minor releases.
> >>
> >> The topic of deprecation IMHO is the same discussion as when to upgrade
> >> language runtimes in action containers: at least for us the deprecation
> >> policy is something that we need to be in control of (which is a given
> as
> >> we roll our own containers).
> >> +1 to the deprecation tooling you hint at regarding not creating actions
> >> for deprecated containers (this is needed regardless of any deprecation
> >> policies). This is not covered by the issues you linked to, is it?
> >>
> >> Cheers
> >> Michael
> >>
> >>
> >>
> >> On 14/07/17 19:08, "Rob Allen" <ro...@akrabat.com> wrote:
> >>
> >> >Hi Michael, all,
> >> >
> >> >I wonder how many deployments roll their own action containers? I
> >> suspect not many.
> >> >
> >> >I agree with you both that it would be nice to keep packages (modules)
> >> up to date on a regular basis within a kind makes the most sense and a
> >> regular schedule would be nice, especially if there's a documented way
> to
> >> do the update for each kind.
> >> >
> >> >I think we should also keep the language of the kind up to date at the
> >> *patch* level on the same cadence too. e.g. Node 6.11.0 to Node 6.11.1 -
> >> unless there are language kinds that break semver at the patch level?
> (In
> >> which case my head may explode!)
> >> >
> >> >For minor releases, it's a little more wooly at the moment at least.
> For
> >> NodeJS, [PR 2078  [1] simply updates the `node6` kind from 6.9.1 to
> 6.11.0
> >> without creating a new new "kind". However, for Swift, [PR] [2120] gave
> us
> >> a new `swift:3.1.1` kind rather than updating the `swift3` kind which
> runs
> >> Swift 3.0. I'm not sure that Swift 3.1.2 will need its own kind though,
> so
> >> maybe it should have been called `swift:3.1`?
> >> >
> >> >My thoughts on this is that we should create new kinds for minor
> >> language releases on the principle of least surprise for the container
> >> users.
> >> >
> >> >I would like to fairly aggressively deprecate old ones to stop new
> >> actions being created for out-of-date language versions, but as
> deprecating
> >> a kind prevents invoking any actions for that kind that's not possible
> >> unless some work is done to allow invoking actions on a deprecated kind,
> >> but not creating or updating them would be necessary for that to be a
> >> possibility.
> >> >
> >> >I think GitHub Issue [319] [3] is relevant in terms of the tooling
> >> around this.
> >> >
> >> >
> >> >Regards,
> >> >
> >> >Rob...
> >> >
> >> >[1]: https://github.com/apache/incubator-openwhisk/pull/2078
> >> >[2]: https://github.com/apache/incubator-openwhisk/pull/2120
> >> >[3]: https://github.com/apache/incubator-openwhisk/issues/319
> >> >
> >> >
> >> >> On 14 Jul 2017, at 16:27, Michael Marth <mm...@adobe.com.INVALID>
> >> wrote:
> >> >>
> >> >> Rob, all,
> >> >>
> >> >> I largely echo what James wrote below (except moving this discussion
> >> off-list :) )
> >> >>
> >> >> Incidentally, we had a very similar discussion within my team
> recently
> >> on how to handle this topic for our OW deployment. The bottom line is
> that
> >> the trade-off between stability and being up-to-date can only be
> decided in
> >> a given business context of the OW deployment.
> >> >> We (as a project) do not have that context and cannot really make the
> >> decision, IMHO. In that light the only reasonable policy in my view is
> to
> >> strive to keep everything updated as best as we can, like monthly or
> every
> >> second month, etc.
> >> >> If we miss a beat (i.e. don’t immediately update to latest release) I
> >> don’t think that’d be critical. Like James said, most OW providers will
> >> have their own runtimes anyway and would (should) take the latest if
> they
> >> need it.
> >> >>
> >> >> Cheers
> >> >> Michael
> >> >>
> >> >>
> >> >> On 13/07/17 11:02, "James Thomas" <jt...@gmail.com> wrote:
> >> >>
> >> >>> Good ideas Rob. I had a similar issue when looking at the Swift
> >> runtime
> >> >>> recently.
> >> >>>
> >> https://lists.apache.org/thread.html/fa01baca7d97d08c855abfc69fc17a
> 23e038115fcfc4f2a31d650fa1@%3Cdev.openwhisk.apache.org%3E
> >> >>>
> >> >>> Would it be possible to have a scheduled upgrade process for
> installed
> >> >>> modules? Once every four, six or eight weeks? If OpenWhisk did start
> >> to
> >> >>> produce "releases", it could tie in with that.
> >> >>>
> >> >>> I'd guess that most people using the built-in packages are more
> >> kicking the
> >> >>> tires than building production apps. Once you start being a
> >> production app,
> >> >>> you'll want to explicitly bundle and control your app dependencies.
> >> I'd +1
> >> >>> on being more aggressive with upgrading module versions.
> >> >>>
> >> >>> I'd like to have a Github issue to follow for this, I find it easier
> >> than
> >> >>> the mailing list.
> >> >>>
> >> >>> On 13 July 2017 at 09:33, Rob Allen <ro...@akrabat.com> wrote:
> >> >>>
> >> >>>> Hi all,
> >> >>>>
> >> >>>> On the PHP PR, @rr [commented] [1]:
> >> >>>>
> >> >>>>> The built in packages are convenient - less zip files for the
> >> initial
> >> >>>> ramp up. But it creates a maintenance issue: when do you pick up
> >> updates to
> >> >>>> the packages (minor/patch level only?) and not break existing
> >> actions using
> >> >>>> the "kind". That is: is the kind itself semantically versioned?
> >> >>>>
> >> >>>> This applies to all kinds and so probably should be discussed
> project
> >> >>>> level and ideally we should document how this is handled.
> >> >>>>
> >> >>>> There are two things here:
> >> >>>>
> >> >>>> 1. The language runtimes release patch updates for minor versions.
> >> e.g.
> >> >>>> PHP `7.1.7` will become `7.1.8` next month with a small number of
> >> bug fixes
> >> >>>> including crashers and possibly security fixes.
> >> >>>>
> >> >>>> 2. Each kind bindles a number of packages via the language's
> standard
> >> >>>> package management system: Swift Package Manager for Swift, NPM for
> >> NodeJs,
> >> >>>> etc. The projects that produce these packages update them with new
> >> versions
> >> >>>> minor and patch versions.
> >> >>>>
> >> >>>> The tension is obviously between keeping updated for fixes vs the
> >> risk of
> >> >>>> breaks due to a project's inability to keep BC between patch
> >> versions. e.g.
> >> >>>> the NodeJS kind comes with the `async v2.1.4` package. However
> >> `v2.1.5` of
> >> >>>> that package fixes a stack overflow issue. Should our actions have
> >> that
> >> >>>> fix? Closer to home, the NodeJS kind ships with `OpenWhisk v3.3.2`,
> >> but the
> >> >>>> latest is `v3.6.0` which is needed for non-experimental API Gateway
> >> support…
> >> >>>>
> >> >>>> Some questions:
> >> >>>>
> >> >>>> 1. Should we update the language runtime for a kind for a patch
> level
> >> >>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the
> >> latest
> >> >>>> `6.9.5`?
> >> >>>> 2. Should we ever update the language runtime for a kind for a
> minor
> >> level
> >> >>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the
> >> latest
> >> >>>> `6.11.1`?
> >> >>>> 3. Should we ever update the packages in a kind to the latest patch
> >> level
> >> >>>> or minor level?
> >> >>>> 4. What's our policy when a security issue is published for a
> >> language or
> >> >>>> a package that we ship in a non-deprecated kind?
> >> >>>>
> >> >>>> Whatever the answers are, I think we should document them clearly
> >> >>>> somewhere.
> >> >>>>
> >> >>>>
> >> >>>> Also, I've started this conversation as a mailing list topic as
> it's
> >> a
> >> >>>> "policy" thing. Given my previous comments on mailing lists, should
> >> I also
> >> >>>> create a GitHub issue prefixed with "Discussion" to provide more
> >> visibility
> >> >>>> in order to garner wider community input?
> >> >>>>
> >> >>>>
> >> >>>> Regards,
> >> >>>>
> >> >>>> Rob...
> >> >>>>
> >> >>>> [1]: https://github.com/apache/incubator-openwhisk/pull/2415#
> >> >>>> issuecomment-314716101 <https://github.com/apache/
> >> >>>> incubator-openwhisk/pull/2415#issuecomment-314716101>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Regards,
> >> >>> James Thomas
> >> >
> >> >--
> >> >Development thoughts at http://akrabat.com
> >> >Daily Jotter for macOS at http://dailyjotter.com
> >> >
> >>
> >
>



-- 
Regards,
James Thomas

Re: Updating Package and Language versions for a kind

[Carlos Santana <cs...@gmail.com>]

Being thinking more about this, and having a core set of packages included
by default is very useful and I think critical for open source integration
and continues integration in the open.
Also keeping large actions away from the system for folks that just want to
get started and kick the tires using the embedded packages

I think we could have a mid way alternative:

What about if we create new kinds every time we want to pickup an update
for new language update but at the minor (y) level (version x.y.z).
Also include a very small set of core packages, for example request for
nodejs very stable and popular package

We create a new kind to pick up a new minor update and at that time will
pickup the latest version of the packages.
For patch updates, at least for security updates this are done at the patch
level, will update the current kind to update the patch level of the
language.

So I think this aligns with what Rob mentioned about trying to keep kinds
versions at the minor level.

Taking nodejs as an example:
we have nodejs:6 (this is set to 6.9)
let's create a new kind for node nodejs:6.11
This will include latest version of 6.11 and latest versions of packages at
the time of creating of the new kind nodejs:6.11
If there are nodejs patch updates to pickup for 6.11 we update without
creating a new kind will be kept at nodejs:6.11 with the packages pinned to
the versions that they shipped.
I think nodejs moves a bit faster than the other languages so we could
pickup  a new minor nodejs:6.12 or 6.14 when ever the community thinks
there is a minor update from nodejs worth the update to create a new kind.

There might be a gap, I think it's OK for those corner cases that an user
wants to use the latest nodejs version but use old version of the packages,
or use the latest version of the package but an old version of nodejs. I
think for this cases user would need to pinned and include the exact
version of the modules to include and that's OK, I think this will be a
normal thing for people going into production and pinning and controlling
dependencies.

I think it's hard to come up with silver bullet for all languages, as every
language updates at a different speed, and dependencies are more slowest
pace than others.







On Wed, Jul 26, 2017 at 8:30 PM Carlos Santana <cs...@gmail.com> wrote:

> One thought I have being debating is that the open source project doesn't
> include any packages going forward.
> The project will have an image that is small and common denominator that
> the operators deploying decide which packages to add on top.
>
> For example for nodejs 8, we create the kind nodejs:8 (at LTS Oct/2017),
>
> Will keep the version of language nodejs 8 up to date to the latest as
> official releases come out from nodejs
> Operators like IBM and Adobe, RH, others, can base their custom images for
> their offerings on the same base image, adding packages that they want base
> on their offering.
>
> Same for swift new kind swift:4, and so on.
>
> This will be a general policy, as every language has their peculiarity and
> probably we need our runtimes SMEs to talk about the risk on updating or
> not updating per language.
>
> dealing with user level packages is always a pickle for example today we
> have nodejs:6 and we have a set of users that benefit from updating the
> packages to the latest latest, and could have users that they depend on a
> current version of the package and updating the package on them could cause
> problems. If I had my way I would say thought luck for using the embedded
> package and not packaging or doing a bundle build, but serverless is on
> baby years of adoption and I don't want to be thought yet :-)
>
> So for nodejs8 we say thought luck no embedded packages, but will do
> something to make it super easy will have the system help you build/compile
> your action and include your dependencies, these system actions that allow
> users to provide only their code/git-url, and package.json/package.swift,
> and these actions will compile/npm install etc..  and install your action
> including the declared dependencies.
>
>
>
>
>
> On Tue, Jul 18, 2017 at 4:08 AM Michael Marth <mm...@adobe.com.invalid>
> wrote:
>
>> Hi Rob,
>>
>> Re “I wonder how many deployments roll their own action containers?”
>>
>> I spoke too general (implying what most OW deployments would want). What
>> I can say is that our team wishes to be in control of the libraries that go
>> into the action containers, that’s why we need to roll our own.
>>
>> Agree with the approach you have outlined on patch and minor releases.
>>
>> The topic of deprecation IMHO is the same discussion as when to upgrade
>> language runtimes in action containers: at least for us the deprecation
>> policy is something that we need to be in control of (which is a given as
>> we roll our own containers).
>> +1 to the deprecation tooling you hint at regarding not creating actions
>> for deprecated containers (this is needed regardless of any deprecation
>> policies). This is not covered by the issues you linked to, is it?
>>
>> Cheers
>> Michael
>>
>>
>>
>> On 14/07/17 19:08, "Rob Allen" <ro...@akrabat.com> wrote:
>>
>> >Hi Michael, all,
>> >
>> >I wonder how many deployments roll their own action containers? I
>> suspect not many.
>> >
>> >I agree with you both that it would be nice to keep packages (modules)
>> up to date on a regular basis within a kind makes the most sense and a
>> regular schedule would be nice, especially if there's a documented way to
>> do the update for each kind.
>> >
>> >I think we should also keep the language of the kind up to date at the
>> *patch* level on the same cadence too. e.g. Node 6.11.0 to Node 6.11.1 -
>> unless there are language kinds that break semver at the patch level? (In
>> which case my head may explode!)
>> >
>> >For minor releases, it's a little more wooly at the moment at least. For
>> NodeJS, [PR 2078  [1] simply updates the `node6` kind from 6.9.1 to 6.11.0
>> without creating a new new "kind". However, for Swift, [PR] [2120] gave us
>> a new `swift:3.1.1` kind rather than updating the `swift3` kind which runs
>> Swift 3.0. I'm not sure that Swift 3.1.2 will need its own kind though, so
>> maybe it should have been called `swift:3.1`?
>> >
>> >My thoughts on this is that we should create new kinds for minor
>> language releases on the principle of least surprise for the container
>> users.
>> >
>> >I would like to fairly aggressively deprecate old ones to stop new
>> actions being created for out-of-date language versions, but as deprecating
>> a kind prevents invoking any actions for that kind that's not possible
>> unless some work is done to allow invoking actions on a deprecated kind,
>> but not creating or updating them would be necessary for that to be a
>> possibility.
>> >
>> >I think GitHub Issue [319] [3] is relevant in terms of the tooling
>> around this.
>> >
>> >
>> >Regards,
>> >
>> >Rob...
>> >
>> >[1]: https://github.com/apache/incubator-openwhisk/pull/2078
>> >[2]: https://github.com/apache/incubator-openwhisk/pull/2120
>> >[3]: https://github.com/apache/incubator-openwhisk/issues/319
>> >
>> >
>> >> On 14 Jul 2017, at 16:27, Michael Marth <mm...@adobe.com.INVALID>
>> wrote:
>> >>
>> >> Rob, all,
>> >>
>> >> I largely echo what James wrote below (except moving this discussion
>> off-list :) )
>> >>
>> >> Incidentally, we had a very similar discussion within my team recently
>> on how to handle this topic for our OW deployment. The bottom line is that
>> the trade-off between stability and being up-to-date can only be decided in
>> a given business context of the OW deployment.
>> >> We (as a project) do not have that context and cannot really make the
>> decision, IMHO. In that light the only reasonable policy in my view is to
>> strive to keep everything updated as best as we can, like monthly or every
>> second month, etc.
>> >> If we miss a beat (i.e. don’t immediately update to latest release) I
>> don’t think that’d be critical. Like James said, most OW providers will
>> have their own runtimes anyway and would (should) take the latest if they
>> need it.
>> >>
>> >> Cheers
>> >> Michael
>> >>
>> >>
>> >> On 13/07/17 11:02, "James Thomas" <jt...@gmail.com> wrote:
>> >>
>> >>> Good ideas Rob. I had a similar issue when looking at the Swift
>> runtime
>> >>> recently.
>> >>>
>> https://lists.apache.org/thread.html/fa01baca7d97d08c855abfc69fc17a23e038115fcfc4f2a31d650fa1@%3Cdev.openwhisk.apache.org%3E
>> >>>
>> >>> Would it be possible to have a scheduled upgrade process for installed
>> >>> modules? Once every four, six or eight weeks? If OpenWhisk did start
>> to
>> >>> produce "releases", it could tie in with that.
>> >>>
>> >>> I'd guess that most people using the built-in packages are more
>> kicking the
>> >>> tires than building production apps. Once you start being a
>> production app,
>> >>> you'll want to explicitly bundle and control your app dependencies.
>> I'd +1
>> >>> on being more aggressive with upgrading module versions.
>> >>>
>> >>> I'd like to have a Github issue to follow for this, I find it easier
>> than
>> >>> the mailing list.
>> >>>
>> >>> On 13 July 2017 at 09:33, Rob Allen <ro...@akrabat.com> wrote:
>> >>>
>> >>>> Hi all,
>> >>>>
>> >>>> On the PHP PR, @rr [commented] [1]:
>> >>>>
>> >>>>> The built in packages are convenient - less zip files for the
>> initial
>> >>>> ramp up. But it creates a maintenance issue: when do you pick up
>> updates to
>> >>>> the packages (minor/patch level only?) and not break existing
>> actions using
>> >>>> the "kind". That is: is the kind itself semantically versioned?
>> >>>>
>> >>>> This applies to all kinds and so probably should be discussed project
>> >>>> level and ideally we should document how this is handled.
>> >>>>
>> >>>> There are two things here:
>> >>>>
>> >>>> 1. The language runtimes release patch updates for minor versions.
>> e.g.
>> >>>> PHP `7.1.7` will become `7.1.8` next month with a small number of
>> bug fixes
>> >>>> including crashers and possibly security fixes.
>> >>>>
>> >>>> 2. Each kind bindles a number of packages via the language's standard
>> >>>> package management system: Swift Package Manager for Swift, NPM for
>> NodeJs,
>> >>>> etc. The projects that produce these packages update them with new
>> versions
>> >>>> minor and patch versions.
>> >>>>
>> >>>> The tension is obviously between keeping updated for fixes vs the
>> risk of
>> >>>> breaks due to a project's inability to keep BC between patch
>> versions. e.g.
>> >>>> the NodeJS kind comes with the `async v2.1.4` package. However
>> `v2.1.5` of
>> >>>> that package fixes a stack overflow issue. Should our actions have
>> that
>> >>>> fix? Closer to home, the NodeJS kind ships with `OpenWhisk v3.3.2`,
>> but the
>> >>>> latest is `v3.6.0` which is needed for non-experimental API Gateway
>> support…
>> >>>>
>> >>>> Some questions:
>> >>>>
>> >>>> 1. Should we update the language runtime for a kind for a patch level
>> >>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the
>> latest
>> >>>> `6.9.5`?
>> >>>> 2. Should we ever update the language runtime for a kind for a minor
>> level
>> >>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the
>> latest
>> >>>> `6.11.1`?
>> >>>> 3. Should we ever update the packages in a kind to the latest patch
>> level
>> >>>> or minor level?
>> >>>> 4. What's our policy when a security issue is published for a
>> language or
>> >>>> a package that we ship in a non-deprecated kind?
>> >>>>
>> >>>> Whatever the answers are, I think we should document them clearly
>> >>>> somewhere.
>> >>>>
>> >>>>
>> >>>> Also, I've started this conversation as a mailing list topic as it's
>> a
>> >>>> "policy" thing. Given my previous comments on mailing lists, should
>> I also
>> >>>> create a GitHub issue prefixed with "Discussion" to provide more
>> visibility
>> >>>> in order to garner wider community input?
>> >>>>
>> >>>>
>> >>>> Regards,
>> >>>>
>> >>>> Rob...
>> >>>>
>> >>>> [1]: https://github.com/apache/incubator-openwhisk/pull/2415#
>> >>>> issuecomment-314716101 <https://github.com/apache/
>> >>>> incubator-openwhisk/pull/2415#issuecomment-314716101>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Regards,
>> >>> James Thomas
>> >
>> >--
>> >Development thoughts at http://akrabat.com
>> >Daily Jotter for macOS at http://dailyjotter.com
>> >
>>
>

Re: Updating Package and Language versions for a kind

[Carlos Santana <cs...@gmail.com>]

One thought I have being debating is that the open source project doesn't
include any packages going forward.
The project will have an image that is small and common denominator that
the operators deploying decide which packages to add on top.

For example for nodejs 8, we create the kind nodejs:8 (at LTS Oct/2017),

Will keep the version of language nodejs 8 up to date to the latest as
official releases come out from nodejs
Operators like IBM and Adobe, RH, others, can base their custom images for
their offerings on the same base image, adding packages that they want base
on their offering.

Same for swift new kind swift:4, and so on.

This will be a general policy, as every language has their peculiarity and
probably we need our runtimes SMEs to talk about the risk on updating or
not updating per language.

dealing with user level packages is always a pickle for example today we
have nodejs:6 and we have a set of users that benefit from updating the
packages to the latest latest, and could have users that they depend on a
current version of the package and updating the package on them could cause
problems. If I had my way I would say thought luck for using the embedded
package and not packaging or doing a bundle build, but serverless is on
baby years of adoption and I don't want to be thought yet :-)

So for nodejs8 we say thought luck no embedded packages, but will do
something to make it super easy will have the system help you build/compile
your action and include your dependencies, these system actions that allow
users to provide only their code/git-url, and package.json/package.swift,
and these actions will compile/npm install etc..  and install your action
including the declared dependencies.





On Tue, Jul 18, 2017 at 4:08 AM Michael Marth <mm...@adobe.com.invalid>
wrote:

> Hi Rob,
>
> Re “I wonder how many deployments roll their own action containers?”
>
> I spoke too general (implying what most OW deployments would want). What I
> can say is that our team wishes to be in control of the libraries that go
> into the action containers, that’s why we need to roll our own.
>
> Agree with the approach you have outlined on patch and minor releases.
>
> The topic of deprecation IMHO is the same discussion as when to upgrade
> language runtimes in action containers: at least for us the deprecation
> policy is something that we need to be in control of (which is a given as
> we roll our own containers).
> +1 to the deprecation tooling you hint at regarding not creating actions
> for deprecated containers (this is needed regardless of any deprecation
> policies). This is not covered by the issues you linked to, is it?
>
> Cheers
> Michael
>
>
>
> On 14/07/17 19:08, "Rob Allen" <ro...@akrabat.com> wrote:
>
> >Hi Michael, all,
> >
> >I wonder how many deployments roll their own action containers? I suspect
> not many.
> >
> >I agree with you both that it would be nice to keep packages (modules) up
> to date on a regular basis within a kind makes the most sense and a regular
> schedule would be nice, especially if there's a documented way to do the
> update for each kind.
> >
> >I think we should also keep the language of the kind up to date at the
> *patch* level on the same cadence too. e.g. Node 6.11.0 to Node 6.11.1 -
> unless there are language kinds that break semver at the patch level? (In
> which case my head may explode!)
> >
> >For minor releases, it's a little more wooly at the moment at least. For
> NodeJS, [PR 2078  [1] simply updates the `node6` kind from 6.9.1 to 6.11.0
> without creating a new new "kind". However, for Swift, [PR] [2120] gave us
> a new `swift:3.1.1` kind rather than updating the `swift3` kind which runs
> Swift 3.0. I'm not sure that Swift 3.1.2 will need its own kind though, so
> maybe it should have been called `swift:3.1`?
> >
> >My thoughts on this is that we should create new kinds for minor language
> releases on the principle of least surprise for the container users.
> >
> >I would like to fairly aggressively deprecate old ones to stop new
> actions being created for out-of-date language versions, but as deprecating
> a kind prevents invoking any actions for that kind that's not possible
> unless some work is done to allow invoking actions on a deprecated kind,
> but not creating or updating them would be necessary for that to be a
> possibility.
> >
> >I think GitHub Issue [319] [3] is relevant in terms of the tooling around
> this.
> >
> >
> >Regards,
> >
> >Rob...
> >
> >[1]: https://github.com/apache/incubator-openwhisk/pull/2078
> >[2]: https://github.com/apache/incubator-openwhisk/pull/2120
> >[3]: https://github.com/apache/incubator-openwhisk/issues/319
> >
> >
> >> On 14 Jul 2017, at 16:27, Michael Marth <mm...@adobe.com.INVALID>
> wrote:
> >>
> >> Rob, all,
> >>
> >> I largely echo what James wrote below (except moving this discussion
> off-list :) )
> >>
> >> Incidentally, we had a very similar discussion within my team recently
> on how to handle this topic for our OW deployment. The bottom line is that
> the trade-off between stability and being up-to-date can only be decided in
> a given business context of the OW deployment.
> >> We (as a project) do not have that context and cannot really make the
> decision, IMHO. In that light the only reasonable policy in my view is to
> strive to keep everything updated as best as we can, like monthly or every
> second month, etc.
> >> If we miss a beat (i.e. don’t immediately update to latest release) I
> don’t think that’d be critical. Like James said, most OW providers will
> have their own runtimes anyway and would (should) take the latest if they
> need it.
> >>
> >> Cheers
> >> Michael
> >>
> >>
> >> On 13/07/17 11:02, "James Thomas" <jt...@gmail.com> wrote:
> >>
> >>> Good ideas Rob. I had a similar issue when looking at the Swift runtime
> >>> recently.
> >>>
> https://lists.apache.org/thread.html/fa01baca7d97d08c855abfc69fc17a23e038115fcfc4f2a31d650fa1@%3Cdev.openwhisk.apache.org%3E
> >>>
> >>> Would it be possible to have a scheduled upgrade process for installed
> >>> modules? Once every four, six or eight weeks? If OpenWhisk did start to
> >>> produce "releases", it could tie in with that.
> >>>
> >>> I'd guess that most people using the built-in packages are more
> kicking the
> >>> tires than building production apps. Once you start being a production
> app,
> >>> you'll want to explicitly bundle and control your app dependencies.
> I'd +1
> >>> on being more aggressive with upgrading module versions.
> >>>
> >>> I'd like to have a Github issue to follow for this, I find it easier
> than
> >>> the mailing list.
> >>>
> >>> On 13 July 2017 at 09:33, Rob Allen <ro...@akrabat.com> wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> On the PHP PR, @rr [commented] [1]:
> >>>>
> >>>>> The built in packages are convenient - less zip files for the initial
> >>>> ramp up. But it creates a maintenance issue: when do you pick up
> updates to
> >>>> the packages (minor/patch level only?) and not break existing actions
> using
> >>>> the "kind". That is: is the kind itself semantically versioned?
> >>>>
> >>>> This applies to all kinds and so probably should be discussed project
> >>>> level and ideally we should document how this is handled.
> >>>>
> >>>> There are two things here:
> >>>>
> >>>> 1. The language runtimes release patch updates for minor versions.
> e.g.
> >>>> PHP `7.1.7` will become `7.1.8` next month with a small number of bug
> fixes
> >>>> including crashers and possibly security fixes.
> >>>>
> >>>> 2. Each kind bindles a number of packages via the language's standard
> >>>> package management system: Swift Package Manager for Swift, NPM for
> NodeJs,
> >>>> etc. The projects that produce these packages update them with new
> versions
> >>>> minor and patch versions.
> >>>>
> >>>> The tension is obviously between keeping updated for fixes vs the
> risk of
> >>>> breaks due to a project's inability to keep BC between patch
> versions. e.g.
> >>>> the NodeJS kind comes with the `async v2.1.4` package. However
> `v2.1.5` of
> >>>> that package fixes a stack overflow issue. Should our actions have
> that
> >>>> fix? Closer to home, the NodeJS kind ships with `OpenWhisk v3.3.2`,
> but the
> >>>> latest is `v3.6.0` which is needed for non-experimental API Gateway
> support…
> >>>>
> >>>> Some questions:
> >>>>
> >>>> 1. Should we update the language runtime for a kind for a patch level
> >>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the
> latest
> >>>> `6.9.5`?
> >>>> 2. Should we ever update the language runtime for a kind for a minor
> level
> >>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the
> latest
> >>>> `6.11.1`?
> >>>> 3. Should we ever update the packages in a kind to the latest patch
> level
> >>>> or minor level?
> >>>> 4. What's our policy when a security issue is published for a
> language or
> >>>> a package that we ship in a non-deprecated kind?
> >>>>
> >>>> Whatever the answers are, I think we should document them clearly
> >>>> somewhere.
> >>>>
> >>>>
> >>>> Also, I've started this conversation as a mailing list topic as it's a
> >>>> "policy" thing. Given my previous comments on mailing lists, should I
> also
> >>>> create a GitHub issue prefixed with "Discussion" to provide more
> visibility
> >>>> in order to garner wider community input?
> >>>>
> >>>>
> >>>> Regards,
> >>>>
> >>>> Rob...
> >>>>
> >>>> [1]: https://github.com/apache/incubator-openwhisk/pull/2415#
> >>>> issuecomment-314716101 <https://github.com/apache/
> >>>> incubator-openwhisk/pull/2415#issuecomment-314716101>
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> Regards,
> >>> James Thomas
> >
> >--
> >Development thoughts at http://akrabat.com
> >Daily Jotter for macOS at http://dailyjotter.com
> >
>

Re: Updating Package and Language versions for a kind

[Michael Marth <mm...@adobe.com.INVALID>]

Hi Rob,

Re “I wonder how many deployments roll their own action containers?”

I spoke too general (implying what most OW deployments would want). What I can say is that our team wishes to be in control of the libraries that go into the action containers, that’s why we need to roll our own.

Agree with the approach you have outlined on patch and minor releases.

The topic of deprecation IMHO is the same discussion as when to upgrade language runtimes in action containers: at least for us the deprecation policy is something that we need to be in control of (which is a given as we roll our own containers).
+1 to the deprecation tooling you hint at regarding not creating actions for deprecated containers (this is needed regardless of any deprecation policies). This is not covered by the issues you linked to, is it?

Cheers
Michael



On 14/07/17 19:08, "Rob Allen" <ro...@akrabat.com> wrote:

>Hi Michael, all,
>
>I wonder how many deployments roll their own action containers? I suspect not many.
>
>I agree with you both that it would be nice to keep packages (modules) up to date on a regular basis within a kind makes the most sense and a regular schedule would be nice, especially if there's a documented way to do the update for each kind. 
>
>I think we should also keep the language of the kind up to date at the *patch* level on the same cadence too. e.g. Node 6.11.0 to Node 6.11.1 - unless there are language kinds that break semver at the patch level? (In which case my head may explode!)
>
>For minor releases, it's a little more wooly at the moment at least. For NodeJS, [PR 2078  [1] simply updates the `node6` kind from 6.9.1 to 6.11.0  without creating a new new "kind". However, for Swift, [PR] [2120] gave us a new `swift:3.1.1` kind rather than updating the `swift3` kind which runs Swift 3.0. I'm not sure that Swift 3.1.2 will need its own kind though, so maybe it should have been called `swift:3.1`? 
>
>My thoughts on this is that we should create new kinds for minor language releases on the principle of least surprise for the container users.
>
>I would like to fairly aggressively deprecate old ones to stop new actions being created for out-of-date language versions, but as deprecating a kind prevents invoking any actions for that kind that's not possible unless some work is done to allow invoking actions on a deprecated kind, but not creating or updating them would be necessary for that to be a possibility. 
>
>I think GitHub Issue [319] [3] is relevant in terms of the tooling around this.
>
>
>Regards,
>
>Rob...
>
>[1]: https://github.com/apache/incubator-openwhisk/pull/2078
>[2]: https://github.com/apache/incubator-openwhisk/pull/2120
>[3]: https://github.com/apache/incubator-openwhisk/issues/319
>
>
>> On 14 Jul 2017, at 16:27, Michael Marth <mm...@adobe.com.INVALID> wrote:
>> 
>> Rob, all,
>> 
>> I largely echo what James wrote below (except moving this discussion off-list :) )
>> 
>> Incidentally, we had a very similar discussion within my team recently on how to handle this topic for our OW deployment. The bottom line is that the trade-off between stability and being up-to-date can only be decided in a given business context of the OW deployment.
>> We (as a project) do not have that context and cannot really make the decision, IMHO. In that light the only reasonable policy in my view is to strive to keep everything updated as best as we can, like monthly or every second month, etc.
>> If we miss a beat (i.e. don’t immediately update to latest release) I don’t think that’d be critical. Like James said, most OW providers will have their own runtimes anyway and would (should) take the latest if they need it.
>> 
>> Cheers
>> Michael
>> 
>> 
>> On 13/07/17 11:02, "James Thomas" <jt...@gmail.com> wrote:
>> 
>>> Good ideas Rob. I had a similar issue when looking at the Swift runtime
>>> recently.
>>> https://lists.apache.org/thread.html/fa01baca7d97d08c855abfc69fc17a23e038115fcfc4f2a31d650fa1@%3Cdev.openwhisk.apache.org%3E
>>> 
>>> Would it be possible to have a scheduled upgrade process for installed
>>> modules? Once every four, six or eight weeks? If OpenWhisk did start to
>>> produce "releases", it could tie in with that.
>>> 
>>> I'd guess that most people using the built-in packages are more kicking the
>>> tires than building production apps. Once you start being a production app,
>>> you'll want to explicitly bundle and control your app dependencies. I'd +1
>>> on being more aggressive with upgrading module versions.
>>> 
>>> I'd like to have a Github issue to follow for this, I find it easier than
>>> the mailing list.
>>> 
>>> On 13 July 2017 at 09:33, Rob Allen <ro...@akrabat.com> wrote:
>>> 
>>>> Hi all,
>>>> 
>>>> On the PHP PR, @rr [commented] [1]:
>>>> 
>>>>> The built in packages are convenient - less zip files for the initial
>>>> ramp up. But it creates a maintenance issue: when do you pick up updates to
>>>> the packages (minor/patch level only?) and not break existing actions using
>>>> the "kind". That is: is the kind itself semantically versioned?
>>>> 
>>>> This applies to all kinds and so probably should be discussed project
>>>> level and ideally we should document how this is handled.
>>>> 
>>>> There are two things here:
>>>> 
>>>> 1. The language runtimes release patch updates for minor versions. e.g.
>>>> PHP `7.1.7` will become `7.1.8` next month with a small number of bug fixes
>>>> including crashers and possibly security fixes.
>>>> 
>>>> 2. Each kind bindles a number of packages via the language's standard
>>>> package management system: Swift Package Manager for Swift, NPM for NodeJs,
>>>> etc. The projects that produce these packages update them with new versions
>>>> minor and patch versions.
>>>> 
>>>> The tension is obviously between keeping updated for fixes vs the risk of
>>>> breaks due to a project's inability to keep BC between patch versions. e.g.
>>>> the NodeJS kind comes with the `async v2.1.4` package. However `v2.1.5` of
>>>> that package fixes a stack overflow issue. Should our actions have that
>>>> fix? Closer to home, the NodeJS kind ships with `OpenWhisk v3.3.2`, but the
>>>> latest is `v3.6.0` which is needed for non-experimental API Gateway support…
>>>> 
>>>> Some questions:
>>>> 
>>>> 1. Should we update the language runtime for a kind for a patch level
>>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the latest
>>>> `6.9.5`?
>>>> 2. Should we ever update the language runtime for a kind for a minor level
>>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the latest
>>>> `6.11.1`?
>>>> 3. Should we ever update the packages in a kind to the latest patch level
>>>> or minor level?
>>>> 4. What's our policy when a security issue is published for a language or
>>>> a package that we ship in a non-deprecated kind?
>>>> 
>>>> Whatever the answers are, I think we should document them clearly
>>>> somewhere.
>>>> 
>>>> 
>>>> Also, I've started this conversation as a mailing list topic as it's a
>>>> "policy" thing. Given my previous comments on mailing lists, should I also
>>>> create a GitHub issue prefixed with "Discussion" to provide more visibility
>>>> in order to garner wider community input?
>>>> 
>>>> 
>>>> Regards,
>>>> 
>>>> Rob...
>>>> 
>>>> [1]: https://github.com/apache/incubator-openwhisk/pull/2415#
>>>> issuecomment-314716101 <https://github.com/apache/
>>>> incubator-openwhisk/pull/2415#issuecomment-314716101>
>>> 
>>> 
>>> 
>>> 
>>> -- 
>>> Regards,
>>> James Thomas
>
>-- 
>Development thoughts at http://akrabat.com
>Daily Jotter for macOS at http://dailyjotter.com
>

Re: Updating Package and Language versions for a kind

[Rob Allen <ro...@akrabat.com>]

Hi Michael, all,

I wonder how many deployments roll their own action containers? I suspect not many.

I agree with you both that it would be nice to keep packages (modules) up to date on a regular basis within a kind makes the most sense and a regular schedule would be nice, especially if there's a documented way to do the update for each kind. 

I think we should also keep the language of the kind up to date at the *patch* level on the same cadence too. e.g. Node 6.11.0 to Node 6.11.1 - unless there are language kinds that break semver at the patch level? (In which case my head may explode!)

For minor releases, it's a little more wooly at the moment at least. For NodeJS, [PR 2078  [1] simply updates the `node6` kind from 6.9.1 to 6.11.0  without creating a new new "kind". However, for Swift, [PR] [2120] gave us a new `swift:3.1.1` kind rather than updating the `swift3` kind which runs Swift 3.0. I'm not sure that Swift 3.1.2 will need its own kind though, so maybe it should have been called `swift:3.1`? 

My thoughts on this is that we should create new kinds for minor language releases on the principle of least surprise for the container users.

I would like to fairly aggressively deprecate old ones to stop new actions being created for out-of-date language versions, but as deprecating a kind prevents invoking any actions for that kind that's not possible unless some work is done to allow invoking actions on a deprecated kind, but not creating or updating them would be necessary for that to be a possibility. 

I think GitHub Issue [319] [3] is relevant in terms of the tooling around this.


Regards,

Rob...

[1]: https://github.com/apache/incubator-openwhisk/pull/2078
[2]: https://github.com/apache/incubator-openwhisk/pull/2120
[3]: https://github.com/apache/incubator-openwhisk/issues/319


> On 14 Jul 2017, at 16:27, Michael Marth <mm...@adobe.com.INVALID> wrote:
> 
> Rob, all,
> 
> I largely echo what James wrote below (except moving this discussion off-list :) )
> 
> Incidentally, we had a very similar discussion within my team recently on how to handle this topic for our OW deployment. The bottom line is that the trade-off between stability and being up-to-date can only be decided in a given business context of the OW deployment.
> We (as a project) do not have that context and cannot really make the decision, IMHO. In that light the only reasonable policy in my view is to strive to keep everything updated as best as we can, like monthly or every second month, etc.
> If we miss a beat (i.e. don’t immediately update to latest release) I don’t think that’d be critical. Like James said, most OW providers will have their own runtimes anyway and would (should) take the latest if they need it.
> 
> Cheers
> Michael
> 
> 
> On 13/07/17 11:02, "James Thomas" <jt...@gmail.com> wrote:
> 
>> Good ideas Rob. I had a similar issue when looking at the Swift runtime
>> recently.
>> https://lists.apache.org/thread.html/fa01baca7d97d08c855abfc69fc17a23e038115fcfc4f2a31d650fa1@%3Cdev.openwhisk.apache.org%3E
>> 
>> Would it be possible to have a scheduled upgrade process for installed
>> modules? Once every four, six or eight weeks? If OpenWhisk did start to
>> produce "releases", it could tie in with that.
>> 
>> I'd guess that most people using the built-in packages are more kicking the
>> tires than building production apps. Once you start being a production app,
>> you'll want to explicitly bundle and control your app dependencies. I'd +1
>> on being more aggressive with upgrading module versions.
>> 
>> I'd like to have a Github issue to follow for this, I find it easier than
>> the mailing list.
>> 
>> On 13 July 2017 at 09:33, Rob Allen <ro...@akrabat.com> wrote:
>> 
>>> Hi all,
>>> 
>>> On the PHP PR, @rr [commented] [1]:
>>> 
>>>> The built in packages are convenient - less zip files for the initial
>>> ramp up. But it creates a maintenance issue: when do you pick up updates to
>>> the packages (minor/patch level only?) and not break existing actions using
>>> the "kind". That is: is the kind itself semantically versioned?
>>> 
>>> This applies to all kinds and so probably should be discussed project
>>> level and ideally we should document how this is handled.
>>> 
>>> There are two things here:
>>> 
>>> 1. The language runtimes release patch updates for minor versions. e.g.
>>> PHP `7.1.7` will become `7.1.8` next month with a small number of bug fixes
>>> including crashers and possibly security fixes.
>>> 
>>> 2. Each kind bindles a number of packages via the language's standard
>>> package management system: Swift Package Manager for Swift, NPM for NodeJs,
>>> etc. The projects that produce these packages update them with new versions
>>> minor and patch versions.
>>> 
>>> The tension is obviously between keeping updated for fixes vs the risk of
>>> breaks due to a project's inability to keep BC between patch versions. e.g.
>>> the NodeJS kind comes with the `async v2.1.4` package. However `v2.1.5` of
>>> that package fixes a stack overflow issue. Should our actions have that
>>> fix? Closer to home, the NodeJS kind ships with `OpenWhisk v3.3.2`, but the
>>> latest is `v3.6.0` which is needed for non-experimental API Gateway support…
>>> 
>>> Some questions:
>>> 
>>> 1. Should we update the language runtime for a kind for a patch level
>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the latest
>>> `6.9.5`?
>>> 2. Should we ever update the language runtime for a kind for a minor level
>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the latest
>>> `6.11.1`?
>>> 3. Should we ever update the packages in a kind to the latest patch level
>>> or minor level?
>>> 4. What's our policy when a security issue is published for a language or
>>> a package that we ship in a non-deprecated kind?
>>> 
>>> Whatever the answers are, I think we should document them clearly
>>> somewhere.
>>> 
>>> 
>>> Also, I've started this conversation as a mailing list topic as it's a
>>> "policy" thing. Given my previous comments on mailing lists, should I also
>>> create a GitHub issue prefixed with "Discussion" to provide more visibility
>>> in order to garner wider community input?
>>> 
>>> 
>>> Regards,
>>> 
>>> Rob...
>>> 
>>> [1]: https://github.com/apache/incubator-openwhisk/pull/2415#
>>> issuecomment-314716101 <https://github.com/apache/
>>> incubator-openwhisk/pull/2415#issuecomment-314716101>
>> 
>> 
>> 
>> 
>> -- 
>> Regards,
>> James Thomas

-- 
Development thoughts at http://akrabat.com
Daily Jotter for macOS at http://dailyjotter.com

Re: Updating Package and Language versions for a kind

[Michael Marth <mm...@adobe.com.INVALID>]

Rob, all,

I largely echo what James wrote below (except moving this discussion off-list :) )

Incidentally, we had a very similar discussion within my team recently on how to handle this topic for our OW deployment. The bottom line is that the trade-off between stability and being up-to-date can only be decided in a given business context of the OW deployment.
We (as a project) do not have that context and cannot really make the decision, IMHO. In that light the only reasonable policy in my view is to strive to keep everything updated as best as we can, like monthly or every second month, etc.
If we miss a beat (i.e. don’t immediately update to latest release) I don’t think that’d be critical. Like James said, most OW providers will have their own runtimes anyway and would (should) take the latest if they need it.

Cheers
Michael



On 13/07/17 11:02, "James Thomas" <jt...@gmail.com> wrote:

>Good ideas Rob. I had a similar issue when looking at the Swift runtime
>recently.
>https://lists.apache.org/thread.html/fa01baca7d97d08c855abfc69fc17a23e038115fcfc4f2a31d650fa1@%3Cdev.openwhisk.apache.org%3E
>
>Would it be possible to have a scheduled upgrade process for installed
>modules? Once every four, six or eight weeks? If OpenWhisk did start to
>produce "releases", it could tie in with that.
>
>I'd guess that most people using the built-in packages are more kicking the
>tires than building production apps. Once you start being a production app,
>you'll want to explicitly bundle and control your app dependencies. I'd +1
>on being more aggressive with upgrading module versions.
>
>I'd like to have a Github issue to follow for this, I find it easier than
>the mailing list.
>
>On 13 July 2017 at 09:33, Rob Allen <ro...@akrabat.com> wrote:
>
>> Hi all,
>>
>> On the PHP PR, @rr [commented] [1]:
>>
>> > The built in packages are convenient - less zip files for the initial
>> ramp up. But it creates a maintenance issue: when do you pick up updates to
>> the packages (minor/patch level only?) and not break existing actions using
>> the "kind". That is: is the kind itself semantically versioned?
>>
>> This applies to all kinds and so probably should be discussed project
>> level and ideally we should document how this is handled.
>>
>> There are two things here:
>>
>> 1. The language runtimes release patch updates for minor versions. e.g.
>> PHP `7.1.7` will become `7.1.8` next month with a small number of bug fixes
>> including crashers and possibly security fixes.
>>
>> 2. Each kind bindles a number of packages via the language's standard
>> package management system: Swift Package Manager for Swift, NPM for NodeJs,
>> etc. The projects that produce these packages update them with new versions
>> minor and patch versions.
>>
>> The tension is obviously between keeping updated for fixes vs the risk of
>> breaks due to a project's inability to keep BC between patch versions. e.g.
>> the NodeJS kind comes with the `async v2.1.4` package. However `v2.1.5` of
>> that package fixes a stack overflow issue. Should our actions have that
>> fix? Closer to home, the NodeJS kind ships with `OpenWhisk v3.3.2`, but the
>> latest is `v3.6.0` which is needed for non-experimental API Gateway support…
>>
>> Some questions:
>>
>> 1. Should we update the language runtime for a kind for a patch level
>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the latest
>> `6.9.5`?
>> 2. Should we ever update the language runtime for a kind for a minor level
>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the latest
>> `6.11.1`?
>> 3. Should we ever update the packages in a kind to the latest patch level
>> or minor level?
>> 4. What's our policy when a security issue is published for a language or
>> a package that we ship in a non-deprecated kind?
>>
>> Whatever the answers are, I think we should document them clearly
>> somewhere.
>>
>>
>> Also, I've started this conversation as a mailing list topic as it's a
>> "policy" thing. Given my previous comments on mailing lists, should I also
>> create a GitHub issue prefixed with "Discussion" to provide more visibility
>> in order to garner wider community input?
>>
>>
>> Regards,
>>
>> Rob...
>>
>> [1]: https://github.com/apache/incubator-openwhisk/pull/2415#
>> issuecomment-314716101 <https://github.com/apache/
>> incubator-openwhisk/pull/2415#issuecomment-314716101>
>
>
>
>
>-- 
>Regards,
>James Thomas

Re: Updating Package and Language versions for a kind

[James Thomas <jt...@gmail.com>]

Good ideas Rob. I had a similar issue when looking at the Swift runtime
recently.
https://lists.apache.org/thread.html/fa01baca7d97d08c855abfc69fc17a23e038115fcfc4f2a31d650fa1@%3Cdev.openwhisk.apache.org%3E

Would it be possible to have a scheduled upgrade process for installed
modules? Once every four, six or eight weeks? If OpenWhisk did start to
produce "releases", it could tie in with that.

I'd guess that most people using the built-in packages are more kicking the
tires than building production apps. Once you start being a production app,
you'll want to explicitly bundle and control your app dependencies. I'd +1
on being more aggressive with upgrading module versions.

I'd like to have a Github issue to follow for this, I find it easier than
the mailing list.

On 13 July 2017 at 09:33, Rob Allen <ro...@akrabat.com> wrote:

> Hi all,
>
> On the PHP PR, @rr [commented] [1]:
>
> > The built in packages are convenient - less zip files for the initial
> ramp up. But it creates a maintenance issue: when do you pick up updates to
> the packages (minor/patch level only?) and not break existing actions using
> the "kind". That is: is the kind itself semantically versioned?
>
> This applies to all kinds and so probably should be discussed project
> level and ideally we should document how this is handled.
>
> There are two things here:
>
> 1. The language runtimes release patch updates for minor versions. e.g.
> PHP `7.1.7` will become `7.1.8` next month with a small number of bug fixes
> including crashers and possibly security fixes.
>
> 2. Each kind bindles a number of packages via the language's standard
> package management system: Swift Package Manager for Swift, NPM for NodeJs,
> etc. The projects that produce these packages update them with new versions
> minor and patch versions.
>
> The tension is obviously between keeping updated for fixes vs the risk of
> breaks due to a project's inability to keep BC between patch versions. e.g.
> the NodeJS kind comes with the `async v2.1.4` package. However `v2.1.5` of
> that package fixes a stack overflow issue. Should our actions have that
> fix? Closer to home, the NodeJS kind ships with `OpenWhisk v3.3.2`, but the
> latest is `v3.6.0` which is needed for non-experimental API Gateway support…
>
> Some questions:
>
> 1. Should we update the language runtime for a kind for a patch level
> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the latest
> `6.9.5`?
> 2. Should we ever update the language runtime for a kind for a minor level
> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the latest
> `6.11.1`?
> 3. Should we ever update the packages in a kind to the latest patch level
> or minor level?
> 4. What's our policy when a security issue is published for a language or
> a package that we ship in a non-deprecated kind?
>
> Whatever the answers are, I think we should document them clearly
> somewhere.
>
>
> Also, I've started this conversation as a mailing list topic as it's a
> "policy" thing. Given my previous comments on mailing lists, should I also
> create a GitHub issue prefixed with "Discussion" to provide more visibility
> in order to garner wider community input?
>
>
> Regards,
>
> Rob...
>
> [1]: https://github.com/apache/incubator-openwhisk/pull/2415#
> issuecomment-314716101 <https://github.com/apache/
> incubator-openwhisk/pull/2415#issuecomment-314716101>




-- 
Regards,
James Thomas

Re: Updating Package and Language versions for a kind

[Rodric Rabbah <ro...@gmail.com>]

Thanks for outlining the issues around this.

Indeed this is not just a php kind issue and in fact comes up here https://github.com/apache/incubator-openwhisk/pull/2078 as well.

-r