You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ariatosca.apache.org by DeWayne Filppi <de...@cloudify.co> on 2017/09/08 21:45:01 UTC
operation dependencies
I see in the examples a list of dependencies for scripts in operations, for
example:
implementation:
primary: scripts/configure.sh
dependencies:
- "ssh.user > { get_input: ssh_username }"
- "ssh.key_filename > { get_input: private_key_path }"
- "ssh.address > { get_attribute: [ virtual_ip,
floating_ip_address ] }"
The spec seems to indicate that the dependency list is for resources that
need to be made available so the main script can be run. Clearly that
isn't the case in the example. Is this '>' syntax documented anywhere?
Re: operation dependencies
Posted by Avia Efrat <av...@cloudify.co>.
Never mind, I already got an answer from the spec =)
On Wed, Nov 22, 2017 at 6:29 PM, Tal Liron <ta...@cloudify.co> wrote:
> Less cumbersome, sure. But it also breaks the object-oriented contract.
>
> Imagine this situation: a third party develops a powerful node type (let's
> say: a virtual router) with many well-defined and polished operations on
> custom interfaces (with their own types), custom workflows for various
> advanced features, and even comes with custom ARIA-based tooling for
> real-time analysis and monitoring.
>
> Someone can come along and create a node template based on this node type
> that changes the operation inputs. For this to work properly, that someone
> *should* also provide a new operation implementation that would deal with
> these different inputs. But, these inputs must be sent for this to work: so
> the custom workflows must also be changed, as well as the custom tooling.
> And if new tools come along from that that third party, they will not work.
>
> Object-oriented is cumbersome, sure. :) But that's also the design
> principle of TOSCA, and this is a case that breaks it. We either want our
> node types to have extensible contracts, or allow them to have breakable
> contracts. It doesn't make sense to me to be somewhere in between:
> strictness should be all the way.
>
> Finally, I don't understand this question:
>
> On Wed, Nov 22, 2017 at 7:21 AM, Avia Efrat <av...@cloudify.co> wrote:
>
> > By the way, do you see a way of not being able to derive the value of an
> > intrinsic function's 'result'? (I know it is not currently supported)
> >
>
Re: operation dependencies
Posted by Tal Liron <ta...@cloudify.co>.
Less cumbersome, sure. But it also breaks the object-oriented contract.
Imagine this situation: a third party develops a powerful node type (let's
say: a virtual router) with many well-defined and polished operations on
custom interfaces (with their own types), custom workflows for various
advanced features, and even comes with custom ARIA-based tooling for
real-time analysis and monitoring.
Someone can come along and create a node template based on this node type
that changes the operation inputs. For this to work properly, that someone
*should* also provide a new operation implementation that would deal with
these different inputs. But, these inputs must be sent for this to work: so
the custom workflows must also be changed, as well as the custom tooling.
And if new tools come along from that that third party, they will not work.
Object-oriented is cumbersome, sure. :) But that's also the design
principle of TOSCA, and this is a case that breaks it. We either want our
node types to have extensible contracts, or allow them to have breakable
contracts. It doesn't make sense to me to be somewhere in between:
strictness should be all the way.
Finally, I don't understand this question:
On Wed, Nov 22, 2017 at 7:21 AM, Avia Efrat <av...@cloudify.co> wrote:
> By the way, do you see a way of not being able to derive the value of an
> intrinsic function's 'result'? (I know it is not currently supported)
>
Re: operation dependencies
Posted by Avia Efrat <av...@cloudify.co>.
I think keeping the ability to accept ad-hoc inputs (at least for now) is a
good idea =). This will (among other thing) make the job of writing custom
service template less cumbersome. Just mentioning again that this is the
place I see a possible justification for ad-hoc 'additions', as we don't
have an operation type, in contrast to other entities.
the suggestion of 'cfg' is only valid if we enable ad-hoc inputs, so if at
the end we will drop this support, I don't think 'cfg' is a good idea.
By the way, do you see a way of not being able to derive the value of an
intrinsic function's 'result'? (I know it is not currently supported)
On Thu, Nov 9, 2017 at 9:02 PM, Tal Liron <ta...@cloudify.co> wrote:
> Thanks for keeping the discussion going, Avia.
>
> Yeah, I do not think that my interpretation is rock solid at all and it's
> definitely forced. It's not hard to find examples in the spec that
> contradict my interpretation, but also there are others that contradict the
> opposite. I think we can all agree at least that the TOSCA spec writers did
> not pin down this topic well enough so it's necessary for us to fill in the
> gaps. I'm still sure that my interpretation is the most inherently correct
> because it preserves the OOP base contract. If we allow ad-hoc inputs we
> are making it that much harder to inherit existing node types while
> expecting their existing implementations to continue working. If we allow
> ah-hoc inputs, why not go ahead and allow a complete free-for-all here and
> allow dynamic monkey patching of derived types? (This is actually what the
> NFV csd04 profile is trying to do: "deprecate" parts of
> tosca.nodes.Compute, which of course will break tooling that relies on that
> contract.) The whole point of having type derivation in TOSCA seems to be
> to allow reuse of tools, and I insist that is especially true for operation
> inputs.
>
> Your idea of supporting a "cfg" (or similar) input in ARIA is something we
> considered. (I don't think it's something we will have in the ARIA profile,
> because we need it supported by *all* operations, including the normative
> ones, and including custom ones created by template writers: so it would be
> have to be hardcoded support in the ARIA codebase.) However, the problem
> with this idea is that non-ARIA TOSCA parsers will fail. The whole idea of
> using the "dependencies" hack was to allow such templates to at least parse
> successfully with other parsers. Their are pros and cons to each approach,
> though, and I'm open to reopening this.
>
> By the way, the next commit to ARIA adds a configuration option for the
> parser: whether to support "ad-hoc inputs" or not. The default is to allow
> them: something that as you know I am very reluctant to do, but feel we
> have no choice at present. But, to do this properly, I think we need to
> also find a way to preserve the type in intrinsic functions, as mentioned
> in this discussion. It would still leave cases where it's possible to have
> type-less inputs, but I guess that's just how it is right now.
>
> [as for the side note: there are *lots* of bugs in the included TOSCA
> examples, including outright typos, so it's hard to consider them as
> canonical...]
>
>
> On Sat, Nov 4, 2017 at 9:26 AM, Avia Efrat <av...@cloudify.co> wrote:
>
> > I know this is an old thread, but since this is an important issue
> (and
> > I'm doing a review of old and interesting mails), I thought I'll take a
> > shot at a reply.
> >
> > I'm not relying on one example in the spec. actually, we can see that
> > inputs are defined under the normative "Standard" and "Configure"
> > interfaces and their operations many time throughout the spec. Isn't the
> > fact that inputs are assigned to interfaces and operations that do not
> have
> > the corresponding definition, in addition to the sentence from
> [3.5.13.3],
> > in addition to the interfaces being are the only entities in TOSCA that
> do
> > not have explicit distinction between definition and assignment enough to
> > suggest that this may have been intentional?
> >
> > These usages of untyped inputs are still there in v1.2, and
> unfortunately,
> > artifact definitions still do not have a properties-like field.
> >
> > I liked your idea suggesting that "untyped" inputs get their type
> > definition from the intrinsic functions, that seems to settle between the
> > unusualness of interface definitions, and the fact that TOSCA gives us
> the
> > notion that assignment should be derived from a definition. This could
> also
> > be a (temporary?) solution to the lack of "operation type" concept, since
> > it does not seem to solved in v1.2.
> >
> > In addition, I agree that this does not solve the problem of your
> proposed
> > inputs/meta inputs (inputs/configurations) distinction. Just a thought,
> we
> > could decide that an input with a special name such as "cfg" could be a
> map
> > containing "configurations", while other inputs are treated as regular
> > inputs. This could part of the ARIA profile.
> >
> > [side note:
> > You said:
> > And indeed this is exactly what we see example 2.14.2, too, where we see
> > property assignments. But this example snippet doesn't prove much of
> > anything, because we don't see the node type defined, just used here.
> > The "wp_db_port" input could/should be declared there.
> > But in fact this type is defined in [8.3
> > <http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-
> > YAML/v1.0/os/TOSCA-Simple-Profile-YAML-v1.0-os.html#_Toc471725259>],
> > The section for non-normative types used for examples throughout the
> spec.
> > This section also exist in v1.2, with no changes to the aforementioned
> node
> > type]
> >
> > On Mon, Sep 11, 2017 at 8:01 PM, Tal Liron <ta...@cloudify.co> wrote:
> >
> > > The single sentence you mention in the 3.5.13.3 is the only place that
> > > *might* be implying that ad hoc, type-less input assignments are
> allowed,
> > > but I actually think it could have a different reading. What i meant
> is:
> > > "...that do not necessarily have a property definition defined in its
> > > corresponding *interface* type." I think what is implied is what I
> > mention
> > > in level #2: a node template could have assignments that are not in the
> > > interface type, *but obviously they have to be defined in the node
> type*.
> > > Obvious, I say, because they have to be defined somewhere: an undefined
> > > property assignment makes little sense in TOSCA. So let me rewrite that
> > > sentence with more clarity, without (I think) changing its original
> > > meaning:
> > >
> > > "Template authors MAY provide property assignments on operation inputs
> on
> > > templates that do not necessarily have a property definition defined in
> > its
> > > corresponding interface type by adding an input definition to the node
> > > type."
> > >
> > > To be more honest here: when I say "I think what is implied" I have to
> > > wonder if the author was completely clear on what was implied. Like
> many
> > > parts of spec, there's a lot of theory that clashes in practice. We are
> > > confused, but it could be that the author was confused as well. By
> > > implementing the spec we have a responsibility to pin down what's
> fuzzy.
> > >
> > > The "partial" differentiation as you call it in 3.5.13.1 is what I
> meant:
> > > in some places you use property definitions, in other assignments. The
> > > language is very clear that you use definitions with types and
> > assignments
> > > with templates. They didn't explicitly have an "operation assignment"
> > > grammar, but it's clear that there are two different grammars. When I
> say
> > > "operation assignment", it's a shorthand for the kind of operation
> > > definition that you find in templates which uses property assignments.
> > This
> > > is consistent with everything else in TOSCA.
> > >
> > > And indeed this is exactly what we see example 2.14.2, too, where we
> see
> > > property assignments. But this example snippet doesn't prove much of
> > > anything, because we don't see the node type defined, just used here.
> > > The "wp_db_port"
> > > input could/should be declared there. The fact that ARIA accepts this
> has
> > > always been very problematic to me: we here have an input that is
> > untyped.
> > > I consider this a bug. As we discussed, this particular example *might*
> > > make a bit of sense because we are using an intrinsic function, which
> > could
> > > carry the type with it... but nowhere in the spec is any mechanism like
> > > that explained. Many of the examples in the TOSCA spec are wrong or
> > > incomplete, too, so I wouldn't be shocked if this one is wrong as well.
> > >
> > > Anyway, none of this addresses the core issue in my opinion, which I
> keep
> > > returning to: these kinds of parameters we add here -- SSH user,
> > password,
> > > etc. -- are meant as *directives to our execution plugin*, conceptually
> > > very different from operation inputs. Mixing them together is confusing
> > > (what do you do with name overlaps?) as well as a major security
> concern.
> > > These two kinds of values simply should not be mixed together. Indeed,
> in
> > > TOSCA 1.2 they might end up being artifact properties.
> > >
> > >
> > > On Mon, Sep 11, 2017 at 11:25 AM, Avia Efrat <av...@cloudify.co> wrote:
> > >
> > > > In contrary to most of the TOSCA entities, TOSCA does not
> differentiate
> > > > between 'definition' and 'assignment' in the context of operations.
> > There
> > > > are only "operation definitions" [3.5.13]. Logically, there is a
> > partial
> > > > differentiation [3.5.13.1], where inputs in node type operations are
> > > > expected to be property definitions, and inputs in node template
> > > operation
> > > > are expected to be property assignments (which are actual value
> > > > assignments). Both of these options are listed under "operation
> > > > definition".
> > > >
> > > > Under [3.5.13.3] it is explicitly stated that
> > > > "Template authors MAY provide property *assignments* on operation
> > inputs
> > > on
> > > > *templates* that do not necessarily have a property definition
> defined
> > in
> > > > its corresponding type." (my emphasis)
> > > >
> > > > That is, from this paragraph, I think that it is clear that you can
> > > define
> > > > operation inputs in node templates (level #3) without them being
> > defined
> > > in
> > > > the node template's type. In fact, [2.14.2] is an example of doing
> just
> > > > that. And out parser treats such a syntax as valid, see:
> > > > https://github.com/apache/incubator-ariatosca/blob/
> > > > master/tests/parser/test_tosca_simple_v1_0/test_end2end.py#L73
> > > >
> > > > Me thinking it was a good idea to construct TOSCA that way is another
> > > thing
> > > > =)
> > > >
> > > > On Mon, Sep 11, 2017 at 6:35 PM, Tal Liron <ta...@cloudify.co> wrote:
> > > >
> > > > > Feel free to change the wiki, Ran, to whatever name you find
> > > appropriate.
> > > > >
> > > > > I think what Avia discovered is not new to us and actually doesn't
> > > solve
> > > > > the problem, unfortunately. Let me go over what is clearly allowed
> > and
> > > > not
> > > > > allowed in TOSCA, confusing because there are a few levels of
> > > inheritance
> > > > > here.
> > > > >
> > > > > 1. Interface types. Obviously, you are allowed to inherit an
> > interface
> > > > type
> > > > > and add or override inputs. (ARIA insists that overridden input
> types
> > > be
> > > > > derived from what it is they are overriding, too, to keep the OO
> > > contract
> > > > > intact.)
> > > > >
> > > > > 2. Node types. In the "interfaces" section you can define several
> > > > > interfaces of various types. Here, again, TOSCA lets you
> add/override
> > > > > inputs. Though note here that the line of inheritance is quite
> > complex:
> > > > you
> > > > > can override inputs from the interface type, but *also* from the
> > parent
> > > > > node type. (ARIA here has to do some work to make sure that you are
> > > doing
> > > > > it all OK and not breaking the OO contract, it's a rather complex
> > part
> > > of
> > > > > the parser code.)
> > > > >
> > > > > I think the above is what Avia discovered. However, the third level
> > is
> > > > > locked to us:
> > > > >
> > > > > 3. Node templates. Unfortunately, here we can not add inputs ad
> hoc.
> > > The
> > > > > "interfaces" section here is not the same DSL format as those
> above:
> > it
> > > > is
> > > > > operation *assignments* rather than operation *definitions*. When
> you
> > > > > assign input values here, they are validated against the defined
> > types.
> > > > It
> > > > > would make no sense in TOSCA to just assign values without a type.
> > > > >
> > > > > So, because we can't add inputs at level #3, we still have a
> problem:
> > > we
> > > > > would have to derive new node types for every type of execution.
> SSH
> > > > would
> > > > > require its own node types, Juju would require its own node types,
> > > Puppet
> > > > > would require its own node types, etc. And that's for *all* your
> node
> > > > > types. This seems extremely un-scalable.
> > > > >
> > > > > But also, as I try to explain in the wiki page, I insist that these
> > > kinds
> > > > > of configuration parameters are essentially not the same as
> operation
> > > > > inputs. They are not meant to be used by the operation itself
> > (script,
> > > > > charm, recipe, etc.), rather by the mechanism that executes the
> > > operation
> > > > > (SSH, Juju, Puppet, etc.). Especially I point out the security
> hole:
> > > you
> > > > > don't want an SSH password exposed and sent over the wire to the
> > script
> > > > > itself. It is simply not an input.
> > > > >
> > > > > By the way, it seems that there's some acknowledgement by other
> folk
> > in
> > > > > OASIS about this gap in TOSCA, and there's an interest to use
> > artifact
> > > > > types in TOSCA 1.2 as a way to solve this problem. I don't think
> > it's a
> > > > bit
> > > > > awkward, but at least there will be a standard solution.
> > > > >
> > > > >
> > > > > On Sun, Sep 10, 2017 at 2:13 AM, Ran Ziv <ra...@cloudify.co> wrote:
> > > > >
> > > > > > Avia's mentioned at one point that we might have misunderstood
> the
> > > spec
> > > > > at
> > > > > > this section, and that in fact it can be possible to pass
> arbitrary
> > > > > inputs
> > > > > > into operations regardless of the interface definition - which
> > would
> > > > mean
> > > > > > this notion of "configuration" might be unnecessary.
> > > > > >
> > > > > > Also, note that the doc page is talking about "executors", which
> is
> > > > > > confusing as that's a different concept in ARIA (see the base
> > > executor
> > > > > > class); Supposedly up until now we've simply called these
> > "operation
> > > > > > plugins".
> > > > > >
> > > > > >
> > > > > > On Sat, Sep 9, 2017 at 1:43 AM, Tal Liron <ta...@cloudify.co>
> wrote:
> > > > > >
> > > > > > > Yes:
> > > > > > > https://cwiki.apache.org/confluence/display/ARIATOSCA/
> > > > > > > Execution+Configuration
> > > > > > >
> > > > > > > On Fri, Sep 8, 2017 at 4:45 PM, DeWayne Filppi <
> > > dewayne@cloudify.co>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I see in the examples a list of dependencies for scripts in
> > > > > operations,
> > > > > > > for
> > > > > > > > example:
> > > > > > > >
> > > > > > > > implementation:
> > > > > > > > primary: scripts/configure.sh
> > > > > > > > dependencies:
> > > > > > > > - "ssh.user > { get_input: ssh_username }"
> > > > > > > > - "ssh.key_filename > { get_input:
> > > private_key_path
> > > > > }"
> > > > > > > > - "ssh.address > { get_attribute: [
> virtual_ip,
> > > > > > > > floating_ip_address ] }"
> > > > > > > >
> > > > > > > > The spec seems to indicate that the dependency list is for
> > > > resources
> > > > > > that
> > > > > > > > need to be made available so the main script can be run.
> > Clearly
> > > > > that
> > > > > > > > isn't the case in the example. Is this '>' syntax documented
> > > > > anywhere?
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: operation dependencies
Posted by Tal Liron <ta...@cloudify.co>.
Thanks for keeping the discussion going, Avia.
Yeah, I do not think that my interpretation is rock solid at all and it's
definitely forced. It's not hard to find examples in the spec that
contradict my interpretation, but also there are others that contradict the
opposite. I think we can all agree at least that the TOSCA spec writers did
not pin down this topic well enough so it's necessary for us to fill in the
gaps. I'm still sure that my interpretation is the most inherently correct
because it preserves the OOP base contract. If we allow ad-hoc inputs we
are making it that much harder to inherit existing node types while
expecting their existing implementations to continue working. If we allow
ah-hoc inputs, why not go ahead and allow a complete free-for-all here and
allow dynamic monkey patching of derived types? (This is actually what the
NFV csd04 profile is trying to do: "deprecate" parts of
tosca.nodes.Compute, which of course will break tooling that relies on that
contract.) The whole point of having type derivation in TOSCA seems to be
to allow reuse of tools, and I insist that is especially true for operation
inputs.
Your idea of supporting a "cfg" (or similar) input in ARIA is something we
considered. (I don't think it's something we will have in the ARIA profile,
because we need it supported by *all* operations, including the normative
ones, and including custom ones created by template writers: so it would be
have to be hardcoded support in the ARIA codebase.) However, the problem
with this idea is that non-ARIA TOSCA parsers will fail. The whole idea of
using the "dependencies" hack was to allow such templates to at least parse
successfully with other parsers. Their are pros and cons to each approach,
though, and I'm open to reopening this.
By the way, the next commit to ARIA adds a configuration option for the
parser: whether to support "ad-hoc inputs" or not. The default is to allow
them: something that as you know I am very reluctant to do, but feel we
have no choice at present. But, to do this properly, I think we need to
also find a way to preserve the type in intrinsic functions, as mentioned
in this discussion. It would still leave cases where it's possible to have
type-less inputs, but I guess that's just how it is right now.
[as for the side note: there are *lots* of bugs in the included TOSCA
examples, including outright typos, so it's hard to consider them as
canonical...]
On Sat, Nov 4, 2017 at 9:26 AM, Avia Efrat <av...@cloudify.co> wrote:
> I know this is an old thread, but since this is an important issue (and
> I'm doing a review of old and interesting mails), I thought I'll take a
> shot at a reply.
>
> I'm not relying on one example in the spec. actually, we can see that
> inputs are defined under the normative "Standard" and "Configure"
> interfaces and their operations many time throughout the spec. Isn't the
> fact that inputs are assigned to interfaces and operations that do not have
> the corresponding definition, in addition to the sentence from [3.5.13.3],
> in addition to the interfaces being are the only entities in TOSCA that do
> not have explicit distinction between definition and assignment enough to
> suggest that this may have been intentional?
>
> These usages of untyped inputs are still there in v1.2, and unfortunately,
> artifact definitions still do not have a properties-like field.
>
> I liked your idea suggesting that "untyped" inputs get their type
> definition from the intrinsic functions, that seems to settle between the
> unusualness of interface definitions, and the fact that TOSCA gives us the
> notion that assignment should be derived from a definition. This could also
> be a (temporary?) solution to the lack of "operation type" concept, since
> it does not seem to solved in v1.2.
>
> In addition, I agree that this does not solve the problem of your proposed
> inputs/meta inputs (inputs/configurations) distinction. Just a thought, we
> could decide that an input with a special name such as "cfg" could be a map
> containing "configurations", while other inputs are treated as regular
> inputs. This could part of the ARIA profile.
>
> [side note:
> You said:
> And indeed this is exactly what we see example 2.14.2, too, where we see
> property assignments. But this example snippet doesn't prove much of
> anything, because we don't see the node type defined, just used here.
> The "wp_db_port" input could/should be declared there.
> But in fact this type is defined in [8.3
> <http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-
> YAML/v1.0/os/TOSCA-Simple-Profile-YAML-v1.0-os.html#_Toc471725259>],
> The section for non-normative types used for examples throughout the spec.
> This section also exist in v1.2, with no changes to the aforementioned node
> type]
>
> On Mon, Sep 11, 2017 at 8:01 PM, Tal Liron <ta...@cloudify.co> wrote:
>
> > The single sentence you mention in the 3.5.13.3 is the only place that
> > *might* be implying that ad hoc, type-less input assignments are allowed,
> > but I actually think it could have a different reading. What i meant is:
> > "...that do not necessarily have a property definition defined in its
> > corresponding *interface* type." I think what is implied is what I
> mention
> > in level #2: a node template could have assignments that are not in the
> > interface type, *but obviously they have to be defined in the node type*.
> > Obvious, I say, because they have to be defined somewhere: an undefined
> > property assignment makes little sense in TOSCA. So let me rewrite that
> > sentence with more clarity, without (I think) changing its original
> > meaning:
> >
> > "Template authors MAY provide property assignments on operation inputs on
> > templates that do not necessarily have a property definition defined in
> its
> > corresponding interface type by adding an input definition to the node
> > type."
> >
> > To be more honest here: when I say "I think what is implied" I have to
> > wonder if the author was completely clear on what was implied. Like many
> > parts of spec, there's a lot of theory that clashes in practice. We are
> > confused, but it could be that the author was confused as well. By
> > implementing the spec we have a responsibility to pin down what's fuzzy.
> >
> > The "partial" differentiation as you call it in 3.5.13.1 is what I meant:
> > in some places you use property definitions, in other assignments. The
> > language is very clear that you use definitions with types and
> assignments
> > with templates. They didn't explicitly have an "operation assignment"
> > grammar, but it's clear that there are two different grammars. When I say
> > "operation assignment", it's a shorthand for the kind of operation
> > definition that you find in templates which uses property assignments.
> This
> > is consistent with everything else in TOSCA.
> >
> > And indeed this is exactly what we see example 2.14.2, too, where we see
> > property assignments. But this example snippet doesn't prove much of
> > anything, because we don't see the node type defined, just used here.
> > The "wp_db_port"
> > input could/should be declared there. The fact that ARIA accepts this has
> > always been very problematic to me: we here have an input that is
> untyped.
> > I consider this a bug. As we discussed, this particular example *might*
> > make a bit of sense because we are using an intrinsic function, which
> could
> > carry the type with it... but nowhere in the spec is any mechanism like
> > that explained. Many of the examples in the TOSCA spec are wrong or
> > incomplete, too, so I wouldn't be shocked if this one is wrong as well.
> >
> > Anyway, none of this addresses the core issue in my opinion, which I keep
> > returning to: these kinds of parameters we add here -- SSH user,
> password,
> > etc. -- are meant as *directives to our execution plugin*, conceptually
> > very different from operation inputs. Mixing them together is confusing
> > (what do you do with name overlaps?) as well as a major security concern.
> > These two kinds of values simply should not be mixed together. Indeed, in
> > TOSCA 1.2 they might end up being artifact properties.
> >
> >
> > On Mon, Sep 11, 2017 at 11:25 AM, Avia Efrat <av...@cloudify.co> wrote:
> >
> > > In contrary to most of the TOSCA entities, TOSCA does not differentiate
> > > between 'definition' and 'assignment' in the context of operations.
> There
> > > are only "operation definitions" [3.5.13]. Logically, there is a
> partial
> > > differentiation [3.5.13.1], where inputs in node type operations are
> > > expected to be property definitions, and inputs in node template
> > operation
> > > are expected to be property assignments (which are actual value
> > > assignments). Both of these options are listed under "operation
> > > definition".
> > >
> > > Under [3.5.13.3] it is explicitly stated that
> > > "Template authors MAY provide property *assignments* on operation
> inputs
> > on
> > > *templates* that do not necessarily have a property definition defined
> in
> > > its corresponding type." (my emphasis)
> > >
> > > That is, from this paragraph, I think that it is clear that you can
> > define
> > > operation inputs in node templates (level #3) without them being
> defined
> > in
> > > the node template's type. In fact, [2.14.2] is an example of doing just
> > > that. And out parser treats such a syntax as valid, see:
> > > https://github.com/apache/incubator-ariatosca/blob/
> > > master/tests/parser/test_tosca_simple_v1_0/test_end2end.py#L73
> > >
> > > Me thinking it was a good idea to construct TOSCA that way is another
> > thing
> > > =)
> > >
> > > On Mon, Sep 11, 2017 at 6:35 PM, Tal Liron <ta...@cloudify.co> wrote:
> > >
> > > > Feel free to change the wiki, Ran, to whatever name you find
> > appropriate.
> > > >
> > > > I think what Avia discovered is not new to us and actually doesn't
> > solve
> > > > the problem, unfortunately. Let me go over what is clearly allowed
> and
> > > not
> > > > allowed in TOSCA, confusing because there are a few levels of
> > inheritance
> > > > here.
> > > >
> > > > 1. Interface types. Obviously, you are allowed to inherit an
> interface
> > > type
> > > > and add or override inputs. (ARIA insists that overridden input types
> > be
> > > > derived from what it is they are overriding, too, to keep the OO
> > contract
> > > > intact.)
> > > >
> > > > 2. Node types. In the "interfaces" section you can define several
> > > > interfaces of various types. Here, again, TOSCA lets you add/override
> > > > inputs. Though note here that the line of inheritance is quite
> complex:
> > > you
> > > > can override inputs from the interface type, but *also* from the
> parent
> > > > node type. (ARIA here has to do some work to make sure that you are
> > doing
> > > > it all OK and not breaking the OO contract, it's a rather complex
> part
> > of
> > > > the parser code.)
> > > >
> > > > I think the above is what Avia discovered. However, the third level
> is
> > > > locked to us:
> > > >
> > > > 3. Node templates. Unfortunately, here we can not add inputs ad hoc.
> > The
> > > > "interfaces" section here is not the same DSL format as those above:
> it
> > > is
> > > > operation *assignments* rather than operation *definitions*. When you
> > > > assign input values here, they are validated against the defined
> types.
> > > It
> > > > would make no sense in TOSCA to just assign values without a type.
> > > >
> > > > So, because we can't add inputs at level #3, we still have a problem:
> > we
> > > > would have to derive new node types for every type of execution. SSH
> > > would
> > > > require its own node types, Juju would require its own node types,
> > Puppet
> > > > would require its own node types, etc. And that's for *all* your node
> > > > types. This seems extremely un-scalable.
> > > >
> > > > But also, as I try to explain in the wiki page, I insist that these
> > kinds
> > > > of configuration parameters are essentially not the same as operation
> > > > inputs. They are not meant to be used by the operation itself
> (script,
> > > > charm, recipe, etc.), rather by the mechanism that executes the
> > operation
> > > > (SSH, Juju, Puppet, etc.). Especially I point out the security hole:
> > you
> > > > don't want an SSH password exposed and sent over the wire to the
> script
> > > > itself. It is simply not an input.
> > > >
> > > > By the way, it seems that there's some acknowledgement by other folk
> in
> > > > OASIS about this gap in TOSCA, and there's an interest to use
> artifact
> > > > types in TOSCA 1.2 as a way to solve this problem. I don't think
> it's a
> > > bit
> > > > awkward, but at least there will be a standard solution.
> > > >
> > > >
> > > > On Sun, Sep 10, 2017 at 2:13 AM, Ran Ziv <ra...@cloudify.co> wrote:
> > > >
> > > > > Avia's mentioned at one point that we might have misunderstood the
> > spec
> > > > at
> > > > > this section, and that in fact it can be possible to pass arbitrary
> > > > inputs
> > > > > into operations regardless of the interface definition - which
> would
> > > mean
> > > > > this notion of "configuration" might be unnecessary.
> > > > >
> > > > > Also, note that the doc page is talking about "executors", which is
> > > > > confusing as that's a different concept in ARIA (see the base
> > executor
> > > > > class); Supposedly up until now we've simply called these
> "operation
> > > > > plugins".
> > > > >
> > > > >
> > > > > On Sat, Sep 9, 2017 at 1:43 AM, Tal Liron <ta...@cloudify.co> wrote:
> > > > >
> > > > > > Yes:
> > > > > > https://cwiki.apache.org/confluence/display/ARIATOSCA/
> > > > > > Execution+Configuration
> > > > > >
> > > > > > On Fri, Sep 8, 2017 at 4:45 PM, DeWayne Filppi <
> > dewayne@cloudify.co>
> > > > > > wrote:
> > > > > >
> > > > > > > I see in the examples a list of dependencies for scripts in
> > > > operations,
> > > > > > for
> > > > > > > example:
> > > > > > >
> > > > > > > implementation:
> > > > > > > primary: scripts/configure.sh
> > > > > > > dependencies:
> > > > > > > - "ssh.user > { get_input: ssh_username }"
> > > > > > > - "ssh.key_filename > { get_input:
> > private_key_path
> > > > }"
> > > > > > > - "ssh.address > { get_attribute: [ virtual_ip,
> > > > > > > floating_ip_address ] }"
> > > > > > >
> > > > > > > The spec seems to indicate that the dependency list is for
> > > resources
> > > > > that
> > > > > > > need to be made available so the main script can be run.
> Clearly
> > > > that
> > > > > > > isn't the case in the example. Is this '>' syntax documented
> > > > anywhere?
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: operation dependencies
Posted by Avia Efrat <av...@cloudify.co>.
I know this is an old thread, but since this is an important issue (and
I'm doing a review of old and interesting mails), I thought I'll take a
shot at a reply.
I'm not relying on one example in the spec. actually, we can see that
inputs are defined under the normative "Standard" and "Configure"
interfaces and their operations many time throughout the spec. Isn't the
fact that inputs are assigned to interfaces and operations that do not have
the corresponding definition, in addition to the sentence from [3.5.13.3],
in addition to the interfaces being are the only entities in TOSCA that do
not have explicit distinction between definition and assignment enough to
suggest that this may have been intentional?
These usages of untyped inputs are still there in v1.2, and unfortunately,
artifact definitions still do not have a properties-like field.
I liked your idea suggesting that "untyped" inputs get their type
definition from the intrinsic functions, that seems to settle between the
unusualness of interface definitions, and the fact that TOSCA gives us the
notion that assignment should be derived from a definition. This could also
be a (temporary?) solution to the lack of "operation type" concept, since
it does not seem to solved in v1.2.
In addition, I agree that this does not solve the problem of your proposed
inputs/meta inputs (inputs/configurations) distinction. Just a thought, we
could decide that an input with a special name such as "cfg" could be a map
containing "configurations", while other inputs are treated as regular
inputs. This could part of the ARIA profile.
[side note:
You said:
And indeed this is exactly what we see example 2.14.2, too, where we see
property assignments. But this example snippet doesn't prove much of
anything, because we don't see the node type defined, just used here.
The "wp_db_port" input could/should be declared there.
But in fact this type is defined in [8.3
<http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.0/os/TOSCA-Simple-Profile-YAML-v1.0-os.html#_Toc471725259>],
The section for non-normative types used for examples throughout the spec.
This section also exist in v1.2, with no changes to the aforementioned node
type]
On Mon, Sep 11, 2017 at 8:01 PM, Tal Liron <ta...@cloudify.co> wrote:
> The single sentence you mention in the 3.5.13.3 is the only place that
> *might* be implying that ad hoc, type-less input assignments are allowed,
> but I actually think it could have a different reading. What i meant is:
> "...that do not necessarily have a property definition defined in its
> corresponding *interface* type." I think what is implied is what I mention
> in level #2: a node template could have assignments that are not in the
> interface type, *but obviously they have to be defined in the node type*.
> Obvious, I say, because they have to be defined somewhere: an undefined
> property assignment makes little sense in TOSCA. So let me rewrite that
> sentence with more clarity, without (I think) changing its original
> meaning:
>
> "Template authors MAY provide property assignments on operation inputs on
> templates that do not necessarily have a property definition defined in its
> corresponding interface type by adding an input definition to the node
> type."
>
> To be more honest here: when I say "I think what is implied" I have to
> wonder if the author was completely clear on what was implied. Like many
> parts of spec, there's a lot of theory that clashes in practice. We are
> confused, but it could be that the author was confused as well. By
> implementing the spec we have a responsibility to pin down what's fuzzy.
>
> The "partial" differentiation as you call it in 3.5.13.1 is what I meant:
> in some places you use property definitions, in other assignments. The
> language is very clear that you use definitions with types and assignments
> with templates. They didn't explicitly have an "operation assignment"
> grammar, but it's clear that there are two different grammars. When I say
> "operation assignment", it's a shorthand for the kind of operation
> definition that you find in templates which uses property assignments. This
> is consistent with everything else in TOSCA.
>
> And indeed this is exactly what we see example 2.14.2, too, where we see
> property assignments. But this example snippet doesn't prove much of
> anything, because we don't see the node type defined, just used here.
> The "wp_db_port"
> input could/should be declared there. The fact that ARIA accepts this has
> always been very problematic to me: we here have an input that is untyped.
> I consider this a bug. As we discussed, this particular example *might*
> make a bit of sense because we are using an intrinsic function, which could
> carry the type with it... but nowhere in the spec is any mechanism like
> that explained. Many of the examples in the TOSCA spec are wrong or
> incomplete, too, so I wouldn't be shocked if this one is wrong as well.
>
> Anyway, none of this addresses the core issue in my opinion, which I keep
> returning to: these kinds of parameters we add here -- SSH user, password,
> etc. -- are meant as *directives to our execution plugin*, conceptually
> very different from operation inputs. Mixing them together is confusing
> (what do you do with name overlaps?) as well as a major security concern.
> These two kinds of values simply should not be mixed together. Indeed, in
> TOSCA 1.2 they might end up being artifact properties.
>
>
> On Mon, Sep 11, 2017 at 11:25 AM, Avia Efrat <av...@cloudify.co> wrote:
>
> > In contrary to most of the TOSCA entities, TOSCA does not differentiate
> > between 'definition' and 'assignment' in the context of operations. There
> > are only "operation definitions" [3.5.13]. Logically, there is a partial
> > differentiation [3.5.13.1], where inputs in node type operations are
> > expected to be property definitions, and inputs in node template
> operation
> > are expected to be property assignments (which are actual value
> > assignments). Both of these options are listed under "operation
> > definition".
> >
> > Under [3.5.13.3] it is explicitly stated that
> > "Template authors MAY provide property *assignments* on operation inputs
> on
> > *templates* that do not necessarily have a property definition defined in
> > its corresponding type." (my emphasis)
> >
> > That is, from this paragraph, I think that it is clear that you can
> define
> > operation inputs in node templates (level #3) without them being defined
> in
> > the node template's type. In fact, [2.14.2] is an example of doing just
> > that. And out parser treats such a syntax as valid, see:
> > https://github.com/apache/incubator-ariatosca/blob/
> > master/tests/parser/test_tosca_simple_v1_0/test_end2end.py#L73
> >
> > Me thinking it was a good idea to construct TOSCA that way is another
> thing
> > =)
> >
> > On Mon, Sep 11, 2017 at 6:35 PM, Tal Liron <ta...@cloudify.co> wrote:
> >
> > > Feel free to change the wiki, Ran, to whatever name you find
> appropriate.
> > >
> > > I think what Avia discovered is not new to us and actually doesn't
> solve
> > > the problem, unfortunately. Let me go over what is clearly allowed and
> > not
> > > allowed in TOSCA, confusing because there are a few levels of
> inheritance
> > > here.
> > >
> > > 1. Interface types. Obviously, you are allowed to inherit an interface
> > type
> > > and add or override inputs. (ARIA insists that overridden input types
> be
> > > derived from what it is they are overriding, too, to keep the OO
> contract
> > > intact.)
> > >
> > > 2. Node types. In the "interfaces" section you can define several
> > > interfaces of various types. Here, again, TOSCA lets you add/override
> > > inputs. Though note here that the line of inheritance is quite complex:
> > you
> > > can override inputs from the interface type, but *also* from the parent
> > > node type. (ARIA here has to do some work to make sure that you are
> doing
> > > it all OK and not breaking the OO contract, it's a rather complex part
> of
> > > the parser code.)
> > >
> > > I think the above is what Avia discovered. However, the third level is
> > > locked to us:
> > >
> > > 3. Node templates. Unfortunately, here we can not add inputs ad hoc.
> The
> > > "interfaces" section here is not the same DSL format as those above: it
> > is
> > > operation *assignments* rather than operation *definitions*. When you
> > > assign input values here, they are validated against the defined types.
> > It
> > > would make no sense in TOSCA to just assign values without a type.
> > >
> > > So, because we can't add inputs at level #3, we still have a problem:
> we
> > > would have to derive new node types for every type of execution. SSH
> > would
> > > require its own node types, Juju would require its own node types,
> Puppet
> > > would require its own node types, etc. And that's for *all* your node
> > > types. This seems extremely un-scalable.
> > >
> > > But also, as I try to explain in the wiki page, I insist that these
> kinds
> > > of configuration parameters are essentially not the same as operation
> > > inputs. They are not meant to be used by the operation itself (script,
> > > charm, recipe, etc.), rather by the mechanism that executes the
> operation
> > > (SSH, Juju, Puppet, etc.). Especially I point out the security hole:
> you
> > > don't want an SSH password exposed and sent over the wire to the script
> > > itself. It is simply not an input.
> > >
> > > By the way, it seems that there's some acknowledgement by other folk in
> > > OASIS about this gap in TOSCA, and there's an interest to use artifact
> > > types in TOSCA 1.2 as a way to solve this problem. I don't think it's a
> > bit
> > > awkward, but at least there will be a standard solution.
> > >
> > >
> > > On Sun, Sep 10, 2017 at 2:13 AM, Ran Ziv <ra...@cloudify.co> wrote:
> > >
> > > > Avia's mentioned at one point that we might have misunderstood the
> spec
> > > at
> > > > this section, and that in fact it can be possible to pass arbitrary
> > > inputs
> > > > into operations regardless of the interface definition - which would
> > mean
> > > > this notion of "configuration" might be unnecessary.
> > > >
> > > > Also, note that the doc page is talking about "executors", which is
> > > > confusing as that's a different concept in ARIA (see the base
> executor
> > > > class); Supposedly up until now we've simply called these "operation
> > > > plugins".
> > > >
> > > >
> > > > On Sat, Sep 9, 2017 at 1:43 AM, Tal Liron <ta...@cloudify.co> wrote:
> > > >
> > > > > Yes:
> > > > > https://cwiki.apache.org/confluence/display/ARIATOSCA/
> > > > > Execution+Configuration
> > > > >
> > > > > On Fri, Sep 8, 2017 at 4:45 PM, DeWayne Filppi <
> dewayne@cloudify.co>
> > > > > wrote:
> > > > >
> > > > > > I see in the examples a list of dependencies for scripts in
> > > operations,
> > > > > for
> > > > > > example:
> > > > > >
> > > > > > implementation:
> > > > > > primary: scripts/configure.sh
> > > > > > dependencies:
> > > > > > - "ssh.user > { get_input: ssh_username }"
> > > > > > - "ssh.key_filename > { get_input:
> private_key_path
> > > }"
> > > > > > - "ssh.address > { get_attribute: [ virtual_ip,
> > > > > > floating_ip_address ] }"
> > > > > >
> > > > > > The spec seems to indicate that the dependency list is for
> > resources
> > > > that
> > > > > > need to be made available so the main script can be run. Clearly
> > > that
> > > > > > isn't the case in the example. Is this '>' syntax documented
> > > anywhere?
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: operation dependencies
Posted by Tal Liron <ta...@cloudify.co>.
The single sentence you mention in the 3.5.13.3 is the only place that
*might* be implying that ad hoc, type-less input assignments are allowed,
but I actually think it could have a different reading. What i meant is:
"...that do not necessarily have a property definition defined in its
corresponding *interface* type." I think what is implied is what I mention
in level #2: a node template could have assignments that are not in the
interface type, *but obviously they have to be defined in the node type*.
Obvious, I say, because they have to be defined somewhere: an undefined
property assignment makes little sense in TOSCA. So let me rewrite that
sentence with more clarity, without (I think) changing its original meaning:
"Template authors MAY provide property assignments on operation inputs on
templates that do not necessarily have a property definition defined in its
corresponding interface type by adding an input definition to the node
type."
To be more honest here: when I say "I think what is implied" I have to
wonder if the author was completely clear on what was implied. Like many
parts of spec, there's a lot of theory that clashes in practice. We are
confused, but it could be that the author was confused as well. By
implementing the spec we have a responsibility to pin down what's fuzzy.
The "partial" differentiation as you call it in 3.5.13.1 is what I meant:
in some places you use property definitions, in other assignments. The
language is very clear that you use definitions with types and assignments
with templates. They didn't explicitly have an "operation assignment"
grammar, but it's clear that there are two different grammars. When I say
"operation assignment", it's a shorthand for the kind of operation
definition that you find in templates which uses property assignments. This
is consistent with everything else in TOSCA.
And indeed this is exactly what we see example 2.14.2, too, where we see
property assignments. But this example snippet doesn't prove much of
anything, because we don't see the node type defined, just used here.
The "wp_db_port"
input could/should be declared there. The fact that ARIA accepts this has
always been very problematic to me: we here have an input that is untyped.
I consider this a bug. As we discussed, this particular example *might*
make a bit of sense because we are using an intrinsic function, which could
carry the type with it... but nowhere in the spec is any mechanism like
that explained. Many of the examples in the TOSCA spec are wrong or
incomplete, too, so I wouldn't be shocked if this one is wrong as well.
Anyway, none of this addresses the core issue in my opinion, which I keep
returning to: these kinds of parameters we add here -- SSH user, password,
etc. -- are meant as *directives to our execution plugin*, conceptually
very different from operation inputs. Mixing them together is confusing
(what do you do with name overlaps?) as well as a major security concern.
These two kinds of values simply should not be mixed together. Indeed, in
TOSCA 1.2 they might end up being artifact properties.
On Mon, Sep 11, 2017 at 11:25 AM, Avia Efrat <av...@cloudify.co> wrote:
> In contrary to most of the TOSCA entities, TOSCA does not differentiate
> between 'definition' and 'assignment' in the context of operations. There
> are only "operation definitions" [3.5.13]. Logically, there is a partial
> differentiation [3.5.13.1], where inputs in node type operations are
> expected to be property definitions, and inputs in node template operation
> are expected to be property assignments (which are actual value
> assignments). Both of these options are listed under "operation
> definition".
>
> Under [3.5.13.3] it is explicitly stated that
> "Template authors MAY provide property *assignments* on operation inputs on
> *templates* that do not necessarily have a property definition defined in
> its corresponding type." (my emphasis)
>
> That is, from this paragraph, I think that it is clear that you can define
> operation inputs in node templates (level #3) without them being defined in
> the node template's type. In fact, [2.14.2] is an example of doing just
> that. And out parser treats such a syntax as valid, see:
> https://github.com/apache/incubator-ariatosca/blob/
> master/tests/parser/test_tosca_simple_v1_0/test_end2end.py#L73
>
> Me thinking it was a good idea to construct TOSCA that way is another thing
> =)
>
> On Mon, Sep 11, 2017 at 6:35 PM, Tal Liron <ta...@cloudify.co> wrote:
>
> > Feel free to change the wiki, Ran, to whatever name you find appropriate.
> >
> > I think what Avia discovered is not new to us and actually doesn't solve
> > the problem, unfortunately. Let me go over what is clearly allowed and
> not
> > allowed in TOSCA, confusing because there are a few levels of inheritance
> > here.
> >
> > 1. Interface types. Obviously, you are allowed to inherit an interface
> type
> > and add or override inputs. (ARIA insists that overridden input types be
> > derived from what it is they are overriding, too, to keep the OO contract
> > intact.)
> >
> > 2. Node types. In the "interfaces" section you can define several
> > interfaces of various types. Here, again, TOSCA lets you add/override
> > inputs. Though note here that the line of inheritance is quite complex:
> you
> > can override inputs from the interface type, but *also* from the parent
> > node type. (ARIA here has to do some work to make sure that you are doing
> > it all OK and not breaking the OO contract, it's a rather complex part of
> > the parser code.)
> >
> > I think the above is what Avia discovered. However, the third level is
> > locked to us:
> >
> > 3. Node templates. Unfortunately, here we can not add inputs ad hoc. The
> > "interfaces" section here is not the same DSL format as those above: it
> is
> > operation *assignments* rather than operation *definitions*. When you
> > assign input values here, they are validated against the defined types.
> It
> > would make no sense in TOSCA to just assign values without a type.
> >
> > So, because we can't add inputs at level #3, we still have a problem: we
> > would have to derive new node types for every type of execution. SSH
> would
> > require its own node types, Juju would require its own node types, Puppet
> > would require its own node types, etc. And that's for *all* your node
> > types. This seems extremely un-scalable.
> >
> > But also, as I try to explain in the wiki page, I insist that these kinds
> > of configuration parameters are essentially not the same as operation
> > inputs. They are not meant to be used by the operation itself (script,
> > charm, recipe, etc.), rather by the mechanism that executes the operation
> > (SSH, Juju, Puppet, etc.). Especially I point out the security hole: you
> > don't want an SSH password exposed and sent over the wire to the script
> > itself. It is simply not an input.
> >
> > By the way, it seems that there's some acknowledgement by other folk in
> > OASIS about this gap in TOSCA, and there's an interest to use artifact
> > types in TOSCA 1.2 as a way to solve this problem. I don't think it's a
> bit
> > awkward, but at least there will be a standard solution.
> >
> >
> > On Sun, Sep 10, 2017 at 2:13 AM, Ran Ziv <ra...@cloudify.co> wrote:
> >
> > > Avia's mentioned at one point that we might have misunderstood the spec
> > at
> > > this section, and that in fact it can be possible to pass arbitrary
> > inputs
> > > into operations regardless of the interface definition - which would
> mean
> > > this notion of "configuration" might be unnecessary.
> > >
> > > Also, note that the doc page is talking about "executors", which is
> > > confusing as that's a different concept in ARIA (see the base executor
> > > class); Supposedly up until now we've simply called these "operation
> > > plugins".
> > >
> > >
> > > On Sat, Sep 9, 2017 at 1:43 AM, Tal Liron <ta...@cloudify.co> wrote:
> > >
> > > > Yes:
> > > > https://cwiki.apache.org/confluence/display/ARIATOSCA/
> > > > Execution+Configuration
> > > >
> > > > On Fri, Sep 8, 2017 at 4:45 PM, DeWayne Filppi <de...@cloudify.co>
> > > > wrote:
> > > >
> > > > > I see in the examples a list of dependencies for scripts in
> > operations,
> > > > for
> > > > > example:
> > > > >
> > > > > implementation:
> > > > > primary: scripts/configure.sh
> > > > > dependencies:
> > > > > - "ssh.user > { get_input: ssh_username }"
> > > > > - "ssh.key_filename > { get_input: private_key_path
> > }"
> > > > > - "ssh.address > { get_attribute: [ virtual_ip,
> > > > > floating_ip_address ] }"
> > > > >
> > > > > The spec seems to indicate that the dependency list is for
> resources
> > > that
> > > > > need to be made available so the main script can be run. Clearly
> > that
> > > > > isn't the case in the example. Is this '>' syntax documented
> > anywhere?
> > > > >
> > > >
> > >
> >
>
Re: operation dependencies
Posted by Avia Efrat <av...@cloudify.co>.
In contrary to most of the TOSCA entities, TOSCA does not differentiate
between 'definition' and 'assignment' in the context of operations. There
are only "operation definitions" [3.5.13]. Logically, there is a partial
differentiation [3.5.13.1], where inputs in node type operations are
expected to be property definitions, and inputs in node template operation
are expected to be property assignments (which are actual value
assignments). Both of these options are listed under "operation
definition".
Under [3.5.13.3] it is explicitly stated that
"Template authors MAY provide property *assignments* on operation inputs on
*templates* that do not necessarily have a property definition defined in
its corresponding type." (my emphasis)
That is, from this paragraph, I think that it is clear that you can define
operation inputs in node templates (level #3) without them being defined in
the node template's type. In fact, [2.14.2] is an example of doing just
that. And out parser treats such a syntax as valid, see:
https://github.com/apache/incubator-ariatosca/blob/master/tests/parser/test_tosca_simple_v1_0/test_end2end.py#L73
Me thinking it was a good idea to construct TOSCA that way is another thing
=)
On Mon, Sep 11, 2017 at 6:35 PM, Tal Liron <ta...@cloudify.co> wrote:
> Feel free to change the wiki, Ran, to whatever name you find appropriate.
>
> I think what Avia discovered is not new to us and actually doesn't solve
> the problem, unfortunately. Let me go over what is clearly allowed and not
> allowed in TOSCA, confusing because there are a few levels of inheritance
> here.
>
> 1. Interface types. Obviously, you are allowed to inherit an interface type
> and add or override inputs. (ARIA insists that overridden input types be
> derived from what it is they are overriding, too, to keep the OO contract
> intact.)
>
> 2. Node types. In the "interfaces" section you can define several
> interfaces of various types. Here, again, TOSCA lets you add/override
> inputs. Though note here that the line of inheritance is quite complex: you
> can override inputs from the interface type, but *also* from the parent
> node type. (ARIA here has to do some work to make sure that you are doing
> it all OK and not breaking the OO contract, it's a rather complex part of
> the parser code.)
>
> I think the above is what Avia discovered. However, the third level is
> locked to us:
>
> 3. Node templates. Unfortunately, here we can not add inputs ad hoc. The
> "interfaces" section here is not the same DSL format as those above: it is
> operation *assignments* rather than operation *definitions*. When you
> assign input values here, they are validated against the defined types. It
> would make no sense in TOSCA to just assign values without a type.
>
> So, because we can't add inputs at level #3, we still have a problem: we
> would have to derive new node types for every type of execution. SSH would
> require its own node types, Juju would require its own node types, Puppet
> would require its own node types, etc. And that's for *all* your node
> types. This seems extremely un-scalable.
>
> But also, as I try to explain in the wiki page, I insist that these kinds
> of configuration parameters are essentially not the same as operation
> inputs. They are not meant to be used by the operation itself (script,
> charm, recipe, etc.), rather by the mechanism that executes the operation
> (SSH, Juju, Puppet, etc.). Especially I point out the security hole: you
> don't want an SSH password exposed and sent over the wire to the script
> itself. It is simply not an input.
>
> By the way, it seems that there's some acknowledgement by other folk in
> OASIS about this gap in TOSCA, and there's an interest to use artifact
> types in TOSCA 1.2 as a way to solve this problem. I don't think it's a bit
> awkward, but at least there will be a standard solution.
>
>
> On Sun, Sep 10, 2017 at 2:13 AM, Ran Ziv <ra...@cloudify.co> wrote:
>
> > Avia's mentioned at one point that we might have misunderstood the spec
> at
> > this section, and that in fact it can be possible to pass arbitrary
> inputs
> > into operations regardless of the interface definition - which would mean
> > this notion of "configuration" might be unnecessary.
> >
> > Also, note that the doc page is talking about "executors", which is
> > confusing as that's a different concept in ARIA (see the base executor
> > class); Supposedly up until now we've simply called these "operation
> > plugins".
> >
> >
> > On Sat, Sep 9, 2017 at 1:43 AM, Tal Liron <ta...@cloudify.co> wrote:
> >
> > > Yes:
> > > https://cwiki.apache.org/confluence/display/ARIATOSCA/
> > > Execution+Configuration
> > >
> > > On Fri, Sep 8, 2017 at 4:45 PM, DeWayne Filppi <de...@cloudify.co>
> > > wrote:
> > >
> > > > I see in the examples a list of dependencies for scripts in
> operations,
> > > for
> > > > example:
> > > >
> > > > implementation:
> > > > primary: scripts/configure.sh
> > > > dependencies:
> > > > - "ssh.user > { get_input: ssh_username }"
> > > > - "ssh.key_filename > { get_input: private_key_path
> }"
> > > > - "ssh.address > { get_attribute: [ virtual_ip,
> > > > floating_ip_address ] }"
> > > >
> > > > The spec seems to indicate that the dependency list is for resources
> > that
> > > > need to be made available so the main script can be run. Clearly
> that
> > > > isn't the case in the example. Is this '>' syntax documented
> anywhere?
> > > >
> > >
> >
>
Re: operation dependencies
Posted by Tal Liron <ta...@cloudify.co>.
Feel free to change the wiki, Ran, to whatever name you find appropriate.
I think what Avia discovered is not new to us and actually doesn't solve
the problem, unfortunately. Let me go over what is clearly allowed and not
allowed in TOSCA, confusing because there are a few levels of inheritance
here.
1. Interface types. Obviously, you are allowed to inherit an interface type
and add or override inputs. (ARIA insists that overridden input types be
derived from what it is they are overriding, too, to keep the OO contract
intact.)
2. Node types. In the "interfaces" section you can define several
interfaces of various types. Here, again, TOSCA lets you add/override
inputs. Though note here that the line of inheritance is quite complex: you
can override inputs from the interface type, but *also* from the parent
node type. (ARIA here has to do some work to make sure that you are doing
it all OK and not breaking the OO contract, it's a rather complex part of
the parser code.)
I think the above is what Avia discovered. However, the third level is
locked to us:
3. Node templates. Unfortunately, here we can not add inputs ad hoc. The
"interfaces" section here is not the same DSL format as those above: it is
operation *assignments* rather than operation *definitions*. When you
assign input values here, they are validated against the defined types. It
would make no sense in TOSCA to just assign values without a type.
So, because we can't add inputs at level #3, we still have a problem: we
would have to derive new node types for every type of execution. SSH would
require its own node types, Juju would require its own node types, Puppet
would require its own node types, etc. And that's for *all* your node
types. This seems extremely un-scalable.
But also, as I try to explain in the wiki page, I insist that these kinds
of configuration parameters are essentially not the same as operation
inputs. They are not meant to be used by the operation itself (script,
charm, recipe, etc.), rather by the mechanism that executes the operation
(SSH, Juju, Puppet, etc.). Especially I point out the security hole: you
don't want an SSH password exposed and sent over the wire to the script
itself. It is simply not an input.
By the way, it seems that there's some acknowledgement by other folk in
OASIS about this gap in TOSCA, and there's an interest to use artifact
types in TOSCA 1.2 as a way to solve this problem. I don't think it's a bit
awkward, but at least there will be a standard solution.
On Sun, Sep 10, 2017 at 2:13 AM, Ran Ziv <ra...@cloudify.co> wrote:
> Avia's mentioned at one point that we might have misunderstood the spec at
> this section, and that in fact it can be possible to pass arbitrary inputs
> into operations regardless of the interface definition - which would mean
> this notion of "configuration" might be unnecessary.
>
> Also, note that the doc page is talking about "executors", which is
> confusing as that's a different concept in ARIA (see the base executor
> class); Supposedly up until now we've simply called these "operation
> plugins".
>
>
> On Sat, Sep 9, 2017 at 1:43 AM, Tal Liron <ta...@cloudify.co> wrote:
>
> > Yes:
> > https://cwiki.apache.org/confluence/display/ARIATOSCA/
> > Execution+Configuration
> >
> > On Fri, Sep 8, 2017 at 4:45 PM, DeWayne Filppi <de...@cloudify.co>
> > wrote:
> >
> > > I see in the examples a list of dependencies for scripts in operations,
> > for
> > > example:
> > >
> > > implementation:
> > > primary: scripts/configure.sh
> > > dependencies:
> > > - "ssh.user > { get_input: ssh_username }"
> > > - "ssh.key_filename > { get_input: private_key_path }"
> > > - "ssh.address > { get_attribute: [ virtual_ip,
> > > floating_ip_address ] }"
> > >
> > > The spec seems to indicate that the dependency list is for resources
> that
> > > need to be made available so the main script can be run. Clearly that
> > > isn't the case in the example. Is this '>' syntax documented anywhere?
> > >
> >
>
Re: operation dependencies
Posted by Ran Ziv <ra...@cloudify.co>.
Avia's mentioned at one point that we might have misunderstood the spec at
this section, and that in fact it can be possible to pass arbitrary inputs
into operations regardless of the interface definition - which would mean
this notion of "configuration" might be unnecessary.
Also, note that the doc page is talking about "executors", which is
confusing as that's a different concept in ARIA (see the base executor
class); Supposedly up until now we've simply called these "operation
plugins".
On Sat, Sep 9, 2017 at 1:43 AM, Tal Liron <ta...@cloudify.co> wrote:
> Yes:
> https://cwiki.apache.org/confluence/display/ARIATOSCA/
> Execution+Configuration
>
> On Fri, Sep 8, 2017 at 4:45 PM, DeWayne Filppi <de...@cloudify.co>
> wrote:
>
> > I see in the examples a list of dependencies for scripts in operations,
> for
> > example:
> >
> > implementation:
> > primary: scripts/configure.sh
> > dependencies:
> > - "ssh.user > { get_input: ssh_username }"
> > - "ssh.key_filename > { get_input: private_key_path }"
> > - "ssh.address > { get_attribute: [ virtual_ip,
> > floating_ip_address ] }"
> >
> > The spec seems to indicate that the dependency list is for resources that
> > need to be made available so the main script can be run. Clearly that
> > isn't the case in the example. Is this '>' syntax documented anywhere?
> >
>
Re: operation dependencies
Posted by Tal Liron <ta...@cloudify.co>.
Yes:
https://cwiki.apache.org/confluence/display/ARIATOSCA/Execution+Configuration
On Fri, Sep 8, 2017 at 4:45 PM, DeWayne Filppi <de...@cloudify.co> wrote:
> I see in the examples a list of dependencies for scripts in operations, for
> example:
>
> implementation:
> primary: scripts/configure.sh
> dependencies:
> - "ssh.user > { get_input: ssh_username }"
> - "ssh.key_filename > { get_input: private_key_path }"
> - "ssh.address > { get_attribute: [ virtual_ip,
> floating_ip_address ] }"
>
> The spec seems to indicate that the dependency list is for resources that
> need to be made available so the main script can be run. Clearly that
> isn't the case in the example. Is this '>' syntax documented anywhere?
>