You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ariatosca.apache.org by Ran Ziv <ra...@cloudify.co> on 2017/09/03 09:06:23 UTC
Re: Service Composition / Substitution Mapping
I'm not sure I've followed why a policy would be necessary.
I too agree with the notion that picking a substituting service template
explicitly is against the spirit of this feature, for the reasons Avia has
explained.
If a specific template is needed, why not import it directly, or require a
more specific capability offered by the substituting service template?
On Thu, Aug 31, 2017 at 7:09 PM, Tal Liron <ta...@cloudify.co> wrote:
> Regarding #1, I agree with Avia here. Substitution mapping is intended as
> an extension of requirements-matching-capabilities. As you know, a
> requirement can be as minimal as just specifying a capability type, but
> could optionally additionally specify a node and even include a node filter
> with more constraints. Unfortunately the substitution mapping grammar is
> still not very sophisticated in TOSCA 1.0, and while you can require a
> node, you can't specifically require it to be a substituted service
> template. That's why I'm thinking we may need to introduce a policy to fill
> in this gap for now. But indeed, if there is no such policy, available
> service templates could be tried arbitrarily one at a time until one
> satisfies the requirement (as Avia points out, this would require the
> composite service to be *entirely* successfully satisfied).
>
> On Thu, Aug 31, 2017 at 10:23 AM, Avia Efrat <av...@cloudify.co> wrote:
>
> > 1) Substitutable nodes (abstract nodes) are defined by their type. And a
> > node type should represent specific node/subsystem characteristics.
> Having
> > two node types ( with the same name) that represent different logical
> > entities seems odd, and indicates a possible bug in the design. That is
> > another reason why using so-called namespaces while creating type
> > hierarchies is a good practice.
> >
> > You could argue that if you have multiple sources for templates (e.g.
> your
> > own storage and a external catalog), there could be unexpected name
> > collisions of node types names. Having said that, we need to remember the
> > entire process of the substitution mapping.
> >
> > First, as you said, we are trying to locate a service template that has a
> > substitution mapping definition of the same type of the abstract node in
> > the top level template. But choosing the service template is only the
> first
> > step. The second step is instantiating the top level template along with
> > the substitution template. During the instantiation, requirements and
> > capabilities are matched. If not all requirements are satisfied
> > (considering both top level and substituting template), the substitution
> > mapping is not considered successful, and the instantiation fails.
> >
> > If that is the case, it makes sense to go ahead and try the next
> > substitution template that fits the type, and continue like so until we
> > find a substitution where all the requirements are satisfied during
> > instantiation.
> >
> > At this point you may be wondering, "what if the instantiation was
> > successful, but the source of the substitution template wasn't the one
> that
> > *I* wanted?". Well, You have to think about this issue in substitution
> > mapping terms. that is, when you use an abstract node, actually what you
> > are saying is "I want something of type t, the has the capabilities c,
> and
> > requires r. regarding the internals, I don't care, I just want it to
> work".
> > So actually, selecting a specific substituting template (as long as the
> > type, the requirements and the capabilities are as expected, of course),
> is
> > somewhat against the spirit of the substitution mapping feature.
> >
> > 2) I'm sorry, but I didn't get to the bottom of your question. Could you
> > elaborate a bit more, and include a small example?
> >
> >
> >
> >
> > On Thu, Aug 31, 2017 at 11:35 AM, D Jayachandran <
> > d.jayachandran@ericsson.com> wrote:
> >
> > > Hi,
> > >
> > > With respect to substituting stored service templates, I have few
> things
> > > to be clarified
> > >
> > > 1) Handling substitution when multiple service templates matches for
> the
> > > abstract node type .
> > > Would the 1st match would be used for substitution or Are we
> > > looking at policy to enable user to select particular service templates
> > for
> > > substitution with multiple service templates ?
> > > 2) Custom node types as abstract node type.
> > > With custom node types as abstract node type, there seems to
> be a
> > > need to implicitly import that custom node type in our top level
> service
> > > template so that the parser recognizes this custom type.
> > > Assuming the abstract node type would be substituted from a
> > stored
> > > substituting service template, we need to at least import the custom
> node
> > > types and have it part of the same CSAR package.
> > > Would this be a challenge for the top-level service template
> > > author in including and importing the custom node types as abstraction
> ?
> > or
> > > Is this how we are looking at custom node types ?
> > > Is it possible to identify an abstract node during parsing ,
> such
> > > as if it does not contain any implementation ( The SPEC does not say
> > > anything on this )?
> > >
> > > Regards,
> > > DJ
> > > -----Original Message-----
> > > From: Ran Ziv [mailto:ran@cloudify.co]
> > > Sent: Wednesday, August 16, 2017 6:19 PM
> > > To: dev@ariatosca.incubator.apache.org
> > > Subject: Re: Service Composition / Substitution Mapping
> > >
> > > I agree, especially when the benefit of being able to use an existing
> > > service - yet only one which hasn't been deployed via a workflow -
> > doesn't
> > > seem all that interesting IMO.
> > >
> > > Another concern I could add to the ones you've mentioned is the
> service's
> > > inputs - the substituting template's inputs should be received via the
> > > properties of the abstract node in the top level service template. If
> the
> > > service already exists, these inputs would not be passed as expected.
> > >
> > > Ran
> > >
> > > On Wed, Aug 16, 2017 at 3:25 PM, D Jayachandran <
> > > d.jayachandran@ericsson.com
> > > > wrote:
> > >
> > > > Hi Ran,
> > > >
> > > > When Tal mentioned about "substituting service", I thought it was
> > > > about the services which dint have any associated
> executions/workflows
> > > triggered.
> > > > Am also in favor of "substituting service templates" rather than
> > > > "substituting service".
> > > > With "substituting service" approach (when the service is not
> > > > instantiated), I see some open points
> > > > - In a multi-user scenario, what will happen when a service
> is
> > > > composed using the substituting service and at the sametime a
> > > > workflow is triggered for the substituting service. ?
> > > > - Is it okay to delete(dissolve) the substituting service
> > > > after it is used to create the composed service. ?
> > > >
> > > > Starting with it might be a good idea to only have "substituting
> > > > service templates" approach.
> > > >
> > > > Regards,
> > > > DJ
> > > > -----Original Message-----
> > > > From: Ran Ziv [mailto:ran@cloudify.co]
> > > > Sent: Wednesday, August 16, 2017 4:29 PM
> > > > To: dev@ariatosca.incubator.apache.org
> > > > Subject: Re: Service Composition / Substitution Mapping
> > > >
> > > > I'd say right now we're looking at "static service composition" which
> > > > is only about "substituting service templates", not "substituting
> > > > service". If a service is already running, it will not be used.
> > > >
> > > > I think what Tal meant was that each service template - whether the
> > > > top level one or one of the substituting templates - needs to resolve
> > > > its inner reqs&caps internally first, and then resolve substitution
> > > > reqs&caps across service templates.
> > > >
> > > >
> > > > On Wed, Aug 16, 2017 at 12:00 PM, D Jayachandran <
> > > > d.jayachandran@ericsson.com> wrote:
> > > >
> > > > > Hi Tal,
> > > > >
> > > > > Thanks for organizing the points.
> > > > > So if I understand correctly we are looking only at "Static service
> > > > > composition" which includes "substituting service template" and
> > > > > "substituting service".
> > > > >
> > > > > As you said with "substituting service template" approach ,we will
> > > > > have all the nodes aggregated from other service templates and a
> > > > > single workflow would be triggered to perform life-cycle operation
> > > > > on
> > > > all the nodes.
> > > > > Am not sure why the workflows needs to be "boundary aware" for
> nodes
> > > > > being substituted ? I see nodes are already part of the composed
> > > > > service, Could you please help me understand this ?
> > > > >
> > > > >
> > > > > Regards,
> > > > > DJ
> > > > > -----Original Message-----
> > > > > From: Tal Liron [mailto:tal@cloudify.co]
> > > > > Sent: Saturday, August 12, 2017 4:52 AM
> > > > > To: dev@ariatosca.incubator.apache.org
> > > > > Subject: Re: Service Composition / Substitution Mapping
> > > > >
> > > > > You are correct -- to participate in this "multi-VIM" scenario, the
> > > > > Openstack plugin would have to know how to translate the TOSCA
> > > > > properties to a flavor ID. This could all be done in 100% TOSCA via
> > > > > policies (say, an aria.Openstack).
> > > > >
> > > > > Doing this automatically might not be a good idea, or even
> necessary.
> > > > > Worst case is you get a validation error if the ARIA plugin can't
> > > > > find a flavor in the table to match your requirements, in which you
> > > > > case you can go and manually find the right ID and add it to the
> > table.
> > > > >
> > > > > And I agree about being fine with imperfection: the rule of thumb
> > > > > for our work is always to allow for sensible defaults even if no
> > > > > explicit policy is given.
> > > > >
> > > > > Anyway, we've gone way off the topic of this thread. We can talk
> > > > > about it more once it comes closer to an implementation.
> > > > >
> > > > > On Fri, Aug 11, 2017 at 3:52 PM, DeWayne Filppi
> > > > > <de...@cloudify.co>
> > > > > wrote:
> > > > >
> > > > > > Interesting. Take Openstack (please <rim shot>). If you model a
> > > > > > compute OS as explicitly as you like, there still has to be a
> > "match"
> > > > > > to an Openstack image ID. Or are you saying you must supply the
> > > > > > image
> > > > > ID for
> > > > > > OSs. Likewise, you can't supply RAM and CPUs without a flavor
> ID.
> > > > > > Openstack does allow for custom flavors, but I doubt the plugin
> is
> > > > > > doing that. Much better to have a "caps-init" interface in some
> > > > > > low down base type that can be triggered at service creation to
> > > > > > support reqs/caps (IMHO). Then the plugin can verify whether the
> > > > > > service can be construct based on fuzzy constraints. Maybe this
> > > > > > is a case that the "general solution" is a nightmare of
> > > > > > complexity, but having a plugin scan the available flavors to
> make
> > > > > > sure a requirement can be met doesn't seem that tough. If TOSCA
> > > > > > provided a formal lifecycle interface for it, then orchestrators
> > > > > > or just plugins could determine how tricky they wanted to be.
> > > > > > IOW, let not the perfect be the enemy of
> > > > > the good.
> > > > > >
> > > > > > DeWayne
> > > > > >
> > > > > >
> > > > > > On Fri, Aug 11, 2017 at 1:26 PM, Tal Liron <ta...@cloudify.co>
> > wrote:
> > > > > >
> > > > > > > OK, that's a whole different can of worms. :)
> > > > > > >
> > > > > > > TOSCA's Compute capabilities (Container and OperatingSystem)
> are
> > > > > > explicit.
> > > > > > > You specify which OS you want, how much RAM you want, how many
> > > > > > > CPUs,
> > > > > etc.
> > > > > > > ARIA's explicit node types (for example, the AWS Compute node)
> > > > > > > are
> > > > > > likewise
> > > > > > > explicit. So there is not querying here: the plugin will
> attempt
> > > > > > > to spin
> > > > > > up
> > > > > > > exactly the virtual machine you asked for. If it fails, the
> > > > > > > workflow will fail.
> > > > > > >
> > > > > > > This is not good enough, I think, for real world scenarios.
> > > > > > > There are two possible solutions:
> > > > > > >
> > > > > > > 1) Support ranges or fallbacks. So instead of saying "I need 4
> > > > > > > GB of
> > > > > RAM"
> > > > > > > you can say "I would like 4 GB of RAM, but 2 GB would also be
> > OK."
> > > > > > There's
> > > > > > > no easy way to do this now without totally changing how the
> > > > > > > capability types are designed. But, it may be possible to
> > > > > > > override this via
> > > > > > policies.
> > > > > > > So, the capabilities would perhaps specify the minimal
> > > > > > > requirements,
> > > > > > while
> > > > > > > policies specify preferences. Some aspects of this were
> > > > > > > discussed in the OPEN-O project. DeWayne, has any of this
> > > > > > > survived in ONAP, or have we not reached that point in the
> > > discussion yet?
> > > > > > >
> > > > > > > 2) Incorporate this into the bigger topic of resource
> > > orchestration.
> > > > > > This a
> > > > > > > huge challenge for the industry. The problem field contains not
> > > > > > > just "I need X amount of RAM" but also "I want all my virtual
> > > > > > > machines and containers running on the same high-performance
> > > > > > > network backend and have these two nodes on the same bare-metal
> > > > > > > machine or at least in the same
> > > > > > data
> > > > > > > center rack with a NUMA interconnect, and I also don't want all
> > > > > > > this to cost more that $100 per hour." That's not crazy: these
> > > > > > > are real world requirements for high-performance VNFs and
> > > > > > > service
> > > > chaining.
> > > > > > > Resource orchestration requires a full map of what is available
> > > > > > > in the data
> > > > > > centers,
> > > > > > > a negotiation-based algorithm for properly allocating and
> > > > > > > placing resources, connection to billing services, etc. Of
> > > > > > > course resource orchestration is not within the scope of ARIA,
> > > > > > > but it would be great for ARIA to have plugins for them (and
> > > > > > > TOSCA be able to model resource requirement policies) when
> these
> > > become available.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Aug 11, 2017 at 3:02 PM, DeWayne Filppi
> > > > > > > <de...@cloudify.co>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > For the "resource realization" part, I was not even
> considering
> > > > > > > > multicloud/vim. I was considering single cloud even outside
> > of
> > > > > > > > composition. Just reqs and caps. If my node "requires" a
> > > > > > > > compute node with Suse Linux version X with a minimum of 4GB
> > > > > > > > RAM, how does
> > > > > the
> > > > > > > > orchestrator match that without querying the cloud via the
> > > plugin?
> > > > > If
> > > > > > > it
> > > > > > > > does query it, which it seems it must in addition to the
> > > > > > > > implicit quota query, how is this done? TOSCA seems to not
> > > > > > > > really care, which is fine
> > > > > > > and
> > > > > > > > perhaps a good idea. But ARIA has to care.
> > > > > > > >
> > > > > > > > DeWayne
> > > > > > > >
> > > > > > > > On Fri, Aug 11, 2017 at 12:34 PM, Tal Liron <tal@cloudify.co
> >
> > > > wrote:
> > > > > > > >
> > > > > > > > > In my opinion, the new composite service should indeed be a
> > > > > > > > > single
> > > > > > > > service,
> > > > > > > > > so the ARIA CLI will show it as one (if a substituting
> > > > > > > > > service
> > > > > > already
> > > > > > > > > existed, it would be "dissolved" into the new top-level
> > > > > > > > > service). The composition will show its traces if you look
> > > > > > > > > more deeply, because
> > > > > > > you'll
> > > > > > > > > see that some node templates come from a different service
> > > > > template.
> > > > > > > > > Perhaps our CLI can detect this and tell the user which
> > > > > > > > > service
> > > > > > > templates
> > > > > > > > > were used to create the nodes in current service.
> > > > > > > > >
> > > > > > > > > The rest of what you described I think is not related
> > > > > > > > > directly to
> > > > > > > service
> > > > > > > > > composition, but resource realization. The way we currently
> > > > > > > > > handle
> > > > > > this
> > > > > > > > is
> > > > > > > > > by being very explicit: you have to use the derived Compute
> > > > > > > > > types
> > > > > > > > included
> > > > > > > > > in our AWS or Openstack plugins, for example.
> > > > > > > > >
> > > > > > > > > I think we should still keep this, because sometimes you
> > > > > > > > > want to be
> > > > > > > very
> > > > > > > > > explicit (and use specific features of the platform), but I
> > > > > > > > > actually
> > > > > > > have
> > > > > > > > > some ideas for multi-VIM support that would work
> differently:
> > > > > > > > > the
> > > > > > idea
> > > > > > > is
> > > > > > > > > to use the basic Compute type, with detailed properties in
> > > > > > > > > its
> > > > > > > Container
> > > > > > > > > and OperatingSystem capabilities. There would then be a
> > > > > > > > > policy,
> > > > > > perhaps
> > > > > > > > > aria.Realization, that would hint at what kind of Compute
> > > > > > > > > nodes you
> > > > > > are
> > > > > > > > > looking for (you can apply the policy to specific nodes or
> > > > > > > > > groups, or generally). It might be possible to still have
> > > > > > > > > strong
> > > > > > platform-specific
> > > > > > > > > feature support here, perhaps by implementing a general
> > > > > > > > > string
> > > > > > > key-value
> > > > > > > > > dict that could include hints for the specific plugin to
> use.
> > > > > > > > >
> > > > > > > > > Anyway, just ideas at this point...
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, Aug 11, 2017 at 2:25 PM, DeWayne Filppi
> > > > > > > > > <dewayne@cloudify.co
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Good stuff. Obviously a "fancy include" was the wrong
> > > > metaphor.
> > > > > > The
> > > > > > > > > > lifecycles are linked though. When a create the
> > > > > > > > > > referencing
> > > > > > service
> > > > > > > > > (aria
> > > > > > > > > > service create), ARIA will match the other service
> > > > > > > > > > template, and
> > > > > > run
> > > > > > > > > create
> > > > > > > > > > service on it, and then continue on. Uninstall of the
> > > > > > > > > > referencer
> > > > > > > will
> > > > > > > > > > destroy the referenced service. After such an
> > > > > > > > > > instantiation, would
> > > > > > > > > listing
> > > > > > > > > > all services produce two (perhaps with some parent
> > > > > > > > > > indicator) or
> > > > > > one?
> > > > > > > > > From
> > > > > > > > > > what I've seen, reqs and caps don't have the equivalent
> of
> > > > > > > > > > a
> > > > > > > lifecycle
> > > > > > > > > > interface associated. Maybe I missed it. It is implicit
> > > > > > > > > > in the orchestrator to match stuff up at service creation
> > > > > > > > > > time, I'm
> > > > > > > assuming,
> > > > > > > > > and
> > > > > > > > > > using plugins somehow? The sort of flagship scenario is
> > > > > > > > > > matching
> > > > > > > cloud
> > > > > > > > > > compute resources without specifying an image. Since
> > > > > > > > > > discovering
> > > > > > > > > whether a
> > > > > > > > > > particular cloud has a given capability, I guessing a
> > > > > > > > > > plugin for
> > > > > > that
> > > > > > > > > cloud
> > > > > > > > > > has to search cloud inventory and perhaps quota to
> provide
> > > > > > > > > > info for
> > > > > > > the
> > > > > > > > > > match. Since plugins (at least the Cloudify variety)
> tend
> > > > > > > > > > to be
> > > > > > > > > triggered
> > > > > > > > > > due to operations that they connect to lifecycle methods,
> > > > > > > > > > and there
> > > > > > > is
> > > > > > > > no
> > > > > > > > > > "materialize_capabilities" interface that I noticed, some
> > > > > > > > > > other
> > > > > > kind
> > > > > > > of
> > > > > > > > > > magic must be performed. Maybe this is an
> > > > > > > > > > orchestrator-specific
> > > > > > > detail
> > > > > > > > > > outside of TOSCA.
> > > > > > > > > >
> > > > > > > > > > DeWayne
> > > > > > > > > >
> > > > > > > > > > On Fri, Aug 11, 2017 at 10:57 AM, Tal Liron
> > > > > > > > > > <ta...@cloudify.co>
> > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > OK, let me try to organize this differently. Three
> > > > > > > > > > > potential
> > > > > > > > > conceptions
> > > > > > > > > > > here:
> > > > > > > > > > >
> > > > > > > > > > > 1) A "fancy include," as you say. All that would happen
> > > > > > > > > > > here is
> > > > > > > that
> > > > > > > > > the
> > > > > > > > > > > TOSCA parser would automatically find the service
> > > > > > > > > > > template to
> > > > > > > include
> > > > > > > > > > from
> > > > > > > > > > > some kind of repository of recognized service
> templates,
> > > > > > > > > > > and just
> > > > > > > > > include
> > > > > > > > > > > that. The language in the TOSCA spec suggests that this
> > > > > > > > > > > is not
> > > > > > the
> > > > > > > > > > > intention: it is something that happens at the
> > > > "orchestrator,"
> > > > > > not
> > > > > > > > the
> > > > > > > > > > > "parser."
> > > > > > > > > > >
> > > > > > > > > > > 2) Static service composition. This happens not at the
> > > > > > > > > > > parsing
> > > > > > > stage,
> > > > > > > > > but
> > > > > > > > > > > rather the instantiation stage, where reqs-and-caps
> > > happens.
> > > > > > > > > > > I
> > > > > > > think
> > > > > > > > > this
> > > > > > > > > > > is what is intended: substitution mapping is
> > > > > > > > > > > specifically about
> > > > > > > > mapping
> > > > > > > > > > > reqs-and-caps. And this is also where things like
> > > > > > > > > > > scalability and
> > > > > > > > > > placement
> > > > > > > > > > > happen: think of a requirement matching a capability,
> > > > > > > > > > > but that
> > > > > > > > > capability
> > > > > > > > > > > not having enough capacity. So, the node might be able
> > > > > > > > > > > to scale
> > > > > > > out:
> > > > > > > > in
> > > > > > > > > > the
> > > > > > > > > > > case of substitution, this would mean duplicating an
> > > > > > > > > > > entire
> > > > > > service
> > > > > > > > > > > instance. My understanding is that this is what is
> > > > > > > > > > > intended by
> > > > > > > TOSCA,
> > > > > > > > > and
> > > > > > > > > > > it's entirely within the scope of what we can do. We've
> > > > > > > > > > > recently
> > > > > > > just
> > > > > > > > > > > refactored the instantiation phase into a new
> "topology"
> > > > > > > > > > > module
> > > > > > > where
> > > > > > > > > > > exactly all this logic is concentrated. So think of it
> > > > > > > > > > > this way
> > > > > > --
> > > > > > > > ARIA
> > > > > > > > > > has
> > > > > > > > > > > three parts: "parser," "topology manager," and
> "workflow
> > > > > engine"
> > > > > > > > > > > ("orchestrator").
> > > > > > > > > > >
> > > > > > > > > > > (I think I might have confused some thing here a bit
> > > > > > > > > > > when
> > > > > > > mentioning
> > > > > > > > a
> > > > > > > > > > > "logical proxy node." I do not mean an actual piece of
> > > > > > > > > > > proxy
> > > > > > > software
> > > > > > > > > > > running on a machine somewhere. I mean just a data
> point
> > > > > > > > > > > in the ARIA-generated topology that can be used as a
> > > > > > > > > > > barrier of sorts
> > > > > > when
> > > > > > > > > > > constructing the task graph -- because the task graph
> > > > > > > > > > > follows relationships, the edges of the topology. It
> > > > > > > > > > > could be that we
> > > > > > > > discover
> > > > > > > > > in
> > > > > > > > > > > our POC that this is not needed, because actually the
> > > > > > substitution
> > > > > > > > node
> > > > > > > > > > is
> > > > > > > > > > > already part of our topology data model and we might be
> > > > > > > > > > > able to
> > > > > > > > easily
> > > > > > > > > > take
> > > > > > > > > > > that into account when generating the task graph.)
> > > > > > > > > > >
> > > > > > > > > > > 3) Live service composition. I think this is a fantasy
> > > > > > > > > > > of some
> > > > > > > > people:
> > > > > > > > > > that
> > > > > > > > > > > ARIA would be able to take existing, "running" service
> > > > > > > > > > > chains and
> > > > > > > run
> > > > > > > > > > > workflows with them. I do think this is a solvable
> > > > > > > > > > > problem, but
> > > > > > not
> > > > > > > > via
> > > > > > > > > > > substitution mapping per se. A solution could involve
> > > > > > > > > > > deployment
> > > > > > > of a
> > > > > > > > > > proxy
> > > > > > > > > > > service (which could actually be encapsulated in a
> > > > > > > > > > > substitution
> > > > > > > > > mapping,
> > > > > > > > > > > but doesn't have to be), or configuring specialized
> > > > > > > > > > > virtual ports
> > > > > > > via
> > > > > > > > > an
> > > > > > > > > > > SDN controller, or via logical proxy nodes created via
> > > > > > > > > > > an
> > > > > > > inspection
> > > > > > > > > > tool,
> > > > > > > > > > > etc. I cannot believe that there is a one-size-fits-all
> > > > > > > > > > > solution
> > > > > > to
> > > > > > > > > this
> > > > > > > > > > > problem. The dream of NFV might be to have everything
> > > > > > > > > > > connect to
> > > > > > > each
> > > > > > > > > > other
> > > > > > > > > > > like LEGO blocks, but there are far too many protocols,
> > > > > > > > configuration,
> > > > > > > > > > and
> > > > > > > > > > > security standards. I think point #2, what I called
> > > > > > > > > > > "static
> > > > > > service
> > > > > > > > > > > composition," is a realistic compromise and within the
> > > > > > > > > > > scope of
> > > > > > > what
> > > > > > > > > > TOSCA
> > > > > > > > > > > promises.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Aug 11, 2017 at 12:08 PM, DeWayne Filppi <
> > > > > > > > dewayne@cloudify.co>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > To my eyes, the spec doesn't speak of runtime service
> > > > > > > substituion,
> > > > > > > > > but
> > > > > > > > > > > > parsetime template composition. IOW, substitution
> > > > > > > > > > > > mapping is a
> > > > > > > > fancy
> > > > > > > > > > > > "include", or the equivalent of an interface
> > definition.
> > > > > > > > > > > > Is it
> > > > > > > > > > > understood
> > > > > > > > > > > > by the ARIA team that this includes proxying of
> > > > > > > > > > > > running
> > > > > > services?
> > > > > > > > > IOW,
> > > > > > > > > > > if
> > > > > > > > > > > > my template requires a database service that my
> > > > > > > > > > > > template does
> > > > > > > *not*
> > > > > > > > > > want
> > > > > > > > > > > to
> > > > > > > > > > > > control the lifecycle of, I can "substitution map" an
> > > > > > > > > > > > instance
> > > > > > > of a
> > > > > > > > > > > > template (i.e. a running service)? This would be a
> > > > > > > > > > > > lovely
> > > > > > > feature,
> > > > > > > > > but
> > > > > > > > > > > > it's not really a "substitution map", rather more of
> a
> > > > > > > > > > > > "service
> > > > > > > > > proxy"
> > > > > > > > > > > (as
> > > > > > > > > > > > implemented as a plugin in Cloudify). Just trying
> to
> > > > > clarify.
> > > > > > > > > Maybe
> > > > > > > > > > > the
> > > > > > > > > > > > community thinks that "substitution map" as something
> > > > > > > > > > > > that
> > > > > > occurs
> > > > > > > > > > beyond
> > > > > > > > > > > > parsetime, or should.
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Aug 11, 2017 at 9:52 AM, Tal Liron
> > > > > > > > > > > > <ta...@cloudify.co>
> > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Well, DJ, it's all just opinion at this stage and
> > > > > > > > > > > > > feedback is
> > > > > > > > > > welcome!
> > > > > > > > > > > > >
> > > > > > > > > > > > > Option 1:
> > > > > > > > > > > > > > To look at satisfying nodes present in a
> > > > > > substituting
> > > > > > > > > > > service,
> > > > > > > > > > > > > > Have these nodes part of the newly created
> service
> > > > > > > > > > > > > > and
> > > > > > remove
> > > > > > > > the
> > > > > > > > > > > > > > substituting service(nodes with different ID's.
> > > > > > > > > > > > > > Also we are
> > > > > > > > very
> > > > > > > > > > much
> > > > > > > > > > > > in
> > > > > > > > > > > > > > favor of UUID )
> > > > > > > > > > > > > > With this approach I guess the
> > > > > > > > > > > > > > substituting service
> > > > > > > > > should
> > > > > > > > > > > not
> > > > > > > > > > > > > > have any associated workflows running. If at all
> > > > > > > > > > > > > > an
> > > > > > workflow
> > > > > > > > > > > execution
> > > > > > > > > > > > is
> > > > > > > > > > > > > > already triggered I hope this service will not be
> > > > > > considered
> > > > > > > > for
> > > > > > > > > > > > > > substitution.
> > > > > > > > > > > > > > I hope this is the correct approach when
> > > > > > > > > > > > > > we are
> > > > > > > looking
> > > > > > > > > at
> > > > > > > > > > a
> > > > > > > > > > > > > > service for the substitution
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Yes, this is a good idea. It would be easy to
> > > > > > > > > > > > > discover this
> > > > > > > > > according
> > > > > > > > > > > to
> > > > > > > > > > > > > the stored node state -- just make sure that all
> > > > > > > > > > > > > nodes are
> > > > > > in a
> > > > > > > > > > stable
> > > > > > > > > > > > > state before composition.
> > > > > > > > > > > > >
> > > > > > > > > > > > > This leads to a general issue: before composition,
> > > > > > > > > > > > > the
> > > > > > > > substituting
> > > > > > > > > > > > > services must be validated in some way before
> > > > > > > > > > > > > composition
> > > > > > > begins.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Also, let's all start using TOSCA terminology here:
> > > > > > > > > > > > > the
> > > > > > > > containing
> > > > > > > > > > > > service
> > > > > > > > > > > > > is called the "top-level service," and the
> contained
> > > > > > > > > > > > > services
> > > > > > > are
> > > > > > > > > > > called
> > > > > > > > > > > > > "substituting services."
> > > > > > > > > > > > >
> > > > > > > > > > > > > Also, we keep using trivial examples, but it's
> > > > > > > > > > > > > definitely
> > > > > > > > possible
> > > > > > > > > > for
> > > > > > > > > > > a
> > > > > > > > > > > > > top-level service to require several substituting
> > > > > > > > > > > > > services at
> > > > > > > the
> > > > > > > > > > same
> > > > > > > > > > > > > time. I can definitely see such things happening in
> > > > > > > > > > > > > NFV with
> > > > > > > even
> > > > > > > > > > > simple
> > > > > > > > > > > > > service chains. Basically every VNF could be a
> > > > > > > > > > > > > substituting
> > > > > > > > > service.
> > > > > > > > > > > > >
> > > > > > > > > > > > > So, actually one of the validations would be to
> make
> > > > > > > > > > > > > sure you
> > > > > > > do
> > > > > > > > > not
> > > > > > > > > > > > create
> > > > > > > > > > > > > circular composition: if the top-level also has
> > > > > > > > > > > > > substitution
> > > > > > > > > > mappings,
> > > > > > > > > > > > you
> > > > > > > > > > > > > need to make sure that one of the substituting ones
> > > > > > > > > > > > > doesn't
> > > > > > > > require
> > > > > > > > > > it.
> > > > > > > > > > > > :)
> > > > > > > > > > > > > Not a very likely situation, but it would lead to
> > > > failure.
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Option 2:
> > > > > > > > > > > > > > While creating a service look at the
> > > > > > > > > > > > > > req-caps at
> > > > > > the
> > > > > > > > > > > > > > service-template level and create a service
> > > > > > > > > > > > > > including the
> > > > > > > nodes
> > > > > > > > > > > > provided
> > > > > > > > > > > > > by
> > > > > > > > > > > > > > the substituting service-template. With this
> > > > > > > > > > > > > > approach there
> > > > > > > > would
> > > > > > > > > > not
> > > > > > > > > > > > be
> > > > > > > > > > > > > > any service created from the service-template
> > which
> > > > is
> > > > > > > > > providing
> > > > > > > > > > > the
> > > > > > > > > > > > > > substitution functionality. The service-template
> > > > > > > > > > > > > > would
> > > > > > remain
> > > > > > > > the
> > > > > > > > > > > same
> > > > > > > > > > > > > but
> > > > > > > > > > > > > > only the service would be added with extra nodes.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Are you considering both option 1 & 2 for the
> > > > > > implementation
> > > > > > > ?
> > > > > > > > If
> > > > > > > > > > not
> > > > > > > > > > > > > > which one do you feel which take priority. I see
> > > > > > > > > > > > > > option 2
> > > > > > at
> > > > > > > > this
> > > > > > > > > > > stage
> > > > > > > > > > > > > > could be the best possible approach Also could
> you
> > > > > > > > > > > > > > please let me know a tentative time for this
> > > > > > > > > feature
> > > > > > > > > > > to
> > > > > > > > > > > > be
> > > > > > > > > > > > > > available?
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > I think both options 1 and 2 make sense and are
> > > > > > > > > > > > > useful, and
> > > > > > > > > actually
> > > > > > > > > > > one
> > > > > > > > > > > > is
> > > > > > > > > > > > > a subset of the other.
> > > > > > > > > > > > >
> > > > > > > > > > > > > With option #2 (substituting a service template),
> it
> > > > > > > > > > > > > means
> > > > > > new
> > > > > > > > > nodes
> > > > > > > > > > > are
> > > > > > > > > > > > > instantiated and the composed service would include
> > > > > > > > > > > > > all
> > > > > > nodes.
> > > > > > > > So,
> > > > > > > > > an
> > > > > > > > > > > > > "install" workflow would install everything at
> once.
> > > > > > > > > > > > > In this
> > > > > > > case
> > > > > > > > > we
> > > > > > > > > > do
> > > > > > > > > > > > > need to fix the lifecycle workflows to be "boundary
> > > > aware,"
> > > > > > so
> > > > > > > > that
> > > > > > > > > > > > > workflows of substituting service nodes are part of
> > > > > > > > > > > > > their own
> > > > > > > > task
> > > > > > > > > > > > graph. I
> > > > > > > > > > > > > think that possibly using a logical proxy node in
> > > > > > > > > > > > > between
> > > > > > might
> > > > > > > > > solve
> > > > > > > > > > > > this
> > > > > > > > > > > > > situation automatically.
> > > > > > > > > > > > >
> > > > > > > > > > > > > With option #1 (substituting a service) the
> > > > > > > > > > > > > substituting
> > > > > > nodes
> > > > > > > > > might
> > > > > > > > > > > > > already be installed. Or not (they might have been
> > > > > > instantiated
> > > > > > > > but
> > > > > > > > > > > still
> > > > > > > > > > > > > not installed). So, the lifecycle workflows should
> > > > > > > > > > > > > only work
> > > > > > on
> > > > > > > > the
> > > > > > > > > > > nodes
> > > > > > > > > > > > > that have not yet been installed.
> > > > > > > > > > > > >
> > > > > > > > > > > > > The point is that we just need to beef up the
> > > > > > > > > > > > > lifecycle
> > > > > > > workflows
> > > > > > > > > to
> > > > > > > > > > > > > properly work with boundaries and with a mix of
> > > > > > > > > > > > > nodes in
> > > > > > > > different
> > > > > > > > > > > > states.
> > > > > > > > > > > > > If they can do that, then they can handle any kind
> > > > > > > > > > > > > of service
> > > > > > > > > > > > composition,
> > > > > > > > > > > > > whether it's option #1 or option #2.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I don't think we can provide a timeline yet, but I
> > > > > > > > > > > > > will say
> > > > > > > that
> > > > > > > > we
> > > > > > > > > > are
> > > > > > > > > > > > in
> > > > > > > > > > > > > the research phase and may have a POC soon. Avia is
> > > > > > > > > > > > > in charge
> > > > > > > of
> > > > > > > > > this
> > > > > > > > > > > > JIRA,
> > > > > > > > > > > > > so I will let him chime in with the current state
> of
> > > > > things.
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
RE: Service Composition / Substitution Mapping
Posted by D Jayachandran <d....@ericsson.com>.
Thanks Tal. I agree this needs to taken up with OASIS.
Having said that ,there is statement in TOSCA Spec 2.9
" In TOSCA templates, nodes are either:
· Concrete: meaning that they have a deployment and/or one or more implementation artifacts that are declared on the “create” operation of the node’s Standard lifecycle interface, or they are
· Abstract: where the template describes the node type along with its required capabilities and properties that must be satisfied."
I thought if the above statement would anyway help us in identifying an abstract node.
Regards,
DJ
-----Original Message-----
From: Tal Liron [mailto:tal@cloudify.co]
Sent: Tuesday, September 05, 2017 8:30 PM
To: dev@ariatosca.incubator.apache.org
Subject: Re: Service Composition / Substitution Mapping
On Tue, Sep 5, 2017 at 5:33 AM, D Jayachandran <d....@ericsson.com>
wrote:
> Hence I wanted to know if there is any possibility of identifying an
> abstract type (during parsing )so that we need not import or define
> the custom node type in my service template.
>
In my opinion, the import of the abstract type is necessary. TOSCA offers no alternative, and I don't see how ARIA would be able to guess that this type is abstract nor have any notion where to find the type. I think that such usability problems we find in the mechanism need to be addressed at OASIS. Perhaps we can make TOSCA 1.2 easier to use with substitution mapping. For now, though, I think ARIA should adhere to the TOSCA 1.0 standard as closely as possible.
Re: Service Composition / Substitution Mapping
Posted by Tal Liron <ta...@cloudify.co>.
On Tue, Sep 5, 2017 at 5:33 AM, D Jayachandran <d....@ericsson.com>
wrote:
> Hence I wanted to know if there is any possibility of identifying an
> abstract type (during parsing )so that we need not import or define the
> custom node type in my service template.
>
In my opinion, the import of the abstract type is necessary. TOSCA offers
no alternative, and I don't see how ARIA would be able to guess that this
type is abstract nor have any notion where to find the type. I think that
such usability problems we find in the mechanism need to be addressed at
OASIS. Perhaps we can make TOSCA 1.2 easier to use with substitution
mapping. For now, though, I think ARIA should adhere to the TOSCA 1.0
standard as closely as possible.
RE: Service Composition / Substitution Mapping
Posted by D Jayachandran <d....@ericsson.com>.
Hi All,
Thanks for the comments on handling substitution when multiple service templates matches for the abstract node type.
My 2nd point was on importing the non-normative node types in the top level service template. Let me explain this with an example.
1) Normative type (tosca.nodes.Database) as abstract node
db:
# This node is abstract (no Deploment or Implemenation artifacts on create)
# and can be substituted with a topology provided by another template
# that exports a Database type’s capabilities.
type: tosca.nodes.Database
properties:
user: my_db_user
password: secret
name: my_db_name
With this example, the abstract node type is a normative type from the simple YAML 1.0 profile. Hence with our proposed approach we will go through the available service templates and find the required substitution.
2) Non-normative type (myapp.nodes.Database)
If I replace the above example with the non-normative type " myapp.nodes.Database" , we need to import this type in our service template to have the parsing complete.
import types/my_custom.yaml
node_templates:
db:
# This node is abstract (no Deploment or Implemenation artifacts on create)
# and can be substituted with a topology provided by another template
# that exports a Database type’s capabilities.
type: myapp.nodes.Database
properties:
user: my_db_user
password: secret
name: my_db_name
Here "db" is an abstract node of type "myapp.nodes.Database" which needs to be substituted from other service templates. Though it will be substituted we need to have this type "myapp.nodes.Database" defined in my top level service template or imported for the parsing to be sucessfull. Hence I wanted to know if there is any possibility of identifying an abstract type (during parsing )so that we need not import or define the custom node type in my service template.
Regards,
DJ
-----Original Message-----
From: Ran Ziv [mailto:ran@cloudify.co]
Sent: Sunday, September 03, 2017 2:36 PM
To: dev@ariatosca.incubator.apache.org
Subject: Re: Service Composition / Substitution Mapping
I'm not sure I've followed why a policy would be necessary.
I too agree with the notion that picking a substituting service template explicitly is against the spirit of this feature, for the reasons Avia has explained.
If a specific template is needed, why not import it directly, or require a more specific capability offered by the substituting service template?
On Thu, Aug 31, 2017 at 7:09 PM, Tal Liron <ta...@cloudify.co> wrote:
> Regarding #1, I agree with Avia here. Substitution mapping is intended
> as an extension of requirements-matching-capabilities. As you know, a
> requirement can be as minimal as just specifying a capability type,
> but could optionally additionally specify a node and even include a
> node filter with more constraints. Unfortunately the substitution
> mapping grammar is still not very sophisticated in TOSCA 1.0, and
> while you can require a node, you can't specifically require it to be
> a substituted service template. That's why I'm thinking we may need to
> introduce a policy to fill in this gap for now. But indeed, if there
> is no such policy, available service templates could be tried
> arbitrarily one at a time until one satisfies the requirement (as Avia
> points out, this would require the composite service to be *entirely* successfully satisfied).
>
> On Thu, Aug 31, 2017 at 10:23 AM, Avia Efrat <av...@cloudify.co> wrote:
>
> > 1) Substitutable nodes (abstract nodes) are defined by their type.
> > And a node type should represent specific node/subsystem characteristics.
> Having
> > two node types ( with the same name) that represent different
> > logical entities seems odd, and indicates a possible bug in the
> > design. That is another reason why using so-called namespaces while
> > creating type hierarchies is a good practice.
> >
> > You could argue that if you have multiple sources for templates (e.g.
> your
> > own storage and a external catalog), there could be unexpected name
> > collisions of node types names. Having said that, we need to
> > remember the entire process of the substitution mapping.
> >
> > First, as you said, we are trying to locate a service template that
> > has a substitution mapping definition of the same type of the
> > abstract node in the top level template. But choosing the service
> > template is only the
> first
> > step. The second step is instantiating the top level template along
> > with the substitution template. During the instantiation,
> > requirements and capabilities are matched. If not all requirements
> > are satisfied (considering both top level and substituting
> > template), the substitution mapping is not considered successful, and the instantiation fails.
> >
> > If that is the case, it makes sense to go ahead and try the next
> > substitution template that fits the type, and continue like so until
> > we find a substitution where all the requirements are satisfied
> > during instantiation.
> >
> > At this point you may be wondering, "what if the instantiation was
> > successful, but the source of the substitution template wasn't the
> > one
> that
> > *I* wanted?". Well, You have to think about this issue in
> > substitution mapping terms. that is, when you use an abstract node,
> > actually what you are saying is "I want something of type t, the has
> > the capabilities c,
> and
> > requires r. regarding the internals, I don't care, I just want it to
> work".
> > So actually, selecting a specific substituting template (as long as
> > the type, the requirements and the capabilities are as expected, of
> > course),
> is
> > somewhat against the spirit of the substitution mapping feature.
> >
> > 2) I'm sorry, but I didn't get to the bottom of your question. Could
> > you elaborate a bit more, and include a small example?
> >
> >
> >
> >
> > On Thu, Aug 31, 2017 at 11:35 AM, D Jayachandran <
> > d.jayachandran@ericsson.com> wrote:
> >
> > > Hi,
> > >
> > > With respect to substituting stored service templates, I have few
> things
> > > to be clarified
> > >
> > > 1) Handling substitution when multiple service templates matches
> > > for
> the
> > > abstract node type .
> > > Would the 1st match would be used for substitution or Are
> > > we looking at policy to enable user to select particular service
> > > templates
> > for
> > > substitution with multiple service templates ?
> > > 2) Custom node types as abstract node type.
> > > With custom node types as abstract node type, there seems
> > > to
> be a
> > > need to implicitly import that custom node type in our top level
> service
> > > template so that the parser recognizes this custom type.
> > > Assuming the abstract node type would be substituted from
> > > a
> > stored
> > > substituting service template, we need to at least import the
> > > custom
> node
> > > types and have it part of the same CSAR package.
> > > Would this be a challenge for the top-level service
> > > template author in including and importing the custom node types
> > > as abstraction
> ?
> > or
> > > Is this how we are looking at custom node types ?
> > > Is it possible to identify an abstract node during parsing
> > > ,
> such
> > > as if it does not contain any implementation ( The SPEC does not
> > > say anything on this )?
> > >
> > > Regards,
> > > DJ
> > > -----Original Message-----
> > > From: Ran Ziv [mailto:ran@cloudify.co]
> > > Sent: Wednesday, August 16, 2017 6:19 PM
> > > To: dev@ariatosca.incubator.apache.org
> > > Subject: Re: Service Composition / Substitution Mapping
> > >
> > > I agree, especially when the benefit of being able to use an
> > > existing service - yet only one which hasn't been deployed via a
> > > workflow -
> > doesn't
> > > seem all that interesting IMO.
> > >
> > > Another concern I could add to the ones you've mentioned is the
> service's
> > > inputs - the substituting template's inputs should be received via
> > > the properties of the abstract node in the top level service
> > > template. If
> the
> > > service already exists, these inputs would not be passed as expected.
> > >
> > > Ran
> > >
> > > On Wed, Aug 16, 2017 at 3:25 PM, D Jayachandran <
> > > d.jayachandran@ericsson.com
> > > > wrote:
> > >
> > > > Hi Ran,
> > > >
> > > > When Tal mentioned about "substituting service", I thought it
> > > > was about the services which dint have any associated
> executions/workflows
> > > triggered.
> > > > Am also in favor of "substituting service templates" rather
> > > > than "substituting service".
> > > > With "substituting service" approach (when the service is not
> > > > instantiated), I see some open points
> > > > - In a multi-user scenario, what will happen when a
> > > > service
> is
> > > > composed using the substituting service and at the sametime a
> > > > workflow is triggered for the substituting service. ?
> > > > - Is it okay to delete(dissolve) the substituting
> > > > service after it is used to create the composed service. ?
> > > >
> > > > Starting with it might be a good idea to only have "substituting
> > > > service templates" approach.
> > > >
> > > > Regards,
> > > > DJ
> > > > -----Original Message-----
> > > > From: Ran Ziv [mailto:ran@cloudify.co]
> > > > Sent: Wednesday, August 16, 2017 4:29 PM
> > > > To: dev@ariatosca.incubator.apache.org
> > > > Subject: Re: Service Composition / Substitution Mapping
> > > >
> > > > I'd say right now we're looking at "static service composition"
> > > > which is only about "substituting service templates", not
> > > > "substituting service". If a service is already running, it will not be used.
> > > >
> > > > I think what Tal meant was that each service template - whether
> > > > the top level one or one of the substituting templates - needs
> > > > to resolve its inner reqs&caps internally first, and then
> > > > resolve substitution reqs&caps across service templates.
> > > >
> > > >
> > > > On Wed, Aug 16, 2017 at 12:00 PM, D Jayachandran <
> > > > d.jayachandran@ericsson.com> wrote:
> > > >
> > > > > Hi Tal,
> > > > >
> > > > > Thanks for organizing the points.
> > > > > So if I understand correctly we are looking only at "Static
> > > > > service composition" which includes "substituting service
> > > > > template" and "substituting service".
> > > > >
> > > > > As you said with "substituting service template" approach ,we
> > > > > will have all the nodes aggregated from other service
> > > > > templates and a single workflow would be triggered to perform
> > > > > life-cycle operation on
> > > > all the nodes.
> > > > > Am not sure why the workflows needs to be "boundary aware" for
> nodes
> > > > > being substituted ? I see nodes are already part of the
> > > > > composed service, Could you please help me understand this ?
> > > > >
> > > > >
> > > > > Regards,
> > > > > DJ
> > > > > -----Original Message-----
> > > > > From: Tal Liron [mailto:tal@cloudify.co]
> > > > > Sent: Saturday, August 12, 2017 4:52 AM
> > > > > To: dev@ariatosca.incubator.apache.org
> > > > > Subject: Re: Service Composition / Substitution Mapping
> > > > >
> > > > > You are correct -- to participate in this "multi-VIM"
> > > > > scenario, the Openstack plugin would have to know how to
> > > > > translate the TOSCA properties to a flavor ID. This could all
> > > > > be done in 100% TOSCA via policies (say, an aria.Openstack).
> > > > >
> > > > > Doing this automatically might not be a good idea, or even
> necessary.
> > > > > Worst case is you get a validation error if the ARIA plugin
> > > > > can't find a flavor in the table to match your requirements,
> > > > > in which you case you can go and manually find the right ID
> > > > > and add it to the
> > table.
> > > > >
> > > > > And I agree about being fine with imperfection: the rule of
> > > > > thumb for our work is always to allow for sensible defaults
> > > > > even if no explicit policy is given.
> > > > >
> > > > > Anyway, we've gone way off the topic of this thread. We can
> > > > > talk about it more once it comes closer to an implementation.
> > > > >
> > > > > On Fri, Aug 11, 2017 at 3:52 PM, DeWayne Filppi
> > > > > <de...@cloudify.co>
> > > > > wrote:
> > > > >
> > > > > > Interesting. Take Openstack (please <rim shot>). If you
> > > > > > model a compute OS as explicitly as you like, there still
> > > > > > has to be a
> > "match"
> > > > > > to an Openstack image ID. Or are you saying you must supply
> > > > > > the image
> > > > > ID for
> > > > > > OSs. Likewise, you can't supply RAM and CPUs without a flavor
> ID.
> > > > > > Openstack does allow for custom flavors, but I doubt the
> > > > > > plugin
> is
> > > > > > doing that. Much better to have a "caps-init" interface in
> > > > > > some low down base type that can be triggered at service
> > > > > > creation to support reqs/caps (IMHO). Then the plugin can
> > > > > > verify whether the service can be construct based on fuzzy
> > > > > > constraints. Maybe this is a case that the "general
> > > > > > solution" is a nightmare of complexity, but having a plugin
> > > > > > scan the available flavors to
> make
> > > > > > sure a requirement can be met doesn't seem that tough. If
> > > > > > TOSCA provided a formal lifecycle interface for it, then
> > > > > > orchestrators or just plugins could determine how tricky they wanted to be.
> > > > > > IOW, let not the perfect be the enemy of
> > > > > the good.
> > > > > >
> > > > > > DeWayne
> > > > > >
> > > > > >
> > > > > > On Fri, Aug 11, 2017 at 1:26 PM, Tal Liron <ta...@cloudify.co>
> > wrote:
> > > > > >
> > > > > > > OK, that's a whole different can of worms. :)
> > > > > > >
> > > > > > > TOSCA's Compute capabilities (Container and
> > > > > > > OperatingSystem)
> are
> > > > > > explicit.
> > > > > > > You specify which OS you want, how much RAM you want, how
> > > > > > > many CPUs,
> > > > > etc.
> > > > > > > ARIA's explicit node types (for example, the AWS Compute
> > > > > > > node) are
> > > > > > likewise
> > > > > > > explicit. So there is not querying here: the plugin will
> attempt
> > > > > > > to spin
> > > > > > up
> > > > > > > exactly the virtual machine you asked for. If it fails,
> > > > > > > the workflow will fail.
> > > > > > >
> > > > > > > This is not good enough, I think, for real world scenarios.
> > > > > > > There are two possible solutions:
> > > > > > >
> > > > > > > 1) Support ranges or fallbacks. So instead of saying "I
> > > > > > > need 4 GB of
> > > > > RAM"
> > > > > > > you can say "I would like 4 GB of RAM, but 2 GB would also
> > > > > > > be
> > OK."
> > > > > > There's
> > > > > > > no easy way to do this now without totally changing how
> > > > > > > the capability types are designed. But, it may be possible
> > > > > > > to override this via
> > > > > > policies.
> > > > > > > So, the capabilities would perhaps specify the minimal
> > > > > > > requirements,
> > > > > > while
> > > > > > > policies specify preferences. Some aspects of this were
> > > > > > > discussed in the OPEN-O project. DeWayne, has any of this
> > > > > > > survived in ONAP, or have we not reached that point in the
> > > discussion yet?
> > > > > > >
> > > > > > > 2) Incorporate this into the bigger topic of resource
> > > orchestration.
> > > > > > This a
> > > > > > > huge challenge for the industry. The problem field
> > > > > > > contains not just "I need X amount of RAM" but also "I
> > > > > > > want all my virtual machines and containers running on the
> > > > > > > same high-performance network backend and have these two
> > > > > > > nodes on the same bare-metal machine or at least in the
> > > > > > > same
> > > > > > data
> > > > > > > center rack with a NUMA interconnect, and I also don't
> > > > > > > want all this to cost more that $100 per hour." That's not
> > > > > > > crazy: these are real world requirements for
> > > > > > > high-performance VNFs and service
> > > > chaining.
> > > > > > > Resource orchestration requires a full map of what is
> > > > > > > available in the data
> > > > > > centers,
> > > > > > > a negotiation-based algorithm for properly allocating and
> > > > > > > placing resources, connection to billing services, etc. Of
> > > > > > > course resource orchestration is not within the scope of
> > > > > > > ARIA, but it would be great for ARIA to have plugins for
> > > > > > > them (and TOSCA be able to model resource requirement
> > > > > > > policies) when
> these
> > > become available.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Aug 11, 2017 at 3:02 PM, DeWayne Filppi
> > > > > > > <de...@cloudify.co>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > For the "resource realization" part, I was not even
> considering
> > > > > > > > multicloud/vim. I was considering single cloud even outside
> > of
> > > > > > > > composition. Just reqs and caps. If my node "requires"
> > > > > > > > a compute node with Suse Linux version X with a minimum
> > > > > > > > of 4GB RAM, how does
> > > > > the
> > > > > > > > orchestrator match that without querying the cloud via
> > > > > > > > the
> > > plugin?
> > > > > If
> > > > > > > it
> > > > > > > > does query it, which it seems it must in addition to the
> > > > > > > > implicit quota query, how is this done? TOSCA seems to
> > > > > > > > not really care, which is fine
> > > > > > > and
> > > > > > > > perhaps a good idea. But ARIA has to care.
> > > > > > > >
> > > > > > > > DeWayne
> > > > > > > >
> > > > > > > > On Fri, Aug 11, 2017 at 12:34 PM, Tal Liron
> > > > > > > > <tal@cloudify.co
> >
> > > > wrote:
> > > > > > > >
> > > > > > > > > In my opinion, the new composite service should indeed
> > > > > > > > > be a single
> > > > > > > > service,
> > > > > > > > > so the ARIA CLI will show it as one (if a substituting
> > > > > > > > > service
> > > > > > already
> > > > > > > > > existed, it would be "dissolved" into the new
> > > > > > > > > top-level service). The composition will show its
> > > > > > > > > traces if you look more deeply, because
> > > > > > > you'll
> > > > > > > > > see that some node templates come from a different
> > > > > > > > > service
> > > > > template.
> > > > > > > > > Perhaps our CLI can detect this and tell the user
> > > > > > > > > which service
> > > > > > > templates
> > > > > > > > > were used to create the nodes in current service.
> > > > > > > > >
> > > > > > > > > The rest of what you described I think is not related
> > > > > > > > > directly to
> > > > > > > service
> > > > > > > > > composition, but resource realization. The way we
> > > > > > > > > currently handle
> > > > > > this
> > > > > > > > is
> > > > > > > > > by being very explicit: you have to use the derived
> > > > > > > > > Compute types
> > > > > > > > included
> > > > > > > > > in our AWS or Openstack plugins, for example.
> > > > > > > > >
> > > > > > > > > I think we should still keep this, because sometimes
> > > > > > > > > you want to be
> > > > > > > very
> > > > > > > > > explicit (and use specific features of the platform),
> > > > > > > > > but I actually
> > > > > > > have
> > > > > > > > > some ideas for multi-VIM support that would work
> differently:
> > > > > > > > > the
> > > > > > idea
> > > > > > > is
> > > > > > > > > to use the basic Compute type, with detailed
> > > > > > > > > properties in its
> > > > > > > Container
> > > > > > > > > and OperatingSystem capabilities. There would then be
> > > > > > > > > a policy,
> > > > > > perhaps
> > > > > > > > > aria.Realization, that would hint at what kind of
> > > > > > > > > Compute nodes you
> > > > > > are
> > > > > > > > > looking for (you can apply the policy to specific
> > > > > > > > > nodes or groups, or generally). It might be possible
> > > > > > > > > to still have strong
> > > > > > platform-specific
> > > > > > > > > feature support here, perhaps by implementing a
> > > > > > > > > general string
> > > > > > > key-value
> > > > > > > > > dict that could include hints for the specific plugin
> > > > > > > > > to
> use.
> > > > > > > > >
> > > > > > > > > Anyway, just ideas at this point...
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, Aug 11, 2017 at 2:25 PM, DeWayne Filppi
> > > > > > > > > <dewayne@cloudify.co
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Good stuff. Obviously a "fancy include" was the
> > > > > > > > > > wrong
> > > > metaphor.
> > > > > > The
> > > > > > > > > > lifecycles are linked though. When a create the
> > > > > > > > > > referencing
> > > > > > service
> > > > > > > > > (aria
> > > > > > > > > > service create), ARIA will match the other service
> > > > > > > > > > template, and
> > > > > > run
> > > > > > > > > create
> > > > > > > > > > service on it, and then continue on. Uninstall of
> > > > > > > > > > the referencer
> > > > > > > will
> > > > > > > > > > destroy the referenced service. After such an
> > > > > > > > > > instantiation, would
> > > > > > > > > listing
> > > > > > > > > > all services produce two (perhaps with some parent
> > > > > > > > > > indicator) or
> > > > > > one?
> > > > > > > > > From
> > > > > > > > > > what I've seen, reqs and caps don't have the
> > > > > > > > > > equivalent
> of
> > > > > > > > > > a
> > > > > > > lifecycle
> > > > > > > > > > interface associated. Maybe I missed it. It is
> > > > > > > > > > implicit in the orchestrator to match stuff up at
> > > > > > > > > > service creation time, I'm
> > > > > > > assuming,
> > > > > > > > > and
> > > > > > > > > > using plugins somehow? The sort of flagship
> > > > > > > > > > scenario is matching
> > > > > > > cloud
> > > > > > > > > > compute resources without specifying an image.
> > > > > > > > > > Since discovering
> > > > > > > > > whether a
> > > > > > > > > > particular cloud has a given capability, I guessing
> > > > > > > > > > a plugin for
> > > > > > that
> > > > > > > > > cloud
> > > > > > > > > > has to search cloud inventory and perhaps quota to
> provide
> > > > > > > > > > info for
> > > > > > > the
> > > > > > > > > > match. Since plugins (at least the Cloudify
> > > > > > > > > > variety)
> tend
> > > > > > > > > > to be
> > > > > > > > > triggered
> > > > > > > > > > due to operations that they connect to lifecycle
> > > > > > > > > > methods, and there
> > > > > > > is
> > > > > > > > no
> > > > > > > > > > "materialize_capabilities" interface that I noticed,
> > > > > > > > > > some other
> > > > > > kind
> > > > > > > of
> > > > > > > > > > magic must be performed. Maybe this is an
> > > > > > > > > > orchestrator-specific
> > > > > > > detail
> > > > > > > > > > outside of TOSCA.
> > > > > > > > > >
> > > > > > > > > > DeWayne
> > > > > > > > > >
> > > > > > > > > > On Fri, Aug 11, 2017 at 10:57 AM, Tal Liron
> > > > > > > > > > <ta...@cloudify.co>
> > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > OK, let me try to organize this differently. Three
> > > > > > > > > > > potential
> > > > > > > > > conceptions
> > > > > > > > > > > here:
> > > > > > > > > > >
> > > > > > > > > > > 1) A "fancy include," as you say. All that would
> > > > > > > > > > > happen here is
> > > > > > > that
> > > > > > > > > the
> > > > > > > > > > > TOSCA parser would automatically find the service
> > > > > > > > > > > template to
> > > > > > > include
> > > > > > > > > > from
> > > > > > > > > > > some kind of repository of recognized service
> templates,
> > > > > > > > > > > and just
> > > > > > > > > include
> > > > > > > > > > > that. The language in the TOSCA spec suggests that
> > > > > > > > > > > this is not
> > > > > > the
> > > > > > > > > > > intention: it is something that happens at the
> > > > "orchestrator,"
> > > > > > not
> > > > > > > > the
> > > > > > > > > > > "parser."
> > > > > > > > > > >
> > > > > > > > > > > 2) Static service composition. This happens not at
> > > > > > > > > > > the parsing
> > > > > > > stage,
> > > > > > > > > but
> > > > > > > > > > > rather the instantiation stage, where
> > > > > > > > > > > reqs-and-caps
> > > happens.
> > > > > > > > > > > I
> > > > > > > think
> > > > > > > > > this
> > > > > > > > > > > is what is intended: substitution mapping is
> > > > > > > > > > > specifically about
> > > > > > > > mapping
> > > > > > > > > > > reqs-and-caps. And this is also where things like
> > > > > > > > > > > scalability and
> > > > > > > > > > placement
> > > > > > > > > > > happen: think of a requirement matching a
> > > > > > > > > > > capability, but that
> > > > > > > > > capability
> > > > > > > > > > > not having enough capacity. So, the node might be
> > > > > > > > > > > able to scale
> > > > > > > out:
> > > > > > > > in
> > > > > > > > > > the
> > > > > > > > > > > case of substitution, this would mean duplicating
> > > > > > > > > > > an entire
> > > > > > service
> > > > > > > > > > > instance. My understanding is that this is what is
> > > > > > > > > > > intended by
> > > > > > > TOSCA,
> > > > > > > > > and
> > > > > > > > > > > it's entirely within the scope of what we can do.
> > > > > > > > > > > We've recently
> > > > > > > just
> > > > > > > > > > > refactored the instantiation phase into a new
> "topology"
> > > > > > > > > > > module
> > > > > > > where
> > > > > > > > > > > exactly all this logic is concentrated. So think
> > > > > > > > > > > of it this way
> > > > > > --
> > > > > > > > ARIA
> > > > > > > > > > has
> > > > > > > > > > > three parts: "parser," "topology manager," and
> "workflow
> > > > > engine"
> > > > > > > > > > > ("orchestrator").
> > > > > > > > > > >
> > > > > > > > > > > (I think I might have confused some thing here a
> > > > > > > > > > > bit when
> > > > > > > mentioning
> > > > > > > > a
> > > > > > > > > > > "logical proxy node." I do not mean an actual
> > > > > > > > > > > piece of proxy
> > > > > > > software
> > > > > > > > > > > running on a machine somewhere. I mean just a data
> point
> > > > > > > > > > > in the ARIA-generated topology that can be used as
> > > > > > > > > > > a barrier of sorts
> > > > > > when
> > > > > > > > > > > constructing the task graph -- because the task
> > > > > > > > > > > graph follows relationships, the edges of the
> > > > > > > > > > > topology. It could be that we
> > > > > > > > discover
> > > > > > > > > in
> > > > > > > > > > > our POC that this is not needed, because actually
> > > > > > > > > > > the
> > > > > > substitution
> > > > > > > > node
> > > > > > > > > > is
> > > > > > > > > > > already part of our topology data model and we
> > > > > > > > > > > might be able to
> > > > > > > > easily
> > > > > > > > > > take
> > > > > > > > > > > that into account when generating the task graph.)
> > > > > > > > > > >
> > > > > > > > > > > 3) Live service composition. I think this is a
> > > > > > > > > > > fantasy of some
> > > > > > > > people:
> > > > > > > > > > that
> > > > > > > > > > > ARIA would be able to take existing, "running"
> > > > > > > > > > > service chains and
> > > > > > > run
> > > > > > > > > > > workflows with them. I do think this is a solvable
> > > > > > > > > > > problem, but
> > > > > > not
> > > > > > > > via
> > > > > > > > > > > substitution mapping per se. A solution could
> > > > > > > > > > > involve deployment
> > > > > > > of a
> > > > > > > > > > proxy
> > > > > > > > > > > service (which could actually be encapsulated in a
> > > > > > > > > > > substitution
> > > > > > > > > mapping,
> > > > > > > > > > > but doesn't have to be), or configuring
> > > > > > > > > > > specialized virtual ports
> > > > > > > via
> > > > > > > > > an
> > > > > > > > > > > SDN controller, or via logical proxy nodes created
> > > > > > > > > > > via an
> > > > > > > inspection
> > > > > > > > > > tool,
> > > > > > > > > > > etc. I cannot believe that there is a
> > > > > > > > > > > one-size-fits-all solution
> > > > > > to
> > > > > > > > > this
> > > > > > > > > > > problem. The dream of NFV might be to have
> > > > > > > > > > > everything connect to
> > > > > > > each
> > > > > > > > > > other
> > > > > > > > > > > like LEGO blocks, but there are far too many
> > > > > > > > > > > protocols,
> > > > > > > > configuration,
> > > > > > > > > > and
> > > > > > > > > > > security standards. I think point #2, what I
> > > > > > > > > > > called "static
> > > > > > service
> > > > > > > > > > > composition," is a realistic compromise and within
> > > > > > > > > > > the scope of
> > > > > > > what
> > > > > > > > > > TOSCA
> > > > > > > > > > > promises.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Aug 11, 2017 at 12:08 PM, DeWayne Filppi <
> > > > > > > > dewayne@cloudify.co>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > To my eyes, the spec doesn't speak of runtime
> > > > > > > > > > > > service
> > > > > > > substituion,
> > > > > > > > > but
> > > > > > > > > > > > parsetime template composition. IOW,
> > > > > > > > > > > > substitution mapping is a
> > > > > > > > fancy
> > > > > > > > > > > > "include", or the equivalent of an interface
> > definition.
> > > > > > > > > > > > Is it
> > > > > > > > > > > understood
> > > > > > > > > > > > by the ARIA team that this includes proxying of
> > > > > > > > > > > > running
> > > > > > services?
> > > > > > > > > IOW,
> > > > > > > > > > > if
> > > > > > > > > > > > my template requires a database service that my
> > > > > > > > > > > > template does
> > > > > > > *not*
> > > > > > > > > > want
> > > > > > > > > > > to
> > > > > > > > > > > > control the lifecycle of, I can "substitution
> > > > > > > > > > > > map" an instance
> > > > > > > of a
> > > > > > > > > > > > template (i.e. a running service)? This would
> > > > > > > > > > > > be a lovely
> > > > > > > feature,
> > > > > > > > > but
> > > > > > > > > > > > it's not really a "substitution map", rather
> > > > > > > > > > > > more of
> a
> > > > > > > > > > > > "service
> > > > > > > > > proxy"
> > > > > > > > > > > (as
> > > > > > > > > > > > implemented as a plugin in Cloudify). Just trying
> to
> > > > > clarify.
> > > > > > > > > Maybe
> > > > > > > > > > > the
> > > > > > > > > > > > community thinks that "substitution map" as
> > > > > > > > > > > > something that
> > > > > > occurs
> > > > > > > > > > beyond
> > > > > > > > > > > > parsetime, or should.
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Aug 11, 2017 at 9:52 AM, Tal Liron
> > > > > > > > > > > > <ta...@cloudify.co>
> > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Well, DJ, it's all just opinion at this stage
> > > > > > > > > > > > > and feedback is
> > > > > > > > > > welcome!
> > > > > > > > > > > > >
> > > > > > > > > > > > > Option 1:
> > > > > > > > > > > > > > To look at satisfying nodes present
> > > > > > > > > > > > > > in a
> > > > > > substituting
> > > > > > > > > > > service,
> > > > > > > > > > > > > > Have these nodes part of the newly created
> service
> > > > > > > > > > > > > > and
> > > > > > remove
> > > > > > > > the
> > > > > > > > > > > > > > substituting service(nodes with different ID's.
> > > > > > > > > > > > > > Also we are
> > > > > > > > very
> > > > > > > > > > much
> > > > > > > > > > > > in
> > > > > > > > > > > > > > favor of UUID )
> > > > > > > > > > > > > > With this approach I guess the
> > > > > > > > > > > > > > substituting service
> > > > > > > > > should
> > > > > > > > > > > not
> > > > > > > > > > > > > > have any associated workflows running. If at
> > > > > > > > > > > > > > all an
> > > > > > workflow
> > > > > > > > > > > execution
> > > > > > > > > > > > is
> > > > > > > > > > > > > > already triggered I hope this service will
> > > > > > > > > > > > > > not be
> > > > > > considered
> > > > > > > > for
> > > > > > > > > > > > > > substitution.
> > > > > > > > > > > > > > I hope this is the correct approach
> > > > > > > > > > > > > > when we are
> > > > > > > looking
> > > > > > > > > at
> > > > > > > > > > a
> > > > > > > > > > > > > > service for the substitution
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Yes, this is a good idea. It would be easy to
> > > > > > > > > > > > > discover this
> > > > > > > > > according
> > > > > > > > > > > to
> > > > > > > > > > > > > the stored node state -- just make sure that
> > > > > > > > > > > > > all nodes are
> > > > > > in a
> > > > > > > > > > stable
> > > > > > > > > > > > > state before composition.
> > > > > > > > > > > > >
> > > > > > > > > > > > > This leads to a general issue: before
> > > > > > > > > > > > > composition, the
> > > > > > > > substituting
> > > > > > > > > > > > > services must be validated in some way before
> > > > > > > > > > > > > composition
> > > > > > > begins.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Also, let's all start using TOSCA terminology here:
> > > > > > > > > > > > > the
> > > > > > > > containing
> > > > > > > > > > > > service
> > > > > > > > > > > > > is called the "top-level service," and the
> contained
> > > > > > > > > > > > > services
> > > > > > > are
> > > > > > > > > > > called
> > > > > > > > > > > > > "substituting services."
> > > > > > > > > > > > >
> > > > > > > > > > > > > Also, we keep using trivial examples, but it's
> > > > > > > > > > > > > definitely
> > > > > > > > possible
> > > > > > > > > > for
> > > > > > > > > > > a
> > > > > > > > > > > > > top-level service to require several
> > > > > > > > > > > > > substituting services at
> > > > > > > the
> > > > > > > > > > same
> > > > > > > > > > > > > time. I can definitely see such things
> > > > > > > > > > > > > happening in NFV with
> > > > > > > even
> > > > > > > > > > > simple
> > > > > > > > > > > > > service chains. Basically every VNF could be a
> > > > > > > > > > > > > substituting
> > > > > > > > > service.
> > > > > > > > > > > > >
> > > > > > > > > > > > > So, actually one of the validations would be
> > > > > > > > > > > > > to
> make
> > > > > > > > > > > > > sure you
> > > > > > > do
> > > > > > > > > not
> > > > > > > > > > > > create
> > > > > > > > > > > > > circular composition: if the top-level also
> > > > > > > > > > > > > has substitution
> > > > > > > > > > mappings,
> > > > > > > > > > > > you
> > > > > > > > > > > > > need to make sure that one of the substituting
> > > > > > > > > > > > > ones doesn't
> > > > > > > > require
> > > > > > > > > > it.
> > > > > > > > > > > > :)
> > > > > > > > > > > > > Not a very likely situation, but it would lead
> > > > > > > > > > > > > to
> > > > failure.
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Option 2:
> > > > > > > > > > > > > > While creating a service look at the
> > > > > > > > > > > > > > req-caps at
> > > > > > the
> > > > > > > > > > > > > > service-template level and create a service
> > > > > > > > > > > > > > including the
> > > > > > > nodes
> > > > > > > > > > > > provided
> > > > > > > > > > > > > by
> > > > > > > > > > > > > > the substituting service-template. With this
> > > > > > > > > > > > > > approach there
> > > > > > > > would
> > > > > > > > > > not
> > > > > > > > > > > > be
> > > > > > > > > > > > > > any service created from the service-template
> > which
> > > > is
> > > > > > > > > providing
> > > > > > > > > > > the
> > > > > > > > > > > > > > substitution functionality. The
> > > > > > > > > > > > > > service-template would
> > > > > > remain
> > > > > > > > the
> > > > > > > > > > > same
> > > > > > > > > > > > > but
> > > > > > > > > > > > > > only the service would be added with extra nodes.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Are you considering both option 1 & 2 for
> > > > > > > > > > > > > > the
> > > > > > implementation
> > > > > > > ?
> > > > > > > > If
> > > > > > > > > > not
> > > > > > > > > > > > > > which one do you feel which take priority. I
> > > > > > > > > > > > > > see option 2
> > > > > > at
> > > > > > > > this
> > > > > > > > > > > stage
> > > > > > > > > > > > > > could be the best possible approach Also
> > > > > > > > > > > > > > could
> you
> > > > > > > > > > > > > > please let me know a tentative time for this
> > > > > > > > > feature
> > > > > > > > > > > to
> > > > > > > > > > > > be
> > > > > > > > > > > > > > available?
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > I think both options 1 and 2 make sense and
> > > > > > > > > > > > > are useful, and
> > > > > > > > > actually
> > > > > > > > > > > one
> > > > > > > > > > > > is
> > > > > > > > > > > > > a subset of the other.
> > > > > > > > > > > > >
> > > > > > > > > > > > > With option #2 (substituting a service
> > > > > > > > > > > > > template),
> it
> > > > > > > > > > > > > means
> > > > > > new
> > > > > > > > > nodes
> > > > > > > > > > > are
> > > > > > > > > > > > > instantiated and the composed service would
> > > > > > > > > > > > > include all
> > > > > > nodes.
> > > > > > > > So,
> > > > > > > > > an
> > > > > > > > > > > > > "install" workflow would install everything at
> once.
> > > > > > > > > > > > > In this
> > > > > > > case
> > > > > > > > > we
> > > > > > > > > > do
> > > > > > > > > > > > > need to fix the lifecycle workflows to be
> > > > > > > > > > > > > "boundary
> > > > aware,"
> > > > > > so
> > > > > > > > that
> > > > > > > > > > > > > workflows of substituting service nodes are
> > > > > > > > > > > > > part of their own
> > > > > > > > task
> > > > > > > > > > > > graph. I
> > > > > > > > > > > > > think that possibly using a logical proxy node
> > > > > > > > > > > > > in between
> > > > > > might
> > > > > > > > > solve
> > > > > > > > > > > > this
> > > > > > > > > > > > > situation automatically.
> > > > > > > > > > > > >
> > > > > > > > > > > > > With option #1 (substituting a service) the
> > > > > > > > > > > > > substituting
> > > > > > nodes
> > > > > > > > > might
> > > > > > > > > > > > > already be installed. Or not (they might have
> > > > > > > > > > > > > been
> > > > > > instantiated
> > > > > > > > but
> > > > > > > > > > > still
> > > > > > > > > > > > > not installed). So, the lifecycle workflows
> > > > > > > > > > > > > should only work
> > > > > > on
> > > > > > > > the
> > > > > > > > > > > nodes
> > > > > > > > > > > > > that have not yet been installed.
> > > > > > > > > > > > >
> > > > > > > > > > > > > The point is that we just need to beef up the
> > > > > > > > > > > > > lifecycle
> > > > > > > workflows
> > > > > > > > > to
> > > > > > > > > > > > > properly work with boundaries and with a mix
> > > > > > > > > > > > > of nodes in
> > > > > > > > different
> > > > > > > > > > > > states.
> > > > > > > > > > > > > If they can do that, then they can handle any
> > > > > > > > > > > > > kind of service
> > > > > > > > > > > > composition,
> > > > > > > > > > > > > whether it's option #1 or option #2.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I don't think we can provide a timeline yet,
> > > > > > > > > > > > > but I will say
> > > > > > > that
> > > > > > > > we
> > > > > > > > > > are
> > > > > > > > > > > > in
> > > > > > > > > > > > > the research phase and may have a POC soon.
> > > > > > > > > > > > > Avia is in charge
> > > > > > > of
> > > > > > > > > this
> > > > > > > > > > > > JIRA,
> > > > > > > > > > > > > so I will let him chime in with the current
> > > > > > > > > > > > > state
> of
> > > > > things.
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Service Composition / Substitution Mapping
Posted by Tal Liron <ta...@cloudify.co>.
"directives" is just a list of strings. Moreover, it seems like a
placeholder for something in the future. Also note that "directives" exist
only for node templates, not node type, so this means that in order to use
substitution we would have to have a node template marked as being
abstract. If we want to use a directive with some sophistication, we would
have to invent some weird string-based language for this (like we did for
operation configuration) in order to configure exactly how substitution
would happen.
I really think that a policy is the correct way to handle this for now, and
possibly for the future. It is the most "TOSCA-in-spirit" solution.
On Wed, Sep 6, 2017 at 3:45 AM, Avia Efrat <av...@cloudify.co> wrote:
> On Tue, Sep 5, 2017 at 6:09 PM, Tal Liron <ta...@cloudify.co> wrote:
>
> >
> > Just like you can specify a specific node template for a requirement, I
> > believe it can be useful and often necessary to specify a substituted
> > service template.
> >
>
> Even if it is needed, the question is how to do that. Using a policy
> seems too much like a catch-all solution. In the spec, there is the notion
> of the "substitutable" directive [2.10.2]. At first I thought it was meant
> as a general concept, but then I noticed that:
>
> 1. node templates have a `directives` field, that is described as "an
> optional list of directive values to provide processing instructions to
> orchestrators and tooling" [3.7.3.1].
> 2. section [2.10.2] does not mention "substitutable directive", but rather
> "substitutable" directive. It seems that their intention is the adding the
> "substitutable" string to the `directives` field of the node template is
> the mean by TOSCA is designating a node to be substituted.
> 3. section [2.9] extensively deals with abstract nodes. It mentions another
> use of the `directives` field, this time the "selectable" directive (also
> in [3.7.3.1], [3.7.3.3]).
>
> Although section [3.3.3] states that there are no defined directives in
> TOSCA 1.0, the de facto usages of the directive field indicates at least
> that there is intent of using this field, and maybe define it more
> explicitly in future versions.
>
> Bottom line, currently I see the `directives` field as the the indicator of
> substitutable nodes (by specifying the "substitutable" directive). It is
> needed to enquire OASIS regarding their future plans of TOSCA.
>
Re: Service Composition / Substitution Mapping
Posted by Avia Efrat <av...@cloudify.co>.
On Tue, Sep 5, 2017 at 6:09 PM, Tal Liron <ta...@cloudify.co> wrote:
>
> Just like you can specify a specific node template for a requirement, I
> believe it can be useful and often necessary to specify a substituted
> service template.
>
Even if it is needed, the question is how to do that. Using a policy
seems too much like a catch-all solution. In the spec, there is the notion
of the "substitutable" directive [2.10.2]. At first I thought it was meant
as a general concept, but then I noticed that:
1. node templates have a `directives` field, that is described as "an
optional list of directive values to provide processing instructions to
orchestrators and tooling" [3.7.3.1].
2. section [2.10.2] does not mention "substitutable directive", but rather
"substitutable" directive. It seems that their intention is the adding the
"substitutable" string to the `directives` field of the node template is
the mean by TOSCA is designating a node to be substituted.
3. section [2.9] extensively deals with abstract nodes. It mentions another
use of the `directives` field, this time the "selectable" directive (also
in [3.7.3.1], [3.7.3.3]).
Although section [3.3.3] states that there are no defined directives in
TOSCA 1.0, the de facto usages of the directive field indicates at least
that there is intent of using this field, and maybe define it more
explicitly in future versions.
Bottom line, currently I see the `directives` field as the the indicator of
substitutable nodes (by specifying the "substitutable" directive). It is
needed to enquire OASIS regarding their future plans of TOSCA.
Re: Service Composition / Substitution Mapping
Posted by Tal Liron <ta...@cloudify.co>.
On Sun, Sep 3, 2017 at 4:06 AM, Ran Ziv <ra...@cloudify.co> wrote:
> I too agree with the notion that picking a substituting service template
> explicitly is against the spirit of this feature, for the reasons Avia has
> explained.
> If a specific template is needed, why not import it directly, or require a
> more specific capability offered by the substituting service template?
>
Just like you can specify a specific node template for a requirement, I
believe it can be useful and often necessary to specify a substituted
service template.
Let me try an example:
topology_template:
node_templates:
my_app:
type: WebApp
requirements:
host:
capability: tosca.capabilities.Container
node: my_host # explicit node template
db:
capability: MongoDb
my_host:
type: VirtualMachine
policies:
substitution:
type: aria.Substitution
targets: [ my_app ]
properties:
requirements: # map of strings to requirement policy infos
db:
service_template: /opt/templates/MongoDbCluster.csar # explicit
service template