You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ariatosca.apache.org by Srinidhi Srivatsan <sr...@globallogic.com.INVALID> on 2017/12/06 06:10:39 UTC
install_aria_extensions called twice
Hi,
We have observed that install_aria_extensions is called twice – once in
process.py(executor) and once in main.py. Could you please confirm as to
why it is called again in process.py?
Regards,
Srinidhi.
Re: install_aria_extensions called twice
Posted by Maxim Orlov <ma...@cloudify.co>.
Ok, so basically the issue is we pickle objects passed to an operation,
when they are unpickled. effectively we call the "install_aria_extensions"
(and later we call it again). What i meant by "flattening the data types"
is turning any "get_input/get_attribute/get_property" to the actual value,
as opposed to a model based reference to that value (before passing it to
an operation).
On Wed, Jan 10, 2018 at 1:58 PM Vaishali Krishnamurthy
<v....@globallogic.com.invalid> wrote:
> Hi Maxim,
>
> For this https://issues.apache.org/jira/browse/ARIA-432 issue, I need
> some
> more clarification on the part where it is mentioned "flatten those
> non-primitive types " . Could you please explain?
>
> Regards,
> Vaishali
>
>
> -----Original Message-----
> From: Srinidhi Srivatsan [mailto:
> srinidhi.srivatsan@globallogic.com.INVALID]
> Sent: Friday, December 08, 2017 11:32 AM
> To: dev@ariatosca.incubator.apache.org
> Subject: RE: install_aria_extensions called twice
>
> Hi,
>
> Thanks for the information. Will this item be prioritized and fixed?
>
> Regards,
> Srinidhi.
>
> -----Original Message-----
> From: Maxim Orlov [mailto:maxim@cloudify.co]
> Sent: Wednesday, December 06, 2017 5:59 PM
> To: dev@ariatosca.incubator.apache.org
> Subject: Re: install_aria_extensions called twice
>
> The removal on the call to install_aria_extensions does solve the problem
> in
> your case, however this might raise other issue. For example: writing a
> process executor extension would have no effect if you'd remove the
> `install_aria_extensions` function call.
>
> The actual problem is caused because you try to pass a complex data_type,
> which get pickled. in the unpicking process the extension gets loaded
> automatically, and then it get loaded again via the
> `install_aria_extensions`. Indeed this is a bug we need to address (I've
> opened a JIRA ticket here <https://issues.apache.org/jira/browse/ARIA-432
> >).
>
> On Wed, Dec 6, 2017 at 8:59 AM Srinidhi Srivatsan
> <sr...@globallogic.com.invalid> wrote:
>
> > Hi,
> >
> >
> >
> > We have observed runtime error “Re-definition of YAML 1.1” error while
> > executing the following service template. This happens when we define
> > a complex type as input into an operation but the first entry must be
> > an intrinsic function even if the type has been defined as string. We
> > think it maybe because of the install_aria_extensions being called
> > twice. When commenting out the “install_aria_extensions” in
> > process.py(executor), execution succeeds.
> >
> >
> >
> > tosca_definitions_version: tosca_simple_yaml_1_0
> >
> >
> >
> > data_types:
> >
> > my.datatypes.operation.inputs.Template:
> >
> > derived_from: tosca.datatypes.Root
> >
> > properties:
> >
> > path:
> >
> > type: string
> >
> > output:
> >
> > type: my.datatypes.operation.inputs.Template.Output
> >
> > required: false
> >
> >
> >
> > my.datatypes.operation.inputs.Template.Output:
> >
> > derived_from: tosca.datatypes.Root
> >
> > properties:
> >
> > runtime_property:
> >
> > type: string
> >
> > required: false
> >
> > remote_file:
> >
> > type: my.datatypes.operation.inputs.Template.Output.RemoteFile
> >
> > required: false
> >
> >
> >
> > my.datatypes.operation.inputs.Template.Output.RemoteFile:
> >
> > derived_from: tosca.datatypes.Root
> >
> > properties:
> >
> > hostname:
> >
> > type: string
> >
> > required: false
> >
> > username:
> >
> > type: string
> >
> > required: false
> >
> > password:
> >
> > type: string
> >
> > required: false
> >
> > path:
> >
> > type: string
> >
> > required: false
> >
> >
> >
> > node_types:
> >
> > my.nodes.BGW:
> >
> > derived_from: tosca.nodes.Root
> >
> > properties:
> >
> > host:
> >
> > type: string
> >
> > description: bgw host address
> >
> > interfaces:
> >
> > Standard:
> >
> > type: tosca.interfaces.node.lifecycle.Standard
> >
> > configure:
> >
> > implementation: scripts/test.py
> >
> > inputs:
> >
> > templates:
> >
> > type: list
> >
> > entry_schema:
> >
> > type: my.datatypes.operation.inputs.Template
> >
> > description: "BGW configuration templates"
> >
> >
> >
> > topology_template:
> >
> >
> >
> > node_templates:
> >
> > bgw:
> >
> > type: my.nodes.BGW
> >
> > properties:
> >
> > host: "{get_input: BGW_HOST }"
> >
> > interfaces:
> >
> > Standard:
> >
> > configure:
> >
> > inputs:
> >
> > templates:
> >
> > - path: "configuration-templates/bgw_epg_add"
> >
> > output:
> >
> > runtime_property: "bgw_epg_add_configuration"
> >
> > remote_file:
> >
> > hostname: "host" # replace the string with a
> > function and you no longer get the error i.e. { get_property: [SELF,
> > host] }
> >
> > username: "Name"
> >
> > password: "Pwd"
> >
> > path: "the path "
> >
> > - path: "configuration-templates/bgw_mme_add"
> >
> >
> >
> > Regards,
> >
> > Srinidhi.
> >
> >
> >
> > *From:* Srinidhi Srivatsan [mailto:srinidhi.srivatsan@globallogic.com]
> > *Sent:* Wednesday, December 06, 2017 11:41 AM
> > *To:* 'dev@ariatosca.incubator.apache.org'
> > *Subject:* install_aria_extensions called twice
> >
> >
> >
> > Hi,
> >
> >
> >
> > We have observed that install_aria_extensions is called twice –
> > once in
> > process.py(executor) and once in main.py. Could you please confirm as
> > to why it is called again in process.py?
> >
> >
> >
> > Regards,
> >
> > Srinidhi.
> >
>
RE: install_aria_extensions called twice
Posted by Vaishali Krishnamurthy <v....@globallogic.com.INVALID>.
Hi Maxim,
For this https://issues.apache.org/jira/browse/ARIA-432 issue, I need some
more clarification on the part where it is mentioned "flatten those
non-primitive types " . Could you please explain?
Regards,
Vaishali
-----Original Message-----
From: Srinidhi Srivatsan [mailto:srinidhi.srivatsan@globallogic.com.INVALID]
Sent: Friday, December 08, 2017 11:32 AM
To: dev@ariatosca.incubator.apache.org
Subject: RE: install_aria_extensions called twice
Hi,
Thanks for the information. Will this item be prioritized and fixed?
Regards,
Srinidhi.
-----Original Message-----
From: Maxim Orlov [mailto:maxim@cloudify.co]
Sent: Wednesday, December 06, 2017 5:59 PM
To: dev@ariatosca.incubator.apache.org
Subject: Re: install_aria_extensions called twice
The removal on the call to install_aria_extensions does solve the problem in
your case, however this might raise other issue. For example: writing a
process executor extension would have no effect if you'd remove the
`install_aria_extensions` function call.
The actual problem is caused because you try to pass a complex data_type,
which get pickled. in the unpicking process the extension gets loaded
automatically, and then it get loaded again via the
`install_aria_extensions`. Indeed this is a bug we need to address (I've
opened a JIRA ticket here <https://issues.apache.org/jira/browse/ARIA-432>).
On Wed, Dec 6, 2017 at 8:59 AM Srinidhi Srivatsan
<sr...@globallogic.com.invalid> wrote:
> Hi,
>
>
>
> We have observed runtime error “Re-definition of YAML 1.1” error while
> executing the following service template. This happens when we define
> a complex type as input into an operation but the first entry must be
> an intrinsic function even if the type has been defined as string. We
> think it maybe because of the install_aria_extensions being called
> twice. When commenting out the “install_aria_extensions” in
> process.py(executor), execution succeeds.
>
>
>
> tosca_definitions_version: tosca_simple_yaml_1_0
>
>
>
> data_types:
>
> my.datatypes.operation.inputs.Template:
>
> derived_from: tosca.datatypes.Root
>
> properties:
>
> path:
>
> type: string
>
> output:
>
> type: my.datatypes.operation.inputs.Template.Output
>
> required: false
>
>
>
> my.datatypes.operation.inputs.Template.Output:
>
> derived_from: tosca.datatypes.Root
>
> properties:
>
> runtime_property:
>
> type: string
>
> required: false
>
> remote_file:
>
> type: my.datatypes.operation.inputs.Template.Output.RemoteFile
>
> required: false
>
>
>
> my.datatypes.operation.inputs.Template.Output.RemoteFile:
>
> derived_from: tosca.datatypes.Root
>
> properties:
>
> hostname:
>
> type: string
>
> required: false
>
> username:
>
> type: string
>
> required: false
>
> password:
>
> type: string
>
> required: false
>
> path:
>
> type: string
>
> required: false
>
>
>
> node_types:
>
> my.nodes.BGW:
>
> derived_from: tosca.nodes.Root
>
> properties:
>
> host:
>
> type: string
>
> description: bgw host address
>
> interfaces:
>
> Standard:
>
> type: tosca.interfaces.node.lifecycle.Standard
>
> configure:
>
> implementation: scripts/test.py
>
> inputs:
>
> templates:
>
> type: list
>
> entry_schema:
>
> type: my.datatypes.operation.inputs.Template
>
> description: "BGW configuration templates"
>
>
>
> topology_template:
>
>
>
> node_templates:
>
> bgw:
>
> type: my.nodes.BGW
>
> properties:
>
> host: "{get_input: BGW_HOST }"
>
> interfaces:
>
> Standard:
>
> configure:
>
> inputs:
>
> templates:
>
> - path: "configuration-templates/bgw_epg_add"
>
> output:
>
> runtime_property: "bgw_epg_add_configuration"
>
> remote_file:
>
> hostname: "host" # replace the string with a
> function and you no longer get the error i.e. { get_property: [SELF,
> host] }
>
> username: "Name"
>
> password: "Pwd"
>
> path: "the path "
>
> - path: "configuration-templates/bgw_mme_add"
>
>
>
> Regards,
>
> Srinidhi.
>
>
>
> *From:* Srinidhi Srivatsan [mailto:srinidhi.srivatsan@globallogic.com]
> *Sent:* Wednesday, December 06, 2017 11:41 AM
> *To:* 'dev@ariatosca.incubator.apache.org'
> *Subject:* install_aria_extensions called twice
>
>
>
> Hi,
>
>
>
> We have observed that install_aria_extensions is called twice –
> once in
> process.py(executor) and once in main.py. Could you please confirm as
> to why it is called again in process.py?
>
>
>
> Regards,
>
> Srinidhi.
>
RE: install_aria_extensions called twice
Posted by Srinidhi Srivatsan <sr...@globallogic.com.INVALID>.
Hi,
Thanks for the information. Will this item be prioritized and fixed?
Regards,
Srinidhi.
-----Original Message-----
From: Maxim Orlov [mailto:maxim@cloudify.co]
Sent: Wednesday, December 06, 2017 5:59 PM
To: dev@ariatosca.incubator.apache.org
Subject: Re: install_aria_extensions called twice
The removal on the call to install_aria_extensions does solve the problem in
your case, however this might raise other issue. For example: writing a
process executor extension would have no effect if you'd remove the
`install_aria_extensions` function call.
The actual problem is caused because you try to pass a complex data_type,
which get pickled. in the unpicking process the extension gets loaded
automatically, and then it get loaded again via the
`install_aria_extensions`. Indeed this is a bug we need to address (I've
opened a JIRA ticket here <https://issues.apache.org/jira/browse/ARIA-432>).
On Wed, Dec 6, 2017 at 8:59 AM Srinidhi Srivatsan
<sr...@globallogic.com.invalid> wrote:
> Hi,
>
>
>
> We have observed runtime error “Re-definition of YAML 1.1” error while
> executing the following service template. This happens when we define
> a complex type as input into an operation but the first entry must be
> an intrinsic function even if the type has been defined as string. We
> think it maybe because of the install_aria_extensions being called
> twice. When commenting out the “install_aria_extensions” in
> process.py(executor), execution succeeds.
>
>
>
> tosca_definitions_version: tosca_simple_yaml_1_0
>
>
>
> data_types:
>
> my.datatypes.operation.inputs.Template:
>
> derived_from: tosca.datatypes.Root
>
> properties:
>
> path:
>
> type: string
>
> output:
>
> type: my.datatypes.operation.inputs.Template.Output
>
> required: false
>
>
>
> my.datatypes.operation.inputs.Template.Output:
>
> derived_from: tosca.datatypes.Root
>
> properties:
>
> runtime_property:
>
> type: string
>
> required: false
>
> remote_file:
>
> type: my.datatypes.operation.inputs.Template.Output.RemoteFile
>
> required: false
>
>
>
> my.datatypes.operation.inputs.Template.Output.RemoteFile:
>
> derived_from: tosca.datatypes.Root
>
> properties:
>
> hostname:
>
> type: string
>
> required: false
>
> username:
>
> type: string
>
> required: false
>
> password:
>
> type: string
>
> required: false
>
> path:
>
> type: string
>
> required: false
>
>
>
> node_types:
>
> my.nodes.BGW:
>
> derived_from: tosca.nodes.Root
>
> properties:
>
> host:
>
> type: string
>
> description: bgw host address
>
> interfaces:
>
> Standard:
>
> type: tosca.interfaces.node.lifecycle.Standard
>
> configure:
>
> implementation: scripts/test.py
>
> inputs:
>
> templates:
>
> type: list
>
> entry_schema:
>
> type: my.datatypes.operation.inputs.Template
>
> description: "BGW configuration templates"
>
>
>
> topology_template:
>
>
>
> node_templates:
>
> bgw:
>
> type: my.nodes.BGW
>
> properties:
>
> host: "{get_input: BGW_HOST }"
>
> interfaces:
>
> Standard:
>
> configure:
>
> inputs:
>
> templates:
>
> - path: "configuration-templates/bgw_epg_add"
>
> output:
>
> runtime_property: "bgw_epg_add_configuration"
>
> remote_file:
>
> hostname: "host" # replace the string with a
> function and you no longer get the error i.e. { get_property: [SELF,
> host] }
>
> username: "Name"
>
> password: "Pwd"
>
> path: "the path "
>
> - path: "configuration-templates/bgw_mme_add"
>
>
>
> Regards,
>
> Srinidhi.
>
>
>
> *From:* Srinidhi Srivatsan [mailto:srinidhi.srivatsan@globallogic.com]
> *Sent:* Wednesday, December 06, 2017 11:41 AM
> *To:* 'dev@ariatosca.incubator.apache.org'
> *Subject:* install_aria_extensions called twice
>
>
>
> Hi,
>
>
>
> We have observed that install_aria_extensions is called twice –
> once in
> process.py(executor) and once in main.py. Could you please confirm as
> to why it is called again in process.py?
>
>
>
> Regards,
>
> Srinidhi.
>
Re: install_aria_extensions called twice
Posted by Maxim Orlov <ma...@cloudify.co>.
The removal on the call to install_aria_extensions does solve the problem
in your case, however this might raise other issue. For example: writing a
process executor extension would have no effect if you'd remove the
`install_aria_extensions` function call.
The actual problem is caused because you try to pass a complex data_type,
which get pickled. in the unpicking process the extension gets loaded
automatically, and then it get loaded again via the
`install_aria_extensions`. Indeed this is a bug we need to address (I've
opened a JIRA ticket here <https://issues.apache.org/jira/browse/ARIA-432>).
On Wed, Dec 6, 2017 at 8:59 AM Srinidhi Srivatsan
<sr...@globallogic.com.invalid> wrote:
> Hi,
>
>
>
> We have observed runtime error “Re-definition of YAML 1.1” error while
> executing the following service template. This happens when we define a
> complex type as input into an operation but the first entry must be an
> intrinsic function even if the type has been defined as string. We think it
> maybe because of the install_aria_extensions being called twice. When
> commenting out the “install_aria_extensions” in process.py(executor),
> execution succeeds.
>
>
>
> tosca_definitions_version: tosca_simple_yaml_1_0
>
>
>
> data_types:
>
> my.datatypes.operation.inputs.Template:
>
> derived_from: tosca.datatypes.Root
>
> properties:
>
> path:
>
> type: string
>
> output:
>
> type: my.datatypes.operation.inputs.Template.Output
>
> required: false
>
>
>
> my.datatypes.operation.inputs.Template.Output:
>
> derived_from: tosca.datatypes.Root
>
> properties:
>
> runtime_property:
>
> type: string
>
> required: false
>
> remote_file:
>
> type: my.datatypes.operation.inputs.Template.Output.RemoteFile
>
> required: false
>
>
>
> my.datatypes.operation.inputs.Template.Output.RemoteFile:
>
> derived_from: tosca.datatypes.Root
>
> properties:
>
> hostname:
>
> type: string
>
> required: false
>
> username:
>
> type: string
>
> required: false
>
> password:
>
> type: string
>
> required: false
>
> path:
>
> type: string
>
> required: false
>
>
>
> node_types:
>
> my.nodes.BGW:
>
> derived_from: tosca.nodes.Root
>
> properties:
>
> host:
>
> type: string
>
> description: bgw host address
>
> interfaces:
>
> Standard:
>
> type: tosca.interfaces.node.lifecycle.Standard
>
> configure:
>
> implementation: scripts/test.py
>
> inputs:
>
> templates:
>
> type: list
>
> entry_schema:
>
> type: my.datatypes.operation.inputs.Template
>
> description: "BGW configuration templates"
>
>
>
> topology_template:
>
>
>
> node_templates:
>
> bgw:
>
> type: my.nodes.BGW
>
> properties:
>
> host: "{get_input: BGW_HOST }"
>
> interfaces:
>
> Standard:
>
> configure:
>
> inputs:
>
> templates:
>
> - path: "configuration-templates/bgw_epg_add"
>
> output:
>
> runtime_property: "bgw_epg_add_configuration"
>
> remote_file:
>
> hostname: "host" # replace the string with a
> function and you no longer get the error i.e. { get_property: [SELF, host]
> }
>
> username: "Name"
>
> password: "Pwd"
>
> path: "the path "
>
> - path: "configuration-templates/bgw_mme_add"
>
>
>
> Regards,
>
> Srinidhi.
>
>
>
> *From:* Srinidhi Srivatsan [mailto:srinidhi.srivatsan@globallogic.com]
> *Sent:* Wednesday, December 06, 2017 11:41 AM
> *To:* 'dev@ariatosca.incubator.apache.org'
> *Subject:* install_aria_extensions called twice
>
>
>
> Hi,
>
>
>
> We have observed that install_aria_extensions is called twice – once in
> process.py(executor) and once in main.py. Could you please confirm as to
> why it is called again in process.py?
>
>
>
> Regards,
>
> Srinidhi.
>
RE: install_aria_extensions called twice
Posted by Srinidhi Srivatsan <sr...@globallogic.com.INVALID>.
Hi,
We have observed runtime error “Re-definition of YAML 1.1” error while
executing the following service template. This happens when we define a
complex type as input into an operation but the first entry must be an
intrinsic function even if the type has been defined as string. We think it
maybe because of the install_aria_extensions being called twice. When
commenting out the “install_aria_extensions” in process.py(executor),
execution succeeds.
tosca_definitions_version: tosca_simple_yaml_1_0
data_types:
my.datatypes.operation.inputs.Template:
derived_from: tosca.datatypes.Root
properties:
path:
type: string
output:
type: my.datatypes.operation.inputs.Template.Output
required: false
my.datatypes.operation.inputs.Template.Output:
derived_from: tosca.datatypes.Root
properties:
runtime_property:
type: string
required: false
remote_file:
type: my.datatypes.operation.inputs.Template.Output.RemoteFile
required: false
my.datatypes.operation.inputs.Template.Output.RemoteFile:
derived_from: tosca.datatypes.Root
properties:
hostname:
type: string
required: false
username:
type: string
required: false
password:
type: string
required: false
path:
type: string
required: false
node_types:
my.nodes.BGW:
derived_from: tosca.nodes.Root
properties:
host:
type: string
description: bgw host address
interfaces:
Standard:
type: tosca.interfaces.node.lifecycle.Standard
configure:
implementation: scripts/test.py
inputs:
templates:
type: list
entry_schema:
type: my.datatypes.operation.inputs.Template
description: "BGW configuration templates"
topology_template:
node_templates:
bgw:
type: my.nodes.BGW
properties:
host: "{get_input: BGW_HOST }"
interfaces:
Standard:
configure:
inputs:
templates:
- path: "configuration-templates/bgw_epg_add"
output:
runtime_property: "bgw_epg_add_configuration"
remote_file:
hostname: "host" # replace the string with a
function and you no longer get the error i.e. { get_property: [SELF, host] }
username: "Name"
password: "Pwd"
path: "the path "
- path: "configuration-templates/bgw_mme_add"
Regards,
Srinidhi.
*From:* Srinidhi Srivatsan [mailto:srinidhi.srivatsan@globallogic.com]
*Sent:* Wednesday, December 06, 2017 11:41 AM
*To:* 'dev@ariatosca.incubator.apache.org'
*Subject:* install_aria_extensions called twice
Hi,
We have observed that install_aria_extensions is called twice – once in
process.py(executor) and once in main.py. Could you please confirm as to
why it is called again in process.py?
Regards,
Srinidhi.