You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ariatosca.apache.org by Srinidhi Srivatsan <sr...@globallogic.com.INVALID> on 2017/12/06 06:10:39 UTC

install_aria_extensions called twice

Hi,



   We have observed that install_aria_extensions is called twice – once in
process.py(executor) and once in main.py. Could you please confirm as to
why it is called again in process.py?



Regards,

Srinidhi.

Re: install_aria_extensions called twice

Posted by Maxim Orlov <ma...@cloudify.co>.
Ok, so basically the issue is we pickle objects passed to an operation,
when they are unpickled. effectively we call the "install_aria_extensions"
(and later we call it again). What i meant by "flattening the data types"
is turning any "get_input/get_attribute/get_property" to the actual value,
as opposed to a model based reference to that value (before passing it to
an operation).

On Wed, Jan 10, 2018 at 1:58 PM Vaishali Krishnamurthy
<v....@globallogic.com.invalid> wrote:

> Hi Maxim,
>
> For this https://issues.apache.org/jira/browse/ARIA-432  issue,  I need
> some
> more clarification on the part where it is mentioned "flatten those
> non-primitive types " . Could you please explain?
>
> Regards,
> Vaishali
>
>
> -----Original Message-----
> From: Srinidhi Srivatsan [mailto:
> srinidhi.srivatsan@globallogic.com.INVALID]
> Sent: Friday, December 08, 2017 11:32 AM
> To: dev@ariatosca.incubator.apache.org
> Subject: RE: install_aria_extensions called twice
>
> Hi,
>
> Thanks for the information. Will this item be prioritized and fixed?
>
> Regards,
> Srinidhi.
>
> -----Original Message-----
> From: Maxim Orlov [mailto:maxim@cloudify.co]
> Sent: Wednesday, December 06, 2017 5:59 PM
> To: dev@ariatosca.incubator.apache.org
> Subject: Re: install_aria_extensions called twice
>
> The removal on the call to install_aria_extensions does solve the problem
> in
> your case, however this might raise other issue. For example: writing a
> process executor extension would have no effect if you'd remove the
> `install_aria_extensions` function call.
>
> The actual problem is caused because you try to pass a complex data_type,
> which get pickled. in the unpicking process the extension gets loaded
> automatically, and then it get loaded again via the
> `install_aria_extensions`. Indeed this is a bug we need to address (I've
> opened a JIRA ticket here <https://issues.apache.org/jira/browse/ARIA-432
> >).
>
> On Wed, Dec 6, 2017 at 8:59 AM Srinidhi Srivatsan
> <sr...@globallogic.com.invalid> wrote:
>
> > Hi,
> >
> >
> >
> > We have observed runtime error “Re-definition of YAML 1.1” error while
> > executing the following service template. This happens when we define
> > a complex type as input into an operation but the first entry must be
> > an intrinsic function even if the type has been defined as string. We
> > think it maybe because of the install_aria_extensions being called
> > twice. When commenting out the “install_aria_extensions” in
> > process.py(executor), execution succeeds.
> >
> >
> >
> > tosca_definitions_version: tosca_simple_yaml_1_0
> >
> >
> >
> > data_types:
> >
> >   my.datatypes.operation.inputs.Template:
> >
> >     derived_from: tosca.datatypes.Root
> >
> >     properties:
> >
> >       path:
> >
> >         type: string
> >
> >       output:
> >
> >         type: my.datatypes.operation.inputs.Template.Output
> >
> >         required: false
> >
> >
> >
> >   my.datatypes.operation.inputs.Template.Output:
> >
> >     derived_from: tosca.datatypes.Root
> >
> >     properties:
> >
> >       runtime_property:
> >
> >         type: string
> >
> >         required: false
> >
> >       remote_file:
> >
> >         type: my.datatypes.operation.inputs.Template.Output.RemoteFile
> >
> >         required: false
> >
> >
> >
> >   my.datatypes.operation.inputs.Template.Output.RemoteFile:
> >
> >     derived_from: tosca.datatypes.Root
> >
> >     properties:
> >
> >       hostname:
> >
> >         type: string
> >
> >         required: false
> >
> >       username:
> >
> >         type: string
> >
> >         required: false
> >
> >       password:
> >
> >         type: string
> >
> >         required: false
> >
> >       path:
> >
> >         type: string
> >
> >         required: false
> >
> >
> >
> > node_types:
> >
> >   my.nodes.BGW:
> >
> >     derived_from: tosca.nodes.Root
> >
> >     properties:
> >
> >       host:
> >
> >         type: string
> >
> >         description: bgw host address
> >
> >     interfaces:
> >
> >       Standard:
> >
> >         type: tosca.interfaces.node.lifecycle.Standard
> >
> >         configure:
> >
> >           implementation: scripts/test.py
> >
> >           inputs:
> >
> >             templates:
> >
> >               type: list
> >
> >               entry_schema:
> >
> >                 type: my.datatypes.operation.inputs.Template
> >
> >                 description: "BGW configuration templates"
> >
> >
> >
> > topology_template:
> >
> >
> >
> >     node_templates:
> >
> >       bgw:
> >
> >         type: my.nodes.BGW
> >
> >         properties:
> >
> >           host: "{get_input: BGW_HOST }"
> >
> >         interfaces:
> >
> >           Standard:
> >
> >             configure:
> >
> >               inputs:
> >
> >                 templates:
> >
> >                 - path: "configuration-templates/bgw_epg_add"
> >
> >                   output:
> >
> >                       runtime_property: "bgw_epg_add_configuration"
> >
> >                       remote_file:
> >
> >                           hostname: "host" # replace the string with a
> > function and you no longer get the error i.e. { get_property: [SELF,
> > host] }
> >
> >                           username: "Name"
> >
> >                           password: "Pwd"
> >
> >                           path:  "the path "
> >
> >                 - path: "configuration-templates/bgw_mme_add"
> >
> >
> >
> > Regards,
> >
> > Srinidhi.
> >
> >
> >
> > *From:* Srinidhi Srivatsan [mailto:srinidhi.srivatsan@globallogic.com]
> > *Sent:* Wednesday, December 06, 2017 11:41 AM
> > *To:* 'dev@ariatosca.incubator.apache.org'
> > *Subject:* install_aria_extensions called twice
> >
> >
> >
> > Hi,
> >
> >
> >
> >    We have observed that install_aria_extensions is called twice –
> > once in
> > process.py(executor) and once in main.py. Could you please confirm as
> > to why it is called again in process.py?
> >
> >
> >
> > Regards,
> >
> > Srinidhi.
> >
>

RE: install_aria_extensions called twice

Posted by Vaishali Krishnamurthy <v....@globallogic.com.INVALID>.
Hi Maxim,

For this https://issues.apache.org/jira/browse/ARIA-432  issue,  I need some
more clarification on the part where it is mentioned "flatten those
non-primitive types " . Could you please explain?

Regards,
Vaishali


-----Original Message-----
From: Srinidhi Srivatsan [mailto:srinidhi.srivatsan@globallogic.com.INVALID]
Sent: Friday, December 08, 2017 11:32 AM
To: dev@ariatosca.incubator.apache.org
Subject: RE: install_aria_extensions called twice

Hi,

Thanks for the information. Will this item be prioritized and fixed?

Regards,
Srinidhi.

-----Original Message-----
From: Maxim Orlov [mailto:maxim@cloudify.co]
Sent: Wednesday, December 06, 2017 5:59 PM
To: dev@ariatosca.incubator.apache.org
Subject: Re: install_aria_extensions called twice

The removal on the call to install_aria_extensions does solve the problem in
your case, however this might raise other issue. For example: writing a
process executor extension would have no effect if you'd remove the
`install_aria_extensions` function call.

The actual problem is caused because you try to pass a complex data_type,
which get pickled. in the unpicking process the extension gets loaded
automatically, and then it get loaded again via the
`install_aria_extensions`. Indeed this is a bug we need to address (I've
opened a JIRA ticket here <https://issues.apache.org/jira/browse/ARIA-432>).

On Wed, Dec 6, 2017 at 8:59 AM Srinidhi Srivatsan
<sr...@globallogic.com.invalid> wrote:

> Hi,
>
>
>
> We have observed runtime error “Re-definition of YAML 1.1” error while
> executing the following service template. This happens when we define
> a complex type as input into an operation but the first entry must be
> an intrinsic function even if the type has been defined as string. We
> think it maybe because of the install_aria_extensions being called
> twice. When commenting out the “install_aria_extensions” in
> process.py(executor), execution succeeds.
>
>
>
> tosca_definitions_version: tosca_simple_yaml_1_0
>
>
>
> data_types:
>
>   my.datatypes.operation.inputs.Template:
>
>     derived_from: tosca.datatypes.Root
>
>     properties:
>
>       path:
>
>         type: string
>
>       output:
>
>         type: my.datatypes.operation.inputs.Template.Output
>
>         required: false
>
>
>
>   my.datatypes.operation.inputs.Template.Output:
>
>     derived_from: tosca.datatypes.Root
>
>     properties:
>
>       runtime_property:
>
>         type: string
>
>         required: false
>
>       remote_file:
>
>         type: my.datatypes.operation.inputs.Template.Output.RemoteFile
>
>         required: false
>
>
>
>   my.datatypes.operation.inputs.Template.Output.RemoteFile:
>
>     derived_from: tosca.datatypes.Root
>
>     properties:
>
>       hostname:
>
>         type: string
>
>         required: false
>
>       username:
>
>         type: string
>
>         required: false
>
>       password:
>
>         type: string
>
>         required: false
>
>       path:
>
>         type: string
>
>         required: false
>
>
>
> node_types:
>
>   my.nodes.BGW:
>
>     derived_from: tosca.nodes.Root
>
>     properties:
>
>       host:
>
>         type: string
>
>         description: bgw host address
>
>     interfaces:
>
>       Standard:
>
>         type: tosca.interfaces.node.lifecycle.Standard
>
>         configure:
>
>           implementation: scripts/test.py
>
>           inputs:
>
>             templates:
>
>               type: list
>
>               entry_schema:
>
>                 type: my.datatypes.operation.inputs.Template
>
>                 description: "BGW configuration templates"
>
>
>
> topology_template:
>
>
>
>     node_templates:
>
>       bgw:
>
>         type: my.nodes.BGW
>
>         properties:
>
>           host: "{get_input: BGW_HOST }"
>
>         interfaces:
>
>           Standard:
>
>             configure:
>
>               inputs:
>
>                 templates:
>
>                 - path: "configuration-templates/bgw_epg_add"
>
>                   output:
>
>                       runtime_property: "bgw_epg_add_configuration"
>
>                       remote_file:
>
>                           hostname: "host" # replace the string with a
> function and you no longer get the error i.e. { get_property: [SELF,
> host] }
>
>                           username: "Name"
>
>                           password: "Pwd"
>
>                           path:  "the path "
>
>                 - path: "configuration-templates/bgw_mme_add"
>
>
>
> Regards,
>
> Srinidhi.
>
>
>
> *From:* Srinidhi Srivatsan [mailto:srinidhi.srivatsan@globallogic.com]
> *Sent:* Wednesday, December 06, 2017 11:41 AM
> *To:* 'dev@ariatosca.incubator.apache.org'
> *Subject:* install_aria_extensions called twice
>
>
>
> Hi,
>
>
>
>    We have observed that install_aria_extensions is called twice –
> once in
> process.py(executor) and once in main.py. Could you please confirm as
> to why it is called again in process.py?
>
>
>
> Regards,
>
> Srinidhi.
>

RE: install_aria_extensions called twice

Posted by Srinidhi Srivatsan <sr...@globallogic.com.INVALID>.
Hi,

Thanks for the information. Will this item be prioritized and fixed?

Regards,
Srinidhi.

-----Original Message-----
From: Maxim Orlov [mailto:maxim@cloudify.co]
Sent: Wednesday, December 06, 2017 5:59 PM
To: dev@ariatosca.incubator.apache.org
Subject: Re: install_aria_extensions called twice

The removal on the call to install_aria_extensions does solve the problem in
your case, however this might raise other issue. For example: writing a
process executor extension would have no effect if you'd remove the
`install_aria_extensions` function call.

The actual problem is caused because you try to pass a complex data_type,
which get pickled. in the unpicking process the extension gets loaded
automatically, and then it get loaded again via the
`install_aria_extensions`. Indeed this is a bug we need to address (I've
opened a JIRA ticket here <https://issues.apache.org/jira/browse/ARIA-432>).

On Wed, Dec 6, 2017 at 8:59 AM Srinidhi Srivatsan
<sr...@globallogic.com.invalid> wrote:

> Hi,
>
>
>
> We have observed runtime error “Re-definition of YAML 1.1” error while
> executing the following service template. This happens when we define
> a complex type as input into an operation but the first entry must be
> an intrinsic function even if the type has been defined as string. We
> think it maybe because of the install_aria_extensions being called
> twice. When commenting out the “install_aria_extensions” in
> process.py(executor), execution succeeds.
>
>
>
> tosca_definitions_version: tosca_simple_yaml_1_0
>
>
>
> data_types:
>
>   my.datatypes.operation.inputs.Template:
>
>     derived_from: tosca.datatypes.Root
>
>     properties:
>
>       path:
>
>         type: string
>
>       output:
>
>         type: my.datatypes.operation.inputs.Template.Output
>
>         required: false
>
>
>
>   my.datatypes.operation.inputs.Template.Output:
>
>     derived_from: tosca.datatypes.Root
>
>     properties:
>
>       runtime_property:
>
>         type: string
>
>         required: false
>
>       remote_file:
>
>         type: my.datatypes.operation.inputs.Template.Output.RemoteFile
>
>         required: false
>
>
>
>   my.datatypes.operation.inputs.Template.Output.RemoteFile:
>
>     derived_from: tosca.datatypes.Root
>
>     properties:
>
>       hostname:
>
>         type: string
>
>         required: false
>
>       username:
>
>         type: string
>
>         required: false
>
>       password:
>
>         type: string
>
>         required: false
>
>       path:
>
>         type: string
>
>         required: false
>
>
>
> node_types:
>
>   my.nodes.BGW:
>
>     derived_from: tosca.nodes.Root
>
>     properties:
>
>       host:
>
>         type: string
>
>         description: bgw host address
>
>     interfaces:
>
>       Standard:
>
>         type: tosca.interfaces.node.lifecycle.Standard
>
>         configure:
>
>           implementation: scripts/test.py
>
>           inputs:
>
>             templates:
>
>               type: list
>
>               entry_schema:
>
>                 type: my.datatypes.operation.inputs.Template
>
>                 description: "BGW configuration templates"
>
>
>
> topology_template:
>
>
>
>     node_templates:
>
>       bgw:
>
>         type: my.nodes.BGW
>
>         properties:
>
>           host: "{get_input: BGW_HOST }"
>
>         interfaces:
>
>           Standard:
>
>             configure:
>
>               inputs:
>
>                 templates:
>
>                 - path: "configuration-templates/bgw_epg_add"
>
>                   output:
>
>                       runtime_property: "bgw_epg_add_configuration"
>
>                       remote_file:
>
>                           hostname: "host" # replace the string with a
> function and you no longer get the error i.e. { get_property: [SELF,
> host] }
>
>                           username: "Name"
>
>                           password: "Pwd"
>
>                           path:  "the path "
>
>                 - path: "configuration-templates/bgw_mme_add"
>
>
>
> Regards,
>
> Srinidhi.
>
>
>
> *From:* Srinidhi Srivatsan [mailto:srinidhi.srivatsan@globallogic.com]
> *Sent:* Wednesday, December 06, 2017 11:41 AM
> *To:* 'dev@ariatosca.incubator.apache.org'
> *Subject:* install_aria_extensions called twice
>
>
>
> Hi,
>
>
>
>    We have observed that install_aria_extensions is called twice –
> once in
> process.py(executor) and once in main.py. Could you please confirm as
> to why it is called again in process.py?
>
>
>
> Regards,
>
> Srinidhi.
>

Re: install_aria_extensions called twice

Posted by Maxim Orlov <ma...@cloudify.co>.
The removal on the call to install_aria_extensions does solve the problem
in your case, however this might raise other issue. For example: writing a
process executor extension would have no effect if you'd remove the
`install_aria_extensions` function call.

The actual problem is caused because you try to pass a complex data_type,
which get pickled. in the unpicking process the extension gets loaded
automatically, and then it get loaded again via the
`install_aria_extensions`. Indeed this is a bug we need to address (I've
opened a JIRA ticket here <https://issues.apache.org/jira/browse/ARIA-432>).

On Wed, Dec 6, 2017 at 8:59 AM Srinidhi Srivatsan
<sr...@globallogic.com.invalid> wrote:

> Hi,
>
>
>
> We have observed runtime error “Re-definition of YAML 1.1” error while
> executing the following service template. This happens when we define a
> complex type as input into an operation but the first entry must be an
> intrinsic function even if the type has been defined as string. We think it
> maybe because of the install_aria_extensions being called twice. When
> commenting out the “install_aria_extensions” in process.py(executor),
> execution succeeds.
>
>
>
> tosca_definitions_version: tosca_simple_yaml_1_0
>
>
>
> data_types:
>
>   my.datatypes.operation.inputs.Template:
>
>     derived_from: tosca.datatypes.Root
>
>     properties:
>
>       path:
>
>         type: string
>
>       output:
>
>         type: my.datatypes.operation.inputs.Template.Output
>
>         required: false
>
>
>
>   my.datatypes.operation.inputs.Template.Output:
>
>     derived_from: tosca.datatypes.Root
>
>     properties:
>
>       runtime_property:
>
>         type: string
>
>         required: false
>
>       remote_file:
>
>         type: my.datatypes.operation.inputs.Template.Output.RemoteFile
>
>         required: false
>
>
>
>   my.datatypes.operation.inputs.Template.Output.RemoteFile:
>
>     derived_from: tosca.datatypes.Root
>
>     properties:
>
>       hostname:
>
>         type: string
>
>         required: false
>
>       username:
>
>         type: string
>
>         required: false
>
>       password:
>
>         type: string
>
>         required: false
>
>       path:
>
>         type: string
>
>         required: false
>
>
>
> node_types:
>
>   my.nodes.BGW:
>
>     derived_from: tosca.nodes.Root
>
>     properties:
>
>       host:
>
>         type: string
>
>         description: bgw host address
>
>     interfaces:
>
>       Standard:
>
>         type: tosca.interfaces.node.lifecycle.Standard
>
>         configure:
>
>           implementation: scripts/test.py
>
>           inputs:
>
>             templates:
>
>               type: list
>
>               entry_schema:
>
>                 type: my.datatypes.operation.inputs.Template
>
>                 description: "BGW configuration templates"
>
>
>
> topology_template:
>
>
>
>     node_templates:
>
>       bgw:
>
>         type: my.nodes.BGW
>
>         properties:
>
>           host: "{get_input: BGW_HOST }"
>
>         interfaces:
>
>           Standard:
>
>             configure:
>
>               inputs:
>
>                 templates:
>
>                 - path: "configuration-templates/bgw_epg_add"
>
>                   output:
>
>                       runtime_property: "bgw_epg_add_configuration"
>
>                       remote_file:
>
>                           hostname: "host" # replace the string with a
> function and you no longer get the error i.e. { get_property: [SELF, host]
> }
>
>                           username: "Name"
>
>                           password: "Pwd"
>
>                           path:  "the path "
>
>                 - path: "configuration-templates/bgw_mme_add"
>
>
>
> Regards,
>
> Srinidhi.
>
>
>
> *From:* Srinidhi Srivatsan [mailto:srinidhi.srivatsan@globallogic.com]
> *Sent:* Wednesday, December 06, 2017 11:41 AM
> *To:* 'dev@ariatosca.incubator.apache.org'
> *Subject:* install_aria_extensions called twice
>
>
>
> Hi,
>
>
>
>    We have observed that install_aria_extensions is called twice – once in
> process.py(executor) and once in main.py. Could you please confirm as to
> why it is called again in process.py?
>
>
>
> Regards,
>
> Srinidhi.
>

RE: install_aria_extensions called twice

Posted by Srinidhi Srivatsan <sr...@globallogic.com.INVALID>.
Hi,



We have observed runtime error “Re-definition of YAML 1.1” error while
executing the following service template. This happens when we define a
complex type as input into an operation but the first entry must be an
intrinsic function even if the type has been defined as string. We think it
maybe because of the install_aria_extensions being called twice. When
commenting out the “install_aria_extensions” in process.py(executor),
execution succeeds.



tosca_definitions_version: tosca_simple_yaml_1_0



data_types:

  my.datatypes.operation.inputs.Template:

    derived_from: tosca.datatypes.Root

    properties:

      path:

        type: string

      output:

        type: my.datatypes.operation.inputs.Template.Output

        required: false



  my.datatypes.operation.inputs.Template.Output:

    derived_from: tosca.datatypes.Root

    properties:

      runtime_property:

        type: string

        required: false

      remote_file:

        type: my.datatypes.operation.inputs.Template.Output.RemoteFile

        required: false



  my.datatypes.operation.inputs.Template.Output.RemoteFile:

    derived_from: tosca.datatypes.Root

    properties:

      hostname:

        type: string

        required: false

      username:

        type: string

        required: false

      password:

        type: string

        required: false

      path:

        type: string

        required: false



node_types:

  my.nodes.BGW:

    derived_from: tosca.nodes.Root

    properties:

      host:

        type: string

        description: bgw host address

    interfaces:

      Standard:

        type: tosca.interfaces.node.lifecycle.Standard

        configure:

          implementation: scripts/test.py

          inputs:

            templates:

              type: list

              entry_schema:

                type: my.datatypes.operation.inputs.Template

                description: "BGW configuration templates"



topology_template:



    node_templates:

      bgw:

        type: my.nodes.BGW

        properties:

          host: "{get_input: BGW_HOST }"

        interfaces:

          Standard:

            configure:

              inputs:

                templates:

                - path: "configuration-templates/bgw_epg_add"

                  output:

                      runtime_property: "bgw_epg_add_configuration"

                      remote_file:

                          hostname: "host" # replace the string with a
function and you no longer get the error i.e. { get_property: [SELF, host] }

                          username: "Name"

                          password: "Pwd"

                          path:  "the path "

                - path: "configuration-templates/bgw_mme_add"



Regards,

Srinidhi.



*From:* Srinidhi Srivatsan [mailto:srinidhi.srivatsan@globallogic.com]
*Sent:* Wednesday, December 06, 2017 11:41 AM
*To:* 'dev@ariatosca.incubator.apache.org'
*Subject:* install_aria_extensions called twice



Hi,



   We have observed that install_aria_extensions is called twice – once in
process.py(executor) and once in main.py. Could you please confirm as to
why it is called again in process.py?



Regards,

Srinidhi.