You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ariatosca.apache.org by mxmrlv <gi...@git.apache.org> on 2017/06/01 14:32:56 UTC
[GitHub] incubator-ariatosca pull request #141: Aria 262 inconsistent node attributes...
GitHub user mxmrlv opened a pull request:
https://github.com/apache/incubator-ariatosca/pull/141
Aria 262 inconsistent node attributes behavior
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/incubator-ariatosca ARIA-262-Inconsistent-node-attributes-behavior
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-ariatosca/pull/141.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #141
----
commit dffcb9883a4c045430b853581a14f1a4d370e882
Author: max-orlov <ma...@gigaspaces.com>
Date: 2017-05-31T18:07:49Z
wip
commit e67a58fda607b1934ddc3088ffbe7b01038d10aa
Author: max-orlov <ma...@gigaspaces.com>
Date: 2017-06-01T08:25:49Z
wip2
commit bd7a43898cf686c29b13abb2e8b06c730dbe5ddb
Author: max-orlov <ma...@gigaspaces.com>
Date: 2017-06-01T13:34:52Z
moved instrumentation into mapi level
commit 77866ab1e770c03504b8b5c2f6f8836d7a991e42
Author: max-orlov <ma...@gigaspaces.com>
Date: 2017-06-01T14:30:39Z
tiny fix for several instrumenetation fields
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #141: ARIA-262 inconsistent node attributes...
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/141#discussion_r120013636
--- Diff: aria/storage/api.py ---
@@ -45,6 +45,7 @@ def __init__(self, model_cls, name=None, **kwargs):
super(ModelAPI, self).__init__(**kwargs)
self._model_cls = model_cls
self._name = name or model_cls.__modelname__
+ self._instrumentation = []
--- End diff --
threadlocal
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #141: ARIA-262 inconsistent node attributes...
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/141#discussion_r120013831
--- Diff: aria/storage/collection_instrumentation.py ---
@@ -204,39 +200,106 @@ def _raw(self):
class _InstrumentedModel(object):
- def __init__(self, field_name, original_model, model_storage):
+ def __init__(self, original_model, mapi, instrumentation):
+ """
+ The original model
+ :param original_model: the model to be instrumented
+ :param mapi: the mapi for that model
+ """
super(_InstrumentedModel, self).__init__()
- self._field_name = field_name
- self._model_storage = model_storage
self._original_model = original_model
+ self._mapi = mapi
+ self._instrumentation = instrumentation
self._apply_instrumentation()
def __getattr__(self, item):
- return getattr(self._original_model, item)
+ return_value = getattr(self._original_model, item)
+ if isinstance(return_value, self._original_model.__class__):
+ return _create_instrumented_model(return_value, self._mapi, self._instrumentation)
+ if isinstance(return_value, (list, dict)):
+ return _create_wrapped_model(return_value, self._mapi, self._instrumentation)
--- End diff --
check this + add test for 'recursive' e.g. get node from relationship from node
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #141: ARIA-262 inconsistent node attributes...
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/141#discussion_r120014223
--- Diff: aria/storage/api.py ---
@@ -45,6 +45,7 @@ def __init__(self, model_cls, name=None, **kwargs):
super(ModelAPI, self).__init__(**kwargs)
self._model_cls = model_cls
self._name = name or model_cls.__modelname__
+ self._instrumentation = []
--- End diff --
another solution is to create mapi for each operation/workflow
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #141: ARIA-262 inconsistent node attributes...
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/141#discussion_r120323930
--- Diff: aria/storage/api.py ---
@@ -45,7 +46,15 @@ def __init__(self, model_cls, name=None, **kwargs):
super(ModelAPI, self).__init__(**kwargs)
self._model_cls = model_cls
self._name = name or model_cls.__modelname__
- self._instrumentation = []
+ self._thread_local = threading.local()
+ self._thread_local._instrumentation = []
+
+ @property
+ def _instrumentation(self):
+ if getattr(self._thread_local, '_instrumentation', None) is None:
--- End diff --
hasattr
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #141: ARIA-262 inconsistent node attributes...
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/141#discussion_r120013538
--- Diff: aria/storage/sql_mapi.py ---
@@ -93,7 +94,7 @@ def list(self,
return ListResult(
dict(total=total, size=size, offset=offset),
- results
+ [self._instrument(result) for result in results]
)
def iter(self,
--- End diff --
consider adding instrumentation
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #141: ARIA-262 inconsistent node attributes...
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/141#discussion_r120013549
--- Diff: aria/orchestrator/decorators.py ---
@@ -68,11 +68,13 @@ def operation(func=None, toolbelt=False, suffix_template='', logging_handlers=No
@wraps(func)
def _wrapper(**func_kwargs):
+ ctx = func_kwargs.pop('ctx')
if toolbelt:
- operation_toolbelt = context.toolbelt(func_kwargs['ctx'])
+ operation_toolbelt = context.toolbelt(ctx)
func_kwargs.setdefault('toolbelt', operation_toolbelt)
- validate_function_arguments(func, func_kwargs)
- return func(**func_kwargs)
+ validate_function_arguments(func, ctx=ctx, **func_kwargs)
--- End diff --
func_kwargs
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #141: ARIA-262 inconsistent node attributes...
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/141#discussion_r120324325
--- Diff: aria/storage/core.py ---
@@ -169,14 +170,16 @@ def drop(self):
@contextmanager
def instrument(self, *instrumentation):
+ original_instrumentation = {}
def _instrument(remove=False):
--- End diff --
remove func
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #141: ARIA-262 inconsistent node attributes...
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/141#discussion_r120013922
--- Diff: aria/storage/collection_instrumentation.py ---
@@ -204,39 +200,106 @@ def _raw(self):
class _InstrumentedModel(object):
- def __init__(self, field_name, original_model, model_storage):
+ def __init__(self, original_model, mapi, instrumentation):
+ """
+ The original model
+ :param original_model: the model to be instrumented
+ :param mapi: the mapi for that model
+ """
super(_InstrumentedModel, self).__init__()
- self._field_name = field_name
- self._model_storage = model_storage
self._original_model = original_model
+ self._mapi = mapi
+ self._instrumentation = instrumentation
self._apply_instrumentation()
def __getattr__(self, item):
- return getattr(self._original_model, item)
+ return_value = getattr(self._original_model, item)
+ if isinstance(return_value, self._original_model.__class__):
+ return _create_instrumented_model(return_value, self._mapi, self._instrumentation)
+ if isinstance(return_value, (list, dict)):
+ return _create_wrapped_model(return_value, self._mapi, self._instrumentation)
+ return return_value
def _apply_instrumentation(self):
+ for field in self._instrumentation:
+ field_name = field.key
+ field_cls = field.mapper.class_
+ field = getattr(self._original_model, field_name)
+
+ # Preserve the original value. e.g. original attributes would be located under
+ # _attributes
+ setattr(self, '_{0}'.format(field_name), field)
+
+ # set instrumented value
+ if isinstance(field, dict):
+ instrumentation_cls = _InstrumentedDict
+ elif isinstance(field, list):
+ instrumentation_cls = _InstrumentedList
+ else:
+ # TODO: raise proper error
+ raise exceptions.StorageError(
+ "ARIA supports instrumentation for dict and list. Field {field} of the "
+ "class {model} is of {type} type.".format(
+ field=field,
+ model=self._original_model,
+ type=type(field)))
+
+ instrumented_class = instrumentation_cls(seq=field,
+ parent=self._original_model,
+ mapi=self._mapi,
+ field_name=field_name,
+ field_cls=field_cls)
+ setattr(self, field_name, instrumented_class)
+
+
+class _WrappedModel(object):
+
+ def __init__(self, wrapped, instrumentation, **kwargs):
+ """
+
+ :param instrumented_cls: The class to be instrumented
+ :param instrumentation_cls: the instrumentation cls
+ :param wrapped: the currently wrapped instance
+ :param kwargs: and kwargs to te passed to the instrumented class.
+ """
+ self._kwargs = kwargs
+ self._instrumentation = instrumentation
+ self._wrapped = wrapped
+
+ def _wrap(self, value):
+ if value.__class__ in (class_.class_ for class_ in self._instrumentation):
+ return _create_instrumented_model(
+ value, instrumentation=self._instrumentation, **self._kwargs)
+ elif getattr(value, 'metadata', True) == getattr(self._wrapped, 'metadata', False):
--- End diff --
comment
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #141: ARIA-262 inconsistent node attributes...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-ariatosca/pull/141
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #141: ARIA-262 inconsistent node attributes...
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/141#discussion_r120013936
--- Diff: aria/storage/collection_instrumentation.py ---
@@ -204,39 +200,106 @@ def _raw(self):
class _InstrumentedModel(object):
- def __init__(self, field_name, original_model, model_storage):
+ def __init__(self, original_model, mapi, instrumentation):
+ """
+ The original model
+ :param original_model: the model to be instrumented
+ :param mapi: the mapi for that model
+ """
super(_InstrumentedModel, self).__init__()
- self._field_name = field_name
- self._model_storage = model_storage
self._original_model = original_model
+ self._mapi = mapi
+ self._instrumentation = instrumentation
self._apply_instrumentation()
def __getattr__(self, item):
- return getattr(self._original_model, item)
+ return_value = getattr(self._original_model, item)
+ if isinstance(return_value, self._original_model.__class__):
+ return _create_instrumented_model(return_value, self._mapi, self._instrumentation)
+ if isinstance(return_value, (list, dict)):
+ return _create_wrapped_model(return_value, self._mapi, self._instrumentation)
+ return return_value
def _apply_instrumentation(self):
+ for field in self._instrumentation:
+ field_name = field.key
+ field_cls = field.mapper.class_
+ field = getattr(self._original_model, field_name)
+
+ # Preserve the original value. e.g. original attributes would be located under
+ # _attributes
+ setattr(self, '_{0}'.format(field_name), field)
+
+ # set instrumented value
+ if isinstance(field, dict):
+ instrumentation_cls = _InstrumentedDict
+ elif isinstance(field, list):
+ instrumentation_cls = _InstrumentedList
+ else:
+ # TODO: raise proper error
+ raise exceptions.StorageError(
+ "ARIA supports instrumentation for dict and list. Field {field} of the "
+ "class {model} is of {type} type.".format(
+ field=field,
+ model=self._original_model,
+ type=type(field)))
+
+ instrumented_class = instrumentation_cls(seq=field,
+ parent=self._original_model,
+ mapi=self._mapi,
+ field_name=field_name,
+ field_cls=field_cls)
+ setattr(self, field_name, instrumented_class)
+
+
+class _WrappedModel(object):
+
+ def __init__(self, wrapped, instrumentation, **kwargs):
+ """
+
+ :param instrumented_cls: The class to be instrumented
+ :param instrumentation_cls: the instrumentation cls
+ :param wrapped: the currently wrapped instance
+ :param kwargs: and kwargs to te passed to the instrumented class.
--- End diff --
te
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #141: ARIA-262 inconsistent node attributes...
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/141#discussion_r120014104
--- Diff: aria/orchestrator/decorators.py ---
@@ -48,7 +48,7 @@ def _wrapper(ctx, **workflow_parameters):
workflow_parameters.setdefault('ctx', ctx)
workflow_parameters.setdefault('graph', task_graph.TaskGraph(workflow_name))
- validate_function_arguments(func, workflow_parameters)
+ validate_function_arguments(func, **workflow_parameters)
--- End diff --
add instrumentation to workflow too?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #141: ARIA-262 inconsistent node attributes...
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/141#discussion_r120013542
--- Diff: aria/storage/core.py ---
@@ -165,3 +166,19 @@ def drop(self):
"""
for mapi in self.registered.values():
mapi.drop()
+
+ @contextmanager
+ def instrument(self, *instrumentation):
+
+ def _instrument(remove=False):
+ for mapi in self.registered.values():
+ for field in instrumentation:
+ if remove is False:
+ mapi._instrumentation.append(field)
--- End diff --
extend
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #141: ARIA-262 inconsistent node attributes...
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/141#discussion_r120014181
--- Diff: aria/orchestrator/decorators.py ---
@@ -68,11 +68,13 @@ def operation(func=None, toolbelt=False, suffix_template='', logging_handlers=No
@wraps(func)
def _wrapper(**func_kwargs):
+ ctx = func_kwargs.pop('ctx')
if toolbelt:
- operation_toolbelt = context.toolbelt(func_kwargs['ctx'])
+ operation_toolbelt = context.toolbelt(ctx)
func_kwargs.setdefault('toolbelt', operation_toolbelt)
- validate_function_arguments(func, func_kwargs)
- return func(**func_kwargs)
+ validate_function_arguments(func, ctx=ctx, **func_kwargs)
--- End diff --
instead of pop, use get, dont use **, later pop ctx.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---