You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@aurora.apache.org by mc...@apache.org on 2014/04/29 21:04:32 UTC
git commit: Stage 1 of implementing command hooks for aurora v2.
Repository: incubator-aurora
Updated Branches:
refs/heads/master 519b5871c -> 891127320
Stage 1 of implementing command hooks for aurora v2.
This change includes:
(1) The ability to add hard-wired hooks, by registering them in ConfigurationPlugins
compiled into a pex;
(2) Dynamically loaded plugins, loaded from plugin files.
The dynamically loaded plugins are *not* currently active outside of tests.
The second stage of this change will activate dynamically loaded plugins, and
provide a mechanism to allow privileged users to override hooks.
Bugs closed: aurora-270
Reviewed at https://reviews.apache.org/r/20687/
Project: http://git-wip-us.apache.org/repos/asf/incubator-aurora/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-aurora/commit/89112732
Tree: http://git-wip-us.apache.org/repos/asf/incubator-aurora/tree/89112732
Diff: http://git-wip-us.apache.org/repos/asf/incubator-aurora/diff/89112732
Branch: refs/heads/master
Commit: 891127320e6aa11f44610ea3e7ed923841544b89
Parents: 519b587
Author: Mark Chu-Carroll <mc...@twopensource.com>
Authored: Tue Apr 29 15:01:51 2014 -0400
Committer: Mark Chu-Carroll <mc...@twitter.com>
Committed: Tue Apr 29 15:01:51 2014 -0400
----------------------------------------------------------------------
docs/design/command-hooks.md | 209 +++++++++++++++++++
src/main/python/apache/aurora/client/cli/BUILD | 1 +
.../python/apache/aurora/client/cli/__init__.py | 36 +++-
.../apache/aurora/client/cli/command_hooks.py | 154 ++++++++++++++
.../python/apache/aurora/client/cli/jobs.py | 2 +-
.../python/apache/aurora/client/cli/AuroraHooks | 29 +++
src/test/python/apache/aurora/client/cli/BUILD | 14 ++
.../cli/hook_test_data/bad_syntax/AuroraHooks | 1 +
.../cli/hook_test_data/exec_error/AuroraHooks | 30 +++
.../aurora/client/cli/test_command_hooks.py | 198 ++++++++++++++++++
10 files changed, 671 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/docs/design/command-hooks.md
----------------------------------------------------------------------
diff --git a/docs/design/command-hooks.md b/docs/design/command-hooks.md
new file mode 100644
index 0000000..ee320ed
--- /dev/null
+++ b/docs/design/command-hooks.md
@@ -0,0 +1,209 @@
+# Command Hooks for the Aurora Client
+
+## Introduction/Motivation
+
+We've got hooks in the client that surround API calls. These are
+pretty awkward, because they don't correlate with user actions. For
+example, suppose we wanted a policy that said users weren't allowed to
+kill all instances of a production job at once.
+
+Right now, all that we could hook would be the "killJob" api call. But
+kill (at least in newer versions of the client) normally runs in
+batches. If a user called killall, what we would see on the API level
+is a series of "killJob" calls, each of which specified a batch of
+instances. We woudn't be able to distinguish between really killing
+all instances of a job (which is forbidden under this policy), and
+carefully killing in batches (which is permitted.) In each case, the
+hook would just see a series of API calls, and couldn't find out what
+the actual command being executed was!
+
+For most policy enforcement, what we really want to be able to do is
+look at and vet the commands that a user is performing, not the API
+calls that the client uses to implement those commands.
+
+So I propose that we add a new kind of hooks, which surround noun/verb
+commands. A hook will register itself to handle a collection of (noun,
+verb) pairs. Whenever any of those noun/verb commands are invoked, the
+hooks methods will be called around the execution of the verb. A
+pre-hook will have the ability to reject a command, preventing the
+verb from being executed.
+
+## Registering Hooks
+
+These hooks will be registered two ways:
+* Project hooks file. If a file named `AuroraHooks` is in the project directory
+ where an aurora command is being executed, that file will be read,
+ and its hooks will be registered.
+* Configuration plugins. A configuration plugin can register hooks using an API.
+ Hooks registered this way are, effectively, hardwired into the client executable.
+
+The order of execution of hooks is unspecified: they may be called in
+any order. There is no way to guarantee that one hook will execute
+before some other hook.
+
+
+### Global Hooks
+
+Commands registered by the python call are called _global_ hooks,
+because they will run for all configurations, whether or not they
+specify any hooks in the configuration file.
+
+In the implementation, hooks are registered in the module
+`apache.aurora.client.cli.command_hooks`, using the class `GlobalCommandHookRegistry`. A
+global hook can be registered by calling `GlobalCommandHookRegistry.register_command_hook`
+in a configuration plugin.
+
+### Hook Files
+
+A hook file is a file containing Python source code. It will be
+dynamically loaded by the Aurora command line executable. After
+loading, the client will check the module for a global variable named
+"hooks", which contains a list of hook objects, which will be added to
+the hook registry.
+
+A project hooks file will be named `AuroraHooks`,
+and will be located in either the directory where the command is being
+executed, or one of its parent directories, up to the nearest git/mercurial
+repository base.
+
+### The API
+
+ class CommandHook(object)
+ @property
+ def name(self):
+ """Returns a name for the hook."
+
+ def get_nouns(self):
+ """Return the nouns that have verbs that should invoke this hook."""
+
+ def get_verbs(self, noun):
+ """Return the verbs for a particular noun that should invoke his hook."""
+
+ @abstractmethod
+ def pre_command(self, noun, verb, context, commandline):
+ """Execute a hook before invoking a verb.
+ * noun: the noun being invoked.
+ * verb: the verb being invoked.
+ * context: the context object that will be used to invoke the verb.
+ The options object will be initialized before calling the hook
+ * commandline: the original argv collection used to invoke the client.
+ Returns: True if the command should be allowed to proceed; False if the command
+ should be rejected.
+ """
+
+ def post_command(self, noun, verb, context, commandline, result):
+ """Execute a hook after invoking a verb.
+ * noun: the noun being invoked.
+ * verb: the verb being invoked.
+ * context: the context object that will be used to invoke the verb.
+ The options object will be initialized before calling the hook
+ * commandline: the original argv collection used to invoke the client.
+ * result: the result code returned by the verb.
+ Returns: nothing
+ """
+
+ class GlobalCommandHookRegistry(object):
+ @classmethod
+ def register_command_hook(self, hook):
+ pass
+
+## Skipping Hooks
+
+In a perfect world, hooks would represent a global property or policy
+that should always be enforced. Unfortunately, we don't live in a
+perfect world, which means that sometimes, every rule needs to get
+broken.
+
+For example, an organization could decide that every configuration
+must be checked in to source control before it could be
+deployed. That's an entirely reasonable policy. It would be easy to
+implement it using a hook. But what if there's a problem, and the
+source repos is down?
+
+The easiest solution is just to allow a user to add a `--skip-hooks`
+flag to the command-line. But doing that means that an organization
+can't actually use hooks to enforce policy, because users can skip
+them whenever they want.
+
+Instead, we'd like to have a system where it's possible to create
+hooks to enforce policy, and then include a way of building policy
+about when hooks can be skipped.
+
+I'm using sudo as a rough model for this. Many organizations need to
+give people the ability to run privileged commands, but they still
+want to have some control. Sudo allows them to specify who is allowed
+to run a privileged command; where they're allowed to run it; and what
+command(s) they're allowed to run. All of that is specified in a
+special system file located in `/etc/sudoers` on a typical unix
+machine.
+
+### Specifying when hooks can be skipped
+
+The sudoers file has a terrible syntax, so I'm not going to try to
+adopt it; instead, I'm going to stick with the Pystachio-based
+configuration syntax that we use in Aurora. A rule that permits a
+group of users to skip hooks is defined using a Pystachio struct:
+
+ class HookRule(Struct):
+ roles = List(String)
+ commands = Map(String, List(String))
+ arg_patterns = List(String)
+ hooks = List(String)
+
+* `roles` is a list of role names, or regular expressions that range over role
+ names. This rule gives permission to those users to skip hooks.
+* `commands` is a map from nouns to lists of verbs. If a command `aurora n v`
+ is being executed, this rule allows the hooks to be skipped if
+ `v` is in `commands[n]`. If this is empty, then all commands can be skipped.
+* `arg_patterns` is a list of regular expressions ranging over parameters.
+ If any of the parameters of the command match the parameters in this list,
+ the hook can be skipped.
+* `hooks` is a list of hook identifiers which can be skipped by a user
+ that satisfies this rule.
+
+The hooks file defines a global variable `hook_rules`, which is a list of
+`HookRule` objects. If any of the hook rules matches, then the command
+can be run with hooks skipped.
+
+For example, the following is a hook rules file which allows:
+* The admin (role admin) to skip any hook.
+* Any user to skip hooks for test jobs.
+* A specific group of users to skip hooks for jobs in cluster `east`
+* Another group of users to skip hooks for `job kill` in cluster `west`.
+
+ allow_admin = HookRule(roles=['admin'])
+ allow_test = HookRule(roles=['.*'], arg_patterns=['.*/.*/test/.*'])
+ allow_east_users = HookRule(roles=['john', 'mary', 'mike', 'sue'],
+ arg_patterns=['east/.*/.*./*'])
+ allow_west_kills = HookRule(roles=['anne', 'bill', 'chris'],
+ commands = { 'job': ['kill']}, arg_patterns = ['west/.*/.*./*'])
+
+ hook_rules = [allow_admin, allow_test, allow_east_users, allow_west_kills]
+
+## Skipping Hooks
+
+To skip a hook, a user uses a command-line option, `--skip-hooks`. The option can either
+specify specific hooks to skip, or "all":
+
+* `aurora --skip-hooks=all job create east/bozo/devel/myjob` will create a job
+ without running any hooks.
+* `aurora --skip-hooks=test,iq create east/bozo/devel/myjob` will create a job,
+ and will skip only the hooks named "test" and "iq".
+
+
+## Changes
+
+Major changes between this and the last version of this proposal.
+* Command hooks can't be declared in a configuration file. There's a simple
+ reason why: hooks run before a command's implementation is invoked.
+ Config files are read during the commands invocation if necessary. If the
+ hook is declared in the config file, by the time you know that it should
+ have been run, it's too late. So I've removed config-hooks from the
+ proposal. (API hooks defined by configs still work.)
+* Skipping hooks. We expect aurora to be used inside of large
+ organizations. One of the primary use-cases of hooks is to create
+ enforcable policy that are specific to an organization. If hooks
+ can just be skipped because a user wants to skip them, it means that
+ the policy can't be enforced, which defeats the purpose of having them.
+ So in this update, I propose a mechanism, loosely based on a `sudo`-like
+ mechanism for defining when hooks can be skipped.
http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/main/python/apache/aurora/client/cli/BUILD
----------------------------------------------------------------------
diff --git a/src/main/python/apache/aurora/client/cli/BUILD b/src/main/python/apache/aurora/client/cli/BUILD
index 17cdc28..1bd565e 100644
--- a/src/main/python/apache/aurora/client/cli/BUILD
+++ b/src/main/python/apache/aurora/client/cli/BUILD
@@ -43,6 +43,7 @@ python_library(
sources = [
'__init__.py',
'context.py',
+ 'command_hooks.py',
'jobs.py',
'options.py',
'quota.py',
http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/main/python/apache/aurora/client/cli/__init__.py
----------------------------------------------------------------------
diff --git a/src/main/python/apache/aurora/client/cli/__init__.py b/src/main/python/apache/aurora/client/cli/__init__.py
index 78c109d..1e396b2 100644
--- a/src/main/python/apache/aurora/client/cli/__init__.py
+++ b/src/main/python/apache/aurora/client/cli/__init__.py
@@ -40,6 +40,7 @@ import sys
from uuid import uuid1
from twitter.common.python.pex import PexInfo
+from .command_hooks import GlobalCommandHookRegistry
# Constants for standard return codes.
EXIT_OK = 0
@@ -308,6 +309,30 @@ class CommandLine(object):
self.register_nouns()
return self.nouns.keys()
+ def run_pre_hooks(self, context, noun, verb, args, command_hooks):
+ try:
+ for hook in command_hooks:
+ result = hook.pre_command(noun, verb, context, args)
+ if result != 0:
+ print_aurora_log(logging.INFO, 'Command hook %s aborted operation with error code %s' %
+ (hook.name, result))
+ self.print_out('Command aborted by command hook %s' % hook.name)
+ return result
+ return EXIT_OK
+ except (Context.CommandError, ConfigurationPlugin.Error) as c:
+ print_aurora_log(logging.INFO, 'Error executing command hook %s: %s' % (hook.name, c))
+ self.print_err('Error executing command hook %s: %s; aborting' % hook.name, c.msg)
+ return c.code
+
+ def run_post_hooks(self, context, noun, verb, args, result, command_hooks):
+ try:
+ for hook in command_hooks:
+ hook.post_command(noun, verb, context, args, result)
+ except (Context.CommandError, ConfigurationPlugin.Error) as c:
+ print_aurora_log(logging.INFO, 'Error executing post-command hook %s: %s' % (hook.name, c))
+ self.print_err('Error executing command hook %s: %s; aborting' % hook.name, c.msg)
+ return c.code
+
def execute(self, args):
"""Execute a command.
:param args: the command-line arguments for the command. This only includes arguments
@@ -337,21 +362,28 @@ class CommandLine(object):
except ConfigurationPlugin.Error as e:
print('Error in configuration plugin before execution: %s' % c.msg, file=sys.stderr)
return c.code
+ command_hooks = GlobalCommandHookRegistry.get_command_hooks_for(options.noun, options.verb)
+ plugin_result = self.run_pre_hooks(context, options.noun, options.verb, args, command_hooks)
+ if plugin_result != 0:
+ return plugin_result
+
try:
result = noun.execute(context)
if result == EXIT_OK:
print_aurora_log(logging.INFO, 'Command terminated successfully')
+ self.run_post_hooks(options.noun, options.verb, context, args, result, command_hooks)
else:
print_aurora_log(logging.INFO, 'Commmand terminated with error code %s', result)
+
for plugin in self.plugins:
try:
plugin.after_execution(context, result)
except ConfigurationPlugin.Error as e:
- print_aurora_log(logging.INFO, 'Error executing post-execution extension hook: %s', e.msg)
+ print_aurora_log(logging.INFO, 'Error executing post-execution plugin: %s', e.msg)
return result
except Context.CommandError as c:
print_aurora_log(logging.INFO, 'Error executing command: %s', c.msg)
- print('Error executing command: %s' % c.msg, file=sys.stderr)
+ self.print_err('Error executing command: %s' % c.msg)
return c.code
http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/main/python/apache/aurora/client/cli/command_hooks.py
----------------------------------------------------------------------
diff --git a/src/main/python/apache/aurora/client/cli/command_hooks.py b/src/main/python/apache/aurora/client/cli/command_hooks.py
new file mode 100644
index 0000000..2d20068
--- /dev/null
+++ b/src/main/python/apache/aurora/client/cli/command_hooks.py
@@ -0,0 +1,154 @@
+#
+# Copyright 2014 Apache Software Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from __future__ import print_function
+
+from abc import abstractmethod
+import logging
+import os
+import sys
+
+from twitter.common.lang import Compatibility
+
+
+class GlobalCommandHookRegistry(object):
+ """Registry for command hooks.
+ See docs/design/command-hooks.md for details on what hooks are, and how
+ they work.
+ """
+
+ COMMAND_HOOKS = []
+ HOOKS_FILE_NAME = 'AuroraHooks'
+
+ @classmethod
+ def load_hooks_file(cls, path):
+ """Load a file containing hooks. If there are any errors compiling or executing the file,
+ the errors will be logged, the hooks from the file will be skipped, but the execution of
+ the command will continue.
+ """
+ # To load a hooks file, we compile and exec the file.
+ with open(path, "r") as hooks_file:
+ hooks_data = hooks_file.read()
+ hooks_code = None
+ try:
+ hooks_code = compile(hooks_data, path, "exec")
+ except (SyntaxError, TypeError) as e:
+ logging.warn("Error compiling hooks file %s: %s" % (path, e))
+ print("Error compiling hooks file %s: %s" % (path, e), file=sys.stderr)
+ return {}
+ hooks_environment = {}
+ try:
+ Compatibility.exec_function(hooks_code, hooks_environment)
+ except Exception as e:
+ # Unfortunately, exec could throw *anything* at all.
+ logging.warn('Warning: error loading hooks file %s: %s' % (path, e))
+ print('Warning: error loading hooks file %s: %s' % (path, e), file=sys.stderr)
+ return {}
+ for hook in hooks_environment.get('hooks', []):
+ cls.register_command_hook(hook)
+ return hooks_environment
+
+ @classmethod
+ def find_project_hooks_file(self, dir):
+ """Search for a file named "AuroraHooks" in current directory or
+ one of its parents, up to the closest repository root. Only one
+ file will be loaded, so creating an AuroraHooks file in a subdirecory will
+ override and replace the one in the parent directory
+ """
+ def is_repos_root(dir):
+ # a directory is a git root if it contains a directory named ".git".
+ # it's an HG root if it contains a directory named ".hg"
+ return any(os.path.isdir(os.path.join(dir, rootname)) for rootname in ['.git', '.hg'])
+
+ filepath = os.path.join(dir, self.HOOKS_FILE_NAME)
+ if os.path.exists(filepath):
+ return filepath
+ elif is_repos_root(dir):
+ # This is a repository root, so stop looking.
+ return None
+ else:
+ parent, _ = os.path.split(os.path.abspath(dir))
+ # if we've reached the filesystem root, then parent==self.
+ if parent == dir:
+ return None
+ else:
+ return find_project_hooks_file(parent)
+ return None
+
+ @classmethod
+ def load_project_hooks(cls, dir="."):
+ """Looks for a project hooks file. Project hooks will be in the current directory,
+ or one of its parents. Stops looking at the root of an SCM repository.
+ """
+ hooks_file = cls.find_project_hooks_file(dir)
+ if hooks_file is not None:
+ return cls.load_hooks_file(hooks_file)
+ else:
+ return None
+
+ @classmethod
+ def reset(cls):
+ """For testing purposes, reset the list of registered hooks"""
+ cls.COMMAND_HOOKS = []
+
+ @classmethod
+ def get_command_hooks_for(cls, noun, verb):
+ """Looks up the set of hooks tht should apply for a (noun, verb) pair.
+ """
+ return [hook for hook in cls.COMMAND_HOOKS if noun in hook.get_nouns() and
+ verb in hook.get_verbs(noun)]
+
+ @classmethod
+ def register_command_hook(cls, hook):
+ cls.COMMAND_HOOKS.append(hook)
+
+
+class CommandHook(object):
+ @property
+ def name(self):
+ return None
+
+ @abstractmethod
+ def get_nouns(self):
+ """Return the nouns that have verbs that should invoke this hook."""
+
+ @abstractmethod
+ def get_verbs(self, noun):
+ """Return the verbs for a particular noun that should invoke his hook."""
+
+ @abstractmethod
+ def pre_command(self, noun, verb, context, commandline):
+ """Execute a hook before invoking a verb.
+ * noun: the noun being invoked.
+ * verb: the verb being invoked.
+ * context: the context object that will be used to invoke the verb.
+ The options object will be initialized before calling the hook
+ * commandline: the original argv collection used to invoke the client.
+ Returns: 0 if the command should be allowed to proceed; otherwise, the exit code
+ to return at the command line.
+ """
+
+ @abstractmethod
+ def post_command(self, noun, verb, context, commandline, result):
+ """Execute a hook after invoking a verb.
+ * noun: the noun being invoked.
+ * verb: the verb being invoked.
+ * context: the context object that will be used to invoke the verb.
+ * commandline: the original argv collection used to invoke the client.
+ * result: the result code returned by the verb.
+ Returns: nothing
+ """
+
http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/main/python/apache/aurora/client/cli/jobs.py
----------------------------------------------------------------------
diff --git a/src/main/python/apache/aurora/client/cli/jobs.py b/src/main/python/apache/aurora/client/cli/jobs.py
index bd7a308..782b348 100644
--- a/src/main/python/apache/aurora/client/cli/jobs.py
+++ b/src/main/python/apache/aurora/client/cli/jobs.py
@@ -304,7 +304,7 @@ class AbstractKillCommand(Verb):
batch.append(instances_to_kill.pop())
resp = api.kill_job(job, batch)
if resp.responseCode is not ResponseCode.OK:
- resp.print_log('Kill of shards %s failed with error %s' % (batch, resp.message))
+ context.print_log('Kill of shards %s failed with error %s' % (batch, resp.message))
errors += 1
if errors > context.options.max_total_failures:
raise context.CommandError(EXIT_COMMAND_FAILURE,
http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/test/python/apache/aurora/client/cli/AuroraHooks
----------------------------------------------------------------------
diff --git a/src/test/python/apache/aurora/client/cli/AuroraHooks b/src/test/python/apache/aurora/client/cli/AuroraHooks
new file mode 100644
index 0000000..c25bcfb
--- /dev/null
+++ b/src/test/python/apache/aurora/client/cli/AuroraHooks
@@ -0,0 +1,29 @@
+from apache.aurora.client.cli.command_hooks import CommandHook
+
+class DynamicHook(CommandHook):
+
+ def __init__(self, succeed):
+ self.succeed = succeed
+ self.ran_pre = False
+ self.ran_post = False
+
+ def get_nouns(self):
+ return ['job']
+
+ def get_verbs(self, noun):
+ if noun == 'job':
+ return ['create', 'status' ]
+ else:
+ return []
+
+ def pre_command(self, noun, verb, context, commandline):
+ self.ran_pre = True
+ if self.succeed:
+ return 0
+ else:
+ return 1
+
+ def post_command(self, noun, verb, context, commandline, result):
+ self.ran_post = True
+
+hooks = [ DynamicHook(True) ]
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/test/python/apache/aurora/client/cli/BUILD
----------------------------------------------------------------------
diff --git a/src/test/python/apache/aurora/client/cli/BUILD b/src/test/python/apache/aurora/client/cli/BUILD
index 34fdb47..1504476 100644
--- a/src/test/python/apache/aurora/client/cli/BUILD
+++ b/src/test/python/apache/aurora/client/cli/BUILD
@@ -18,6 +18,7 @@ python_test_suite(
name = 'all',
dependencies = [
pants(':bridge'),
+ pants(':command_hooks'),
pants(':help'),
pants(':job'),
pants(':logging'),
@@ -49,6 +50,19 @@ python_tests(
)
python_tests(
+ name = 'command_hooks',
+ sources = [ 'test_command_hooks.py' ],
+ dependencies = [
+ pants(':util'),
+ pants('3rdparty/python:mock'),
+ pants('3rdparty/python:twitter.common.contextutil'),
+ pants('src/main/python/apache/aurora/client/cli'),
+ pants('src/main/python/apache/aurora/client/cli:client'),
+ pants('src/test/python/apache/aurora/client/commands:util')
+ ]
+)
+
+python_tests(
name = 'logging',
sources = [ 'test_logging.py' ],
dependencies = [
http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/test/python/apache/aurora/client/cli/hook_test_data/bad_syntax/AuroraHooks
----------------------------------------------------------------------
diff --git a/src/test/python/apache/aurora/client/cli/hook_test_data/bad_syntax/AuroraHooks b/src/test/python/apache/aurora/client/cli/hook_test_data/bad_syntax/AuroraHooks
new file mode 100644
index 0000000..6333986
--- /dev/null
+++ b/src/test/python/apache/aurora/client/cli/hook_test_data/bad_syntax/AuroraHooks
@@ -0,0 +1 @@
+This is not valid python syntax.
http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/test/python/apache/aurora/client/cli/hook_test_data/exec_error/AuroraHooks
----------------------------------------------------------------------
diff --git a/src/test/python/apache/aurora/client/cli/hook_test_data/exec_error/AuroraHooks b/src/test/python/apache/aurora/client/cli/hook_test_data/exec_error/AuroraHooks
new file mode 100644
index 0000000..768c124
--- /dev/null
+++ b/src/test/python/apache/aurora/client/cli/hook_test_data/exec_error/AuroraHooks
@@ -0,0 +1,30 @@
+from apache.aurora.client.cli.command_hooks import CommandHook
+
+class DynamicHook(CommandHook):
+
+ def __init__(self, succeed):
+ self.succeed = succeed
+ self.ran_pre = False
+ self.ran_post = False
+
+ def get_nouns(self):
+ return ['job']
+
+ def get_verbs(self, noun):
+ if noun == 'job':
+ return ['create', 'status' ]
+ else:
+ return []
+
+ def pre_command(self, noun, verb, context, commandline):
+ self.ran_pre = True
+ if self.succeed:
+ return 0
+ else:
+ return 1
+
+ def post_command(self, noun, verb, context, commandline, result):
+ self.ran_post = True
+
+
+hooks = [ DynamicHook(True), 1/0 ]
http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/test/python/apache/aurora/client/cli/test_command_hooks.py
----------------------------------------------------------------------
diff --git a/src/test/python/apache/aurora/client/cli/test_command_hooks.py b/src/test/python/apache/aurora/client/cli/test_command_hooks.py
new file mode 100644
index 0000000..7c6f70c
--- /dev/null
+++ b/src/test/python/apache/aurora/client/cli/test_command_hooks.py
@@ -0,0 +1,198 @@
+#
+# Copyright 2014 Apache Software Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from twitter.common.contextutil import temporary_file
+
+from gen.apache.aurora.api.ttypes import (
+ AssignedTask,
+ Identity,
+ ScheduledTask,
+ ScheduleStatus,
+ ScheduleStatusResult,
+ TaskEvent,
+ TaskQuery,
+)
+
+from apache.aurora.client.cli.client import AuroraCommandLine
+from apache.aurora.client.cli.command_hooks import CommandHook, GlobalCommandHookRegistry
+from apache.aurora.client.cli.util import AuroraClientCommandTest, FakeAuroraCommandContext
+from apache.aurora.config import AuroraConfig
+
+from mock import Mock, patch
+
+
+class HookForTesting(CommandHook):
+ def __init__(self, succeed):
+ self.succeed = succeed
+ self.ran_pre = False
+ self.ran_post = False
+
+ def get_nouns(self):
+ return ['job']
+
+ def get_verbs(self, noun):
+ if noun == 'job':
+ return ['create', 'status']
+ else:
+ return []
+
+ def pre_command(self, noun, verb, context, commandline):
+ self.ran_pre = True
+ if self.succeed:
+ return 0
+ else:
+ return 1
+
+ def post_command(self, noun, verb, context, commandline, result):
+ self.ran_post = True
+
+
+class TestClientCreateCommand(AuroraClientCommandTest):
+
+ @classmethod
+ def create_mock_task(cls, task_id, instance_id, initial_time, status):
+ mock_task = Mock(spec=ScheduledTask)
+ mock_task.assignedTask = Mock(spec=AssignedTask)
+ mock_task.assignedTask.taskId = task_id
+ mock_task.assignedTask.instanceId = instance_id
+ mock_task.status = status
+ mock_task_event = Mock(spec=TaskEvent)
+ mock_task_event.timestamp = initial_time
+ mock_task.taskEvents = [mock_task_event]
+ return mock_task
+
+ @classmethod
+ def create_mock_status_query_result(cls, scheduleStatus):
+ mock_query_result = cls.create_simple_success_response()
+ mock_query_result.result.scheduleStatusResult = Mock(spec=ScheduleStatusResult)
+ if scheduleStatus == ScheduleStatus.INIT:
+ # status query result for before job is launched.
+ mock_query_result.result.scheduleStatusResult.tasks = []
+ else:
+ mock_task_one = cls.create_mock_task('hello', 0, 1000, scheduleStatus)
+ mock_task_two = cls.create_mock_task('hello', 1, 1004, scheduleStatus)
+ mock_query_result.result.scheduleStatusResult.tasks = [mock_task_one, mock_task_two]
+ return mock_query_result
+
+ @classmethod
+ def create_mock_query(cls):
+ return TaskQuery(owner=Identity(role=cls.TEST_ROLE), environment=cls.TEST_ENV,
+ jobName=cls.TEST_JOB)
+
+ @classmethod
+ def get_createjob_response(cls):
+ # Then, we call api.create_job(config)
+ return cls.create_simple_success_response()
+
+ @classmethod
+ def get_failed_createjob_response(cls):
+ return cls.create_error_response()
+
+ @classmethod
+ def assert_create_job_called(cls, mock_api):
+ # Check that create_job was called exactly once, with an AuroraConfig parameter.
+ assert mock_api.create_job.call_count == 1
+ assert isinstance(mock_api.create_job.call_args_list[0][0][0], AuroraConfig)
+
+ @classmethod
+ def assert_scheduler_called(cls, mock_api, mock_query, num_queries):
+ assert mock_api.scheduler_proxy.getTasksStatus.call_count == num_queries
+ mock_api.scheduler_proxy.getTasksStatus.assert_called_with(mock_query)
+
+ def test_create_job_with_successful_hook(self):
+ GlobalCommandHookRegistry.reset()
+ command_hook = HookForTesting(True)
+ GlobalCommandHookRegistry.register_command_hook(command_hook)
+ self.generic_test_successful_hook(command_hook)
+
+ def generic_test_successful_hook(self, command_hook):
+ mock_context = FakeAuroraCommandContext()
+ with patch('apache.aurora.client.cli.jobs.Job.create_context', return_value=mock_context):
+ mock_query = self.create_mock_query()
+ mock_context.add_expected_status_query_result(
+ self.create_mock_status_query_result(ScheduleStatus.INIT))
+ mock_context.add_expected_status_query_result(
+ self.create_mock_status_query_result(ScheduleStatus.RUNNING))
+ api = mock_context.get_api('west')
+ api.create_job.return_value = self.get_createjob_response()
+
+ with temporary_file() as fp:
+ fp.write(self.get_valid_config())
+ fp.flush()
+ cmd = AuroraCommandLine()
+ cmd.execute(['job', 'create', '--wait-until=RUNNING', 'west/bozo/test/hello',
+ fp.name])
+
+ self.assert_create_job_called(api)
+ self.assert_scheduler_called(api, mock_query, 2)
+ assert command_hook.ran_pre
+ assert command_hook.ran_post
+
+ def test_create_job_with_failed_hook(self):
+ GlobalCommandHookRegistry.reset()
+ command_hook = HookForTesting(False)
+ GlobalCommandHookRegistry.register_command_hook(command_hook)
+ mock_context = FakeAuroraCommandContext()
+ with patch('apache.aurora.client.cli.jobs.Job.create_context', return_value=mock_context):
+ mock_context.add_expected_status_query_result(
+ self.create_mock_status_query_result(ScheduleStatus.INIT))
+ mock_context.add_expected_status_query_result(
+ self.create_mock_status_query_result(ScheduleStatus.RUNNING))
+ api = mock_context.get_api('west')
+ api.create_job.return_value = self.get_createjob_response()
+
+ with temporary_file() as fp:
+ fp.write(self.get_valid_config())
+ fp.flush()
+ cmd = AuroraCommandLine()
+ result = cmd.execute(['job', 'create', '--wait-until=RUNNING', 'west/bozo/test/hello',
+ fp.name])
+
+ assert result == 1
+ assert api.create_job.call_count == 0
+ assert command_hook.ran_pre
+ assert not command_hook.ran_post
+
+ def test_load_dynamic_hooks(self):
+ GlobalCommandHookRegistry.reset()
+ hook_locals = GlobalCommandHookRegistry.load_project_hooks(
+ './src/test/python/apache/aurora/client/cli')
+ assert hook_locals['hooks'][0] in GlobalCommandHookRegistry.COMMAND_HOOKS
+
+ def test_successful_dynamic_hook(self):
+ GlobalCommandHookRegistry.reset()
+ hook_locals = GlobalCommandHookRegistry.load_project_hooks(
+ './src/test/python/apache/aurora/client/cli')
+ self.generic_test_successful_hook(hook_locals['hooks'][0])
+
+ def test_dynamic_hook_syntax_error(self):
+ with patch('logging.warn') as log_patch:
+ GlobalCommandHookRegistry.reset()
+ hook_locals = GlobalCommandHookRegistry.load_project_hooks(
+ './src/test/python/apache/aurora/client/cli/hook_test_data/bad_syntax')
+ log_patch.assert_called_with('Error compiling hooks file '
+ './src/test/python/apache/aurora/client/cli/hook_test_data/bad_syntax/AuroraHooks: '
+ 'invalid syntax (AuroraHooks, line 1)')
+
+ def test_dynamic_hook_exec_error(self):
+ with patch('logging.warn') as log_patch:
+ GlobalCommandHookRegistry.reset()
+ hook_locals = GlobalCommandHookRegistry.load_project_hooks(
+ './src/test/python/apache/aurora/client/cli/hook_test_data/exec_error')
+ log_patch.assert_called_with('Warning: error loading hooks file '
+ './src/test/python/apache/aurora/client/cli/hook_test_data/exec_error/AuroraHooks: '
+ 'integer division or modulo by zero')
+