Inheritance in .aurora

[Anindya Sinha <an...@gmail.com>]

Hi

I am playing around with .aurora files and am facing an issue when I am
using a class hierarchy with the base class derived from Struct, as follows:

The good case:
class Profile(Struct):
    environment = Required(String)
    name = Default(String, 'test_app')

PRODUCTION = Profile(environment = 'prod', name = 'my_profile_app')

In this case, I have value populated in the following Mustache variables
(which is fine):
a) {{profile.environment}} = 'prod'
b) {{profile.name}} = 'my_profile_app'

Now, I add the following class that inherits from the base class Profile:
class DerivedProfile(Profile):
    mz_name = Default(String, 'my_derived_profile_app')

and modify PRODUCTION as:
PRODUCTION = DerivedProfile(environment = 'prod', name = 'my_profile_app')

In this case, {{profile.mz_name}} is reported as invalid or as an"unbound
variable" (output of aurora inspect below). Is this supported via Pystachio
templating, ie. create a class hierarchy and ensure that I have access to
the Mustache variables. Any quick pointers before I get into the details?

My use case is that I want to have a base class with all my common
attributes for all my apps, and a derived class for each app that handles
app specific attributes.

Thanks
Anindya

Following is output of aurora inspect:

Traceback (most recent call last):
  File
"/usr/local/bin/aurora/.deps/twitter.common.app-0.1.0-py2.6.egg/twitter/common/app/application.py",
line 738, in _wrap_method
    return_code = method()
  File
"/usr/local/bin/aurora/.deps/twitter.common.app-0.1.0-py2.6.egg/twitter/common/app/application.py",
line 760, in <lambda>
    main = lambda: main_method(*args, **kwargs)
  File "/usr/local/bin/aurora/apache/aurora/client/base.py", line 74, in
wrapped_function
    return fn(*args)
  File "/usr/local/bin/aurora/apache/aurora/client/commands/core.py", line
273, in inspect
    job_thrift = config.job()
  File "/usr/local/bin/aurora/apache/aurora/config/__init__.py", line 198,
in job
    raise self.InvalidConfig(str(e))
InvalidConfig: The configuration was invalid: Config contains unbound
variables: {{profile.mz_name}}

Re: Inheritance in .aurora

[Brian Wickman <wi...@apache.org>]

Inheritance in pystachio is not pythonic, so what you're trying to do won't
work.  I consider this a broken window.  We've been discussing how to
improve this for a while in a backwards compatible fashion and will be
tracking ideas in an epic centered around
AURORA-188<https://issues.apache.org/jira/browse/AURORA-188>.
 What's likely to happen is that we move to a pattern where you create
Traits (Structs as they are today) and then compose them together with
combine()/extend() methods.  This would allow you to e.g. add new traits on
a Job object that are organization specific, like for example DockerImage.

A shorter term, easier-to-implement solution that has been discussed is to
add an .extend() method to Struct such that you could do something like the
following:

class Profile(Struct):
  variable1 = String
  variable2 = Default(Integer, 5)

DerivedProfile = Profile.extend(
  variable2 = Default(Integer, 6),
  variable3 = Required(String))

so long as the derived attributes do not collide with types from the
parent, so as to maintain drop-in compatibility.  There are a lot of
drawbacks with this approach (complexity first and foremost) but it's worth
considering implementing.

In the meantime, you can always do something like:

class BaseProfile(Struct):
  jvm_ram_mb = Default(Integer, 512)
  jvm_flags = Default(String, '-Xmx{{jvm_ram_mb}} {{profile_name}}.conf')
  profile_name = Required(String)

DevProfile = BaseProfile(profile_name = 'dev')
ProdProfile = BaseProfile(profile_name = 'prod', jvm_ram_mb = 16384)
StagingProfile = ProdProfile(profile_name = 'staging')

And essentially achieve a similar effect, if not clean hierarchical
composability.

~brian





On Thu, Apr 3, 2014 at 8:34 PM, Anindya Sinha <an...@gmail.com>wrote:

> Hi
>
> I am playing around with .aurora files and am facing an issue when I am
> using a class hierarchy with the base class derived from Struct, as
> follows:
>
> The good case:
> class Profile(Struct):
>     environment = Required(String)
>     name = Default(String, 'test_app')
>
> PRODUCTION = Profile(environment = 'prod', name = 'my_profile_app')
>
> In this case, I have value populated in the following Mustache variables
> (which is fine):
> a) {{profile.environment}} = 'prod'
> b) {{profile.name}} = 'my_profile_app'
>
> Now, I add the following class that inherits from the base class Profile:
> class DerivedProfile(Profile):
>     mz_name = Default(String, 'my_derived_profile_app')
>
> and modify PRODUCTION as:
> PRODUCTION = DerivedProfile(environment = 'prod', name = 'my_profile_app')
>
> In this case, {{profile.mz_name}} is reported as invalid or as an"unbound
> variable" (output of aurora inspect below). Is this supported via Pystachio
> templating, ie. create a class hierarchy and ensure that I have access to
> the Mustache variables. Any quick pointers before I get into the details?
>
> My use case is that I want to have a base class with all my common
> attributes for all my apps, and a derived class for each app that handles
> app specific attributes.
>
> Thanks
> Anindya
>
> Following is output of aurora inspect:
>
> Traceback (most recent call last):
>   File
>
> "/usr/local/bin/aurora/.deps/twitter.common.app-0.1.0-py2.6.egg/twitter/common/app/application.py",
> line 738, in _wrap_method
>     return_code = method()
>   File
>
> "/usr/local/bin/aurora/.deps/twitter.common.app-0.1.0-py2.6.egg/twitter/common/app/application.py",
> line 760, in <lambda>
>     main = lambda: main_method(*args, **kwargs)
>   File "/usr/local/bin/aurora/apache/aurora/client/base.py", line 74, in
> wrapped_function
>     return fn(*args)
>   File "/usr/local/bin/aurora/apache/aurora/client/commands/core.py", line
> 273, in inspect
>     job_thrift = config.job()
>   File "/usr/local/bin/aurora/apache/aurora/config/__init__.py", line 198,
> in job
>     raise self.InvalidConfig(str(e))
> InvalidConfig: The configuration was invalid: Config contains unbound
> variables: {{profile.mz_name}}
>