You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Renan DelValle <rd...@binghamton.edu> on 2015/08/27 00:19:10 UTC
Review Request 37818: Moved executor settings configuration to
loadable JSON
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/
-----------------------------------------------------------
Review request for Aurora.
Repository: aurora
Description
-------
This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
Diffs
-----
examples/vagrant/executors-config-new.json PRE-CREATION
examples/vagrant/executors-config.json PRE-CREATION
examples/vagrant/upstart/aurora-scheduler-kerberos.conf 744b4a35c61e749734e222b3d4cbd296927665aa
examples/vagrant/upstart/aurora-scheduler.conf 789a3a0315e8530880999432aa9b1e7d0f57d1ff
src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 0c440b5cd5b939872c1ee05d048bf739bfa977cb
src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
src/test/resources/org/apache/aurora/scheduler/configuration/thermos-settings-example.json PRE-CREATION
Diff: https://reviews.apache.org/r/37818/diff/
Testing
-------
./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
Thanks,
Renan DelValle
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Kevin Sweeney <ke...@apache.org>.
> On Aug. 26, 2015, 3:27 p.m., Kevin Sweeney wrote:
> > examples/vagrant/executors-config-new.json, line 18
> > <https://reviews.apache.org/r/37818/diff/1/?file=1055421#file1055421line18>
> >
> > this isn't a global property - can it be pushed into a custom configuration object?
>
> Renan DelValle wrote:
> We can, but this change causes a ripple effect. If we push this into a custom schema, we have to provide a way of unpacking custom configuration object into useful mesos data fields or information usefol to the executor. We need to come to a decision on the best way of doing this if we go for it. Tangentially, as far as I can tell this is only used when building container volumes.
As this is greenfield code I'm okay with a bit of a ripple - I'd much rather cluster operators have to write this config file once than change it as our design for it evolves.
> On Aug. 26, 2015, 3:27 p.m., Kevin Sweeney wrote:
> > examples/vagrant/executors-config-new.json, line 19
> > <https://reviews.apache.org/r/37818/diff/1/?file=1055421#file1055421line19>
> >
> > It would be ergnomically nice to default this to an empty list if it's left unset.
>
> Renan DelValle wrote:
> I'll see if there's a way of doing that. If you come across anything, let me know.
A bit of Googling suggests that setting defaults in a default constructor works with GSON here. In fact it seems that just about any way you can think of doing it GSON supports defaults (confirmed myself here: https://gist.github.com/kevints/d1e26514468863e8c088).
Setting defaults in an initializer works too - I suggest following the pattern in `ImmutableDefaultTest`, or `ImmutableInitializeOverrideTest` if you have a `@VisibleForTesting` constructor.
- Kevin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review96605
-----------------------------------------------------------
On Aug. 26, 2015, 3:19 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Aug. 26, 2015, 3:19 p.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config-new.json PRE-CREATION
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 744b4a35c61e749734e222b3d4cbd296927665aa
> examples/vagrant/upstart/aurora-scheduler.conf 789a3a0315e8530880999432aa9b1e7d0f57d1ff
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 0c440b5cd5b939872c1ee05d048bf739bfa977cb
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/thermos-settings-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Renan DelValle <rd...@binghamton.edu>.
> On Aug. 26, 2015, 10:27 p.m., Kevin Sweeney wrote:
> > examples/vagrant/executors-config-new.json, line 19
> > <https://reviews.apache.org/r/37818/diff/1/?file=1055421#file1055421line19>
> >
> > It would be ergnomically nice to default this to an empty list if it's left unset.
>
> Renan DelValle wrote:
> I'll see if there's a way of doing that. If you come across anything, let me know.
>
> Kevin Sweeney wrote:
> A bit of Googling suggests that setting defaults in a default constructor works with GSON here. In fact it seems that just about any way you can think of doing it GSON supports defaults (confirmed myself here: https://gist.github.com/kevints/d1e26514468863e8c088).
>
> Setting defaults in an initializer works too - I suggest following the pattern in `ImmutableDefaultTest`, or `ImmutableInitializeOverrideTest` if you have a `@VisibleForTesting` constructor.
Awesome. Thanks for the tips. I just realized that since I'm using a builder class to re-enforce all the checks that GSON bypasses with reflection, I can add two lines of code in the ExecutorSettings.Builder class and accomplish this. Let me know if this will fly.
In ExecutorSettings.Builder, the globalContainerMounts is set to an empty immutable list in the constructor. In GSON, if an field isn't found, it's set to null.
With these two assumptions in mind in the builder code I changed the code
From:
```
public Builder setGlobalContainerMounts(List<Volume> globalContainerMounts) {
this.globalContainerMounts = globalContainerMounts;
return this;
}
```
To:
```
public Builder setGlobalContainerMounts(List<Volume> globalContainerMounts) {
if(nonNull(globalContainerMounts)) {
this.globalContainerMounts = globalContainerMounts;
}
return this;
}
```
This way I don't have to create a new class to get this done. I tested this out and both with "globalContainerMounts":[] and globalContainerMounts missing from the schema and both return an empty list.
> On Aug. 26, 2015, 10:27 p.m., Kevin Sweeney wrote:
> > examples/vagrant/executors-config-new.json, line 18
> > <https://reviews.apache.org/r/37818/diff/1/?file=1055421#file1055421line18>
> >
> > this isn't a global property - can it be pushed into a custom configuration object?
>
> Renan DelValle wrote:
> We can, but this change causes a ripple effect. If we push this into a custom schema, we have to provide a way of unpacking custom configuration object into useful mesos data fields or information usefol to the executor. We need to come to a decision on the best way of doing this if we go for it. Tangentially, as far as I can tell this is only used when building container volumes.
>
> Kevin Sweeney wrote:
> As this is greenfield code I'm okay with a bit of a ripple - I'd much rather cluster operators have to write this config file once than change it as our design for it evolves.
Got it, I understand where you're coming from. This is the last thing to take care of before I upload the next patch. I have included a custom field in the schema needs to be unpacked. Any suggestions on how I should approach unpacking the custom schema? For example, should we create an interface that can be used for unpacking the schema and returning a builder of some sort?
- Renan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review96605
-----------------------------------------------------------
On Aug. 26, 2015, 10:19 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Aug. 26, 2015, 10:19 p.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config-new.json PRE-CREATION
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 744b4a35c61e749734e222b3d4cbd296927665aa
> examples/vagrant/upstart/aurora-scheduler.conf 789a3a0315e8530880999432aa9b1e7d0f57d1ff
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 0c440b5cd5b939872c1ee05d048bf739bfa977cb
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/thermos-settings-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Renan DelValle <rd...@binghamton.edu>.
> On Aug. 26, 2015, 10:27 p.m., Kevin Sweeney wrote:
> > src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java, line 114
> > <https://reviews.apache.org/r/37818/diff/1/?file=1055427#file1055427line114>
> >
> > drop redundant use of `this`, here and below.
Done, apologies, this one slipped through the cracks from the previous reviews.
> On Aug. 26, 2015, 10:27 p.m., Kevin Sweeney wrote:
> > examples/vagrant/executors-config.json, line 1
> > <https://reviews.apache.org/r/37818/diff/1/?file=1055422#file1055422line1>
> >
> > delete this file? It looks like it has been superceded by the new format below
This first cut wasn't supposed to include the new schema. I was gonna implement it in the second patch but I guess now is as good a time as any to get this in.
> On Aug. 26, 2015, 10:27 p.m., Kevin Sweeney wrote:
> > examples/vagrant/executors-config-new.json, line 19
> > <https://reviews.apache.org/r/37818/diff/1/?file=1055421#file1055421line19>
> >
> > It would be ergnomically nice to default this to an empty list if it's left unset.
I'll see if there's a way of doing that. If you come across anything, let me know.
> On Aug. 26, 2015, 10:27 p.m., Kevin Sweeney wrote:
> > examples/vagrant/executors-config-new.json, line 18
> > <https://reviews.apache.org/r/37818/diff/1/?file=1055421#file1055421line18>
> >
> > this isn't a global property - can it be pushed into a custom configuration object?
We can, but this change causes a ripple effect. If we push this into a custom schema, we have to provide a way of unpacking custom configuration object into useful mesos data fields or information usefol to the executor. We need to come to a decision on the best way of doing this if we go for it. Tangentially, as far as I can tell this is only used when building container volumes.
> On Aug. 26, 2015, 10:27 p.m., Kevin Sweeney wrote:
> > examples/vagrant/executors-config-new.json, lines 13-15
> > <https://reviews.apache.org/r/37818/diff/1/?file=1055421#file1055421line13>
> >
> > do these have defaults that we can omit? also I don't think we want to cache in the vagrant environment as we want changes to be reflected in new tasks immediately when developing.
As per the mesos documentation, all options but the string value are optional. What isn't very clear is how the extract is true by default if it is optional. (source: https://github.com/apache/mesos/blob/master/docs/fetcher.md)
- Renan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review96605
-----------------------------------------------------------
On Aug. 26, 2015, 10:19 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Aug. 26, 2015, 10:19 p.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config-new.json PRE-CREATION
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 744b4a35c61e749734e222b3d4cbd296927665aa
> examples/vagrant/upstart/aurora-scheduler.conf 789a3a0315e8530880999432aa9b1e7d0f57d1ff
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 0c440b5cd5b939872c1ee05d048bf739bfa977cb
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/thermos-settings-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Kevin Sweeney <ke...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review96605
-----------------------------------------------------------
Quick (incomplete) review on the design of the config object
examples/vagrant/executors-config-new.json (lines 13 - 15)
<https://reviews.apache.org/r/37818/#comment152193>
do these have defaults that we can omit? also I don't think we want to cache in the vagrant environment as we want changes to be reflected in new tasks immediately when developing.
examples/vagrant/executors-config-new.json (line 14)
<https://reviews.apache.org/r/37818/#comment152195>
don't extract this
examples/vagrant/executors-config-new.json (line 18)
<https://reviews.apache.org/r/37818/#comment152194>
this isn't a global property - can it be pushed into a custom configuration object?
examples/vagrant/executors-config-new.json (line 19)
<https://reviews.apache.org/r/37818/#comment152192>
It would be ergnomically nice to default this to an empty list if it's left unset.
examples/vagrant/executors-config.json (line 1)
<https://reviews.apache.org/r/37818/#comment152191>
delete this file? It looks like it has been superceded by the new format below
src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java (line 114)
<https://reviews.apache.org/r/37818/#comment152196>
drop redundant use of `this`, here and below.
- Kevin Sweeney
On Aug. 26, 2015, 3:19 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Aug. 26, 2015, 3:19 p.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config-new.json PRE-CREATION
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 744b4a35c61e749734e222b3d4cbd296927665aa
> examples/vagrant/upstart/aurora-scheduler.conf 789a3a0315e8530880999432aa9b1e7d0f57d1ff
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 0c440b5cd5b939872c1ee05d048bf739bfa977cb
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/thermos-settings-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97389
-----------------------------------------------------------
Ship it!
Master (07838c3) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Sept. 2, 2015, 2:35 a.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 2, 2015, 2:35 a.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Renan DelValle <rd...@binghamton.edu>.
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java, line 87
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061926#file1061926line87>
> >
> > This code will be easier to maintain long-term with 2 goals in mind:
> > - keep json-related code within this class (no annotations on ExecutorSettings)
> > - avoid hand-rolling deserializers
> >
> > This sounds like more work, but it's not that bad. The major difference is that you'll create a class in this file that defines the JSON structure, and does so in a way that the json deserializer can work with.
> >
> > Maxim just did something very similar, which you can crib from:
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55
>
> Bill Farner wrote:
> Oh, the detail i neglected to mention - i suggest you then write code to translate your config schema object into ExecutorSettings. This can provides compile-time guarantees that we don't have with the current code, making it easier to maintain.
>
> Renan DelValle wrote:
> Can you expend on this? I don't have any experience with Jackson. I'm looking through what Maxim did and I'll definitely crib here and there but I can't find anything related to getting this done at compile time.
>
> Bill Farner wrote:
> Don't be thrown by the mention of compile time, i'm just referring to refactor safety if/when the structures changed. The concise way to put it is - use a different class for your config domain object. This way you don't need to change ExecutorSettings to compensate for the fact that some code in another part of the system is using a json mapper.
>
> Start by defining a class that matches your config schema:
> ```
> class ExecutorConfig {
> String command;
> List<Resources> resource;
> ...
> }
>
> class Resource {
> String value; // note - i would like to see this renamed to uri
> boolean executable;
> boolean extract;
> boolean cache;
> }
> ...
> ```
> (that's just a quick example, feel free to adjust as you see fit)
>
> Jackson is very similar to gson, there's just a behavior nuance that is desirable. You'll use an `ObjectMapper` to parse the json to an `ExecutorConfig`, then write a function to transform your `ExecutorConfig` into `ExecutorSettings`.
>
> ```
> private static ExecutorSettings configToSettings(ExecutorConfig config) {
> return new ExecutorSettings(...);
> }
> ```
>
> Bill Farner wrote:
> I took a quick stab at illustrating this, take a look here: https://github.com/wfarner/aurora/commit/a53c215432b34ba95238121581375bde24b88a1e
>
> The remaining work is to complete filling out constructor fields:
> https://github.com/wfarner/aurora/blob/a53c215432b34ba95238121581375bde24b88a1e/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java#L53-L61
>
> This is really a matter of writing functions to translate from types in `ExecutorConfig` to the corresponding types in `ExecutorSettings`.
>
> Depending on your bandwidth over the next few days, i may take over this patch if you don't mind.
>
> Renan DelValle wrote:
> Thanks for the examples, in particular the entry transformer really helped speed things along.
>
> I've almost got this done, and I'm having an issue with using optionals, which in my opinion are necessary if we want to leave the defaults set on the URI resource configuration if nothing is specified. I.e: If executable is not in the schema for a URI, then its better to not set it than to set a default value, IMO.
>
> I've looked around and Jackson has support for them if we load a new module:
> https://github.com/FasterXML/jackson-datatype-jdk8
>
> However, I can't find where the jackson dependecy is. And the fact the jackson pacakges still point to codehaus lead me to believe were using an old version.
>
> Here are some relevant files:
> https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorConfiguration.java
> https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
>
> We're almost there.
>
> Bill Farner wrote:
> Perhaps we should go a step further and try to directly parse `ExecutorInfo`? That way we don't have to maintain parity between the protobuf schema and our own. Jackson can't directly deserialize protobuf-generated objects, but a quick search turned up this promising project: https://github.com/HubSpot/jackson-datatype-protobuf
>
> The deps are defined in build.gradle. You're right about the old jackson library (pulled in by an old jersey-json). However, the fact that they changed packages plays in our favor - we can add an explicit dep on the new jackson version and use the new one in this part of the code.
>
> Bill Farner wrote:
> I put together a proof of concept of this idea here: https://github.com/wfarner/aurora/commit/04e1ff6ad566755823b624303cfd66541076785b
>
> You'll notice that this makes the json more verbose, but it becomes a 1:1 association with the protobuf message.
>
> Renan DelValle wrote:
> Awesome, thanks for that. It's a WIP at the moment, but I just wanted to share what I have so far so I'm not going off on some trail I shouldn't be going on.
>
> I cribbed some of the code from your POC and came up with this. I'm still working on fixing all the tests that changing the ExecutorSettings broke, but after that it should be good to go.
>
> https://github.com/rdelval/aurora/commit/58e4524ab20698d6c665d7736dadbcac651d445e#diff-ca4de6b05130534619820a5f6a521f2d
> https://github.com/rdelval/aurora/commit/38958251833566589d448c9f3a19096201430605#diff-ca4de6b05130534619820a5f6a521f2d
>
> You'll notice I condensed the command list and URIs into a CommandBuilder and I changed the Volume from the Aurora one to the Mesos Protobuf one, I hope this last change isn't an issue, if it is, let me know.
>
> With these changes IMO, src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java is no longer needed, at least I haven't found anywhere where it would be a benefit to have it.
>
> Bill Farner wrote:
> The protobuf `Volume` object is the right thing to use. The most important thing i would suggest is that you remove `ExecutorSettings`, and let the protobuf go all the way to where it is used in `TaskInfo`.
>
> If CommandUtil is no longer used, it should indeed be removed.
>
> Renan DelValle wrote:
> The only thing that concerns me with doing that, is the amount of yak shaving involved in replacing ExecutorSettings everywhere that it's used in the tests. If you don't mind the patch ballooing in size a little bit, I also think this is the proper way to do it. Otherwise, we can leave a transformer for now and do this in a separate patch.
>
> Bill Farner wrote:
> To insulate somewhat from that, you could keep `ExecutorSettings`, but keep `ExecutorInfo.Builder` as a field inside it.
I attempted to do this by keeping ExecutorInfo.Builder but it made little sense the further I progressed. The MesosTaks building process takes care of building almost every single part of the ExecutorInfo, what it mostly needs is a CommandInfo. Furthermore, it's very simple to go from a ResourceSlot to the protobuf version of resources, but the reverse is not true. ResourceSlot is which is used in various calculations throught Aurora, including pre-emption, so getting rid of it in ExecutorSettings or rebuilding it to put it in ExecutorSettings is more work than necessary. Removing the resources from ExecutorInfo and placing them in a seperate data structure made me realize that, for now, it doesn't make sense to have ExecutorInfo as our main object.
Thus, I decided to go with CommandInfo as the main protobuf structure inside of ExecutorSettings. If any one feels strongly it's always up for change, but using CommandInfo as the main structure I hit way less speedbumps.
- Renan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 21, 2015, 11:42 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 21, 2015, 11:42 p.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. Jackson is used to deserialize a CommandInfo.Builder along with two other fields. Volumes are now based on the Protobuf version and not the Aurora Thrift version.
>
>
> Diffs
> -----
>
> .gitignore 86840972c53ea52a793968d3d00df6763a7d6ffb
> 3rdparty/python/requirements.txt 44217469a9583ec50233f34d54a32c105e6bab2c
> CHANGELOG 3b6990fb4dc3cfb8c730c81063413e505c803471
> NEWS a17f0e7c08fd30a0b2db6814a1c755111307228b
> README.md ef9c2f0ecf44b03e2025fe9cab62a3aa5ae30a77
> api/src/main/thrift/org/apache/aurora/gen/BUILD fe3f83b6a7680985dce01efe2d54ccc4b0c2c482
> api/src/main/thrift/org/apache/aurora/gen/api.thrift d740a90e7732f42b43a79f8cf0afe705c061539c
> api/src/main/thrift/org/apache/aurora/gen/internal_rpc.thrift a2c230fa9b5f648c4674042411cbe46fb8bb4faa
> api/src/main/thrift/org/apache/aurora/gen/storage.thrift b7c4665d51f0efeebabd58c978707397b45d275d
> api/src/main/thrift/org/apache/thermos/BUILD d0d789a6ee3971e3070f9397d53929563a77f7ea
> build-support/python/isort 44f9659948703c75372cd70643d5631acb116c2e
> build-support/python/isort-check 646cbf0bae4ccf8eac044817138ed1a7a59b261b
> build-support/python/make-mesos-native-egg fce53e42dde72b893f291db9d78af1dc47fafe2f
> build-support/python/make-pycharm-virtualenv 05a16d0d421982bcfb2e649be7b83a17d813efb1
> build-support/release/make-python-sdists 9608f68e16243da01434ce2fc7d61bb7c7efd712
> build.gradle 700e1ade1470b99e9be390db150cf73aa06f17bc
> buildSrc/gradle.properties 3231a7add9e0503f0ee6779ae3c4000143b174ab
> buildSrc/src/main/groovy/org/apache/aurora/build/CoverageReportCheck.groovy b996d4597492a76db0f2571db4e6c1ae9b4bcf33
> commons-args/src/main/java/org/apache/aurora/common/args/Arg.java PRE-CREATION
> commons-args/src/main/java/org/apache/aurora/common/args/ArgParser.java PRE-CREATION
> commons-args/src/main/java/org/apache/aurora/common/args/CmdLine.java PRE-CREATION
> commons-args/src/main/java/org/apache/aurora/common/args/NoParser.java PRE-CREATION
> commons-args/src/main/java/org/apache/aurora/common/args/Parser.java PRE-CREATION
> commons-args/src/main/java/org/apache/aurora/common/args/ParserOracle.java PRE-CREATION
> commons-args/src/main/java/org/apache/aurora/common/args/Positional.java PRE-CREATION
> commons-args/src/main/java/org/apache/aurora/common/args/Verifier.java PRE-CREATION
> commons-args/src/main/java/org/apache/aurora/common/args/VerifierFor.java PRE-CREATION
> commons-args/src/main/java/org/apache/aurora/common/args/apt/CmdLineProcessor.java PRE-CREATION
> commons-args/src/main/java/org/apache/aurora/common/args/apt/Configuration.java PRE-CREATION
> commons-args/src/main/resources/META-INF/services/javax.annotation.processing.Processor PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/application/Lifecycle.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/application/ShutdownRegistry.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/application/ShutdownStage.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/ArgFilters.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/ArgScanner.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/Args.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/ArgumentInfo.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/OptionInfo.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/Parsers.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/PositionalInfo.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/TypeUtil.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/Verifiers.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/CanExecute.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/CanExecuteFileVerifier.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/CanRead.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/CanReadFileVerifier.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/CanWrite.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/CanWriteFileVerifier.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/Exists.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/ExistsFileVerifier.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/IsDirectory.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/IsDirectoryFileVerifier.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/NotEmpty.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/NotEmptyIterableVerifier.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/NotEmptyStringVerifier.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/NotNegative.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/NotNegativeNumberVerifier.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/NotNull.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/NotNullVerifier.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/Positive.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/PositiveNumberVerifier.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/Range.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/constraints/RangeNumberVerifier.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/AmountParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/BooleanParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/ByteParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/CharacterParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/ClassParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/DateParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/DoubleParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/DurationParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/EnumParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/FileParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/FloatParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/InetSocketAddressParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/IntegerParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/ListParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/LongParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/MapParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/NonParameterizedTypeParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/NumberParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/PairParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/PatternParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/RangeParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/SetParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/ShortParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/StringParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/TypeParameterizedParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/URIParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/URLParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/args/parsers/UnitParser.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/base/Closure.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/base/Closures.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/base/Command.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/base/Commands.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/base/ExceptionalCommand.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/base/ExceptionalSupplier.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/base/MorePreconditions.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/collections/Iterables2.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/collections/Pair.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/inject/Bindings.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/inject/TimedInterceptor.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/io/Codec.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/logging/Glog.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/logging/LogFormatter.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/logging/RootLogConfig.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/logging/log4j/GlogLayout.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/net/InetSocketAddressHelper.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/net/http/handlers/AbortHandler.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/net/http/handlers/ContentionPrinter.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/net/http/handlers/HealthHandler.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/net/http/handlers/HttpServletRequestParams.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/net/http/handlers/LogConfig.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/net/http/handlers/QuitHandler.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/net/http/handlers/ThreadStackPrinter.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/net/http/handlers/TimeSeriesDataSource.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/net/http/handlers/VarsHandler.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/net/http/handlers/VarsJsonHandler.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/net/pool/Connection.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/net/pool/DynamicHostSet.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/quantity/Amount.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/quantity/Data.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/quantity/Time.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/quantity/Unit.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/JvmStats.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/Percentile.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/Rate.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/Ratio.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/RecordingStat.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/RecordingStatImpl.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/RequestStats.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/SampledStat.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/SlidingStats.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/Stat.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/StatRegistry.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/Stats.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/StatsProvider.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/TimeSeries.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/TimeSeriesRepository.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/stats/TimeSeriesRepositoryImpl.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/testing/TearDownTestCase.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/thrift/Util.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/util/BackoffHelper.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/util/BackoffStrategy.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/util/BuildInfo.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/util/Clock.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/util/Random.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/util/Sampler.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/util/StateMachine.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/util/TruncatedBinaryBackoff.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/util/templating/StringTemplateHelper.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/util/testing/FakeBuildInfo.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/util/testing/FakeClock.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/util/testing/FakeTicker.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/zookeeper/Candidate.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/zookeeper/CandidateImpl.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/zookeeper/Group.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/zookeeper/ServerSet.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/zookeeper/ServerSetImpl.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/zookeeper/ServerSets.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/zookeeper/SingletonService.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/zookeeper/ZooKeeperClient.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/zookeeper/ZooKeeperUtils.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/zookeeper/testing/BaseZooKeeperTest.java PRE-CREATION
> commons/src/main/java/org/apache/aurora/common/zookeeper/testing/ZooKeeperTestServer.java PRE-CREATION
> commons/src/main/resources/org/apache/aurora/common/net/http/handlers/logconfig.st PRE-CREATION
> commons/src/main/resources/org/apache/aurora/common/net/http/handlers/logprinter.st PRE-CREATION
> commons/src/main/resources/org/apache/aurora/common/net/http/handlers/thrift.st PRE-CREATION
> commons/src/main/thrift/org/apache/aurora/common/thrift/endpoint.thrift PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/args/ArgFiltersTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/args/ArgScannerTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/args/ArgTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/args/ArgsTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/args/OptionInfoTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/args/ParsersTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/args/argfilterstest/ArgsRoot.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/args/argfilterstest/subpackageA/ArgsA.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/args/argfilterstest/subpackageA/subsubpackage1/ArgsA1.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/args/argfilterstest/subpackageB/ArgsB.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/args/argfilterstest/subpackageBwithSuffix/ArgsBWithSuffix.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/base/ClosuresTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/base/MorePreconditionsTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/collections/Iterables2Test.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/collections/PairTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/inject/BindingsTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/logging/LogFormatterTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/logging/RootLogConfigTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/net/InetSocketAddressHelperTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/net/http/handlers/HttpServletRequestParamsTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/net/http/handlers/StatSupplierTestBase.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/net/http/handlers/TimeSeriesDataSourceTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/net/http/handlers/VarsHandlerTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/net/http/handlers/VarsJsonHandlerTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/quantity/AmountTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/stats/PercentileTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/stats/RateTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/stats/StatsTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/stats/TimeSeriesRepositoryImplTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/testing/easymock/EasyMockTestTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/util/BackoffHelperTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/util/SamplerTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/util/StateMachineTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/util/TruncatedBinaryBackoffTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/util/templating/StringTemplateHelperTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/util/testing/FakeClockTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/zookeeper/CandidateImplTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/zookeeper/GroupTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/zookeeper/ServerSetImplTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/zookeeper/ServerSetsTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/zookeeper/SingletonServiceTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/zookeeper/ZooKeeperClientTest.java PRE-CREATION
> commons/src/test/java/org/apache/aurora/common/zookeeper/ZooKeeperUtilsTest.java PRE-CREATION
> commons/src/test/resources/org/apache/aurora/common/util/templating/template.st PRE-CREATION
> config/checkstyle/checkstyle.xml 9c291f5cc4d70e6c920cbf033a2f5fee18f33229
> config/checkstyle/suppressions.xml PRE-CREATION
> config/findbugs/excludeFilter.xml e1c2503e0c6126c098063e0860f36cebbfe567ee
> config/legacy_untested_classes.txt f50b812459bd6b59d1a03e6814caa2daae133fe9
> config/pmd/custom.xml 8ad62280f69db0fe185be0005225cf2d65d62383
> debian/aurora-doc.docs a188e0692f3dd37b2e8ec2b3d3cfe74ce884d3e9
> debian/aurora-doc.examples e39721e20f0708703dc6d9b607634636d625e9b2
> debian/aurora-executor.dirs f62944bc706454accea767f7b774b9ebf30da8be
> debian/aurora-executor.install 8efb1308caf64a23bed4b580de4b86e7982539e8
> debian/aurora-executor.links 973c7e035c64560d6b8b01252bd7528103dfe438
> debian/aurora-executor.thermos.default 82ab0c8ab585acf253f8fecc0ed88a3e19511ffb
> debian/aurora-executor.thermos.init 24a59d22a93f38b2fa686cc9fee16fbd8548d2a0
> debian/aurora-executor.thermos.upstart ffa4f98ed41e69b448452abc330651394ac8cb95
> debian/aurora-scheduler.default 47fea5b4e6dadab9704c6ea5141268a91803fdc2
> debian/aurora-scheduler.init 2ed0159d36d4be08f2c355dc1c21f36192a819f4
> debian/aurora-scheduler.install 94ae7625d28e8a4df5a419decfd9e15e53e2eb6c
> debian/aurora-scheduler.links bed1c6599dedd734f34634140643bf30b1c263df
> debian/aurora-scheduler.postinst 0d019b6ecace73c69c833ff2267fe95e11cf2f8d
> debian/aurora-scheduler.upstart ed7fe0917147d9c36a0828cc701da902142f72ba
> debian/aurora-tools.install ac8d032d3c6f86ba8e9f73ea7bc712ebde7531f8
> debian/aurora-tools.links 2d69bc53c397485c368f190ca341502dcdb9277b
> debian/changelog 84464646dcd0e0998763af7301b85cdbca742edc
> debian/clusters.json 29c2e0b61c6a2e945152ea3a39cee883696de2c1
> debian/compat ec635144f60048986bc560c5576355344005e6e7
> debian/control 260ac8f42f47d604096282585c092e96448beb32
> debian/copyright 8c0d0f146bfa0e13d8de6e857cb42e94224649ef
> debian/pants.ini ed1a5ce8e8c3928085e4b84a882f3dcce02dadaa
> debian/rules 6ba18cef7fbf0989507d630a1041cdf958742617
> debian/source/format 89ae9db8f88b823b6a7eabf55e203658739da122
> docs/README.md 9893763cb75faf1aeb66ed905e9f696d4b532d16
> docs/build-system.md PRE-CREATION
> docs/client-commands.md f91e5df0c19104a961aa9955478bf2d37a2aa7b6
> docs/configuration-reference.md fb753ead94fcd1ed76352306391d356652934a4e
> docs/configuration-tutorial.md d6e7c352e16964e19a41340cb03a016620c17f7d
> docs/cron-jobs.md 2f36dc97daf65e39adb554cc74981f40a7d64220
> docs/deploying-aurora-scheduler.md 11155b9495b6e3ab59038b34e5e4deab59f7172d
> docs/developing-aurora-scheduler.md 17cfc183cfc5e9359df800e983d8bb45d70d4f9b
> docs/resource-isolation.md 7e8d88d09093d85c07c84bd3d6476fc89ff21c3b
> docs/sla.md 14e9108fda91200bbf56384c96b9cd926689311f
> docs/storage-config.md 971bc1673f8be800f0d081a44018fc17a5ed025a
> docs/test-resource-generation.md 335586d64757f1e6293a89f14c1c3d578321eac6
> docs/vagrant.md 7b796b2d70c9adf4a8b88bda625dcf70fdfb0f73
> examples/scheduler/scheduler-local.sh 6253d505f9851aea613bc95c15c313b03b57af11
> examples/vagrant/aurorabuild.sh 5eb171cf45ffee1287f3ac039ab8cf3db6991a97
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/provision-dev-cluster.sh 853ccac35f2befa31716a9ed686df650cd913c90
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 414539b1917b5d33c577f1539575934c7f7c8167
> examples/vagrant/upstart/aurora-scheduler.conf f4b867cbbcdbcc792518c2f90807834e47dce253
> examples/vagrant/upstart/mesos-slave.conf 2b6a60673fc0a7ea3b73471701cd5d3efd6ce639
> examples/vagrant/upstart/zookeeper.conf fbc0a7983fe0124f86d214ef82bc40184995db81
> gradle/wrapper/gradle-wrapper.properties e713793f7d63e65ea68e091b959ba6a68352b48f
> rbt c082ab4e0f541ff0d8c94719b027902ad1aacb17
> settings.gradle 6180227b91f62df67cc5a53f04d8a699a7967f81
> src/jmh/java/org/apache/aurora/benchmark/JobUpdates.java 48bee5068db5aeacf181c76c4d3add71d045c8a9
> src/jmh/java/org/apache/aurora/benchmark/Offers.java b8e6cb574a76f5105bea2216bcfcc48db0c2ba92
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java d9e5199862f9328ba812e54f231f1c9a43d54c24
> src/jmh/java/org/apache/aurora/benchmark/SnapshotBenchmarks.java 2bb17680eb08f70aa574e9f64622b74b2c1f323a
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 4c63cc7af2fb5702fa649ce2cd4ee619139223e2
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java d3e1295db9b933b5ab60d6446dfea453c6051795
> src/jmh/java/org/apache/aurora/benchmark/ThriftApiBenchmarks.java c3f8b251d80d86323e364c978e77b5615d1f9f65
> src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java cc4274103a8b4577fcbbbceac6d38496bac0c5cd
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java 45849b52ce85dd8eb1f28d4ab0386d2eafd9209d
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeRescheduleCalculator.java 6d710128a2723dc9d39bb2d0ca7544d10739a6c8
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeSchedulerDriver.java 23c0e53b1a708843a9722c8bf1e8f0b739a9f5eb
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeStatsProvider.java 3413b0a480073f39bc01a60bc93b211660d9e278
> src/main/java/org/apache/aurora/GuavaUtils.java 22104e6d005ae6706a06a21ffe4c730b14ecfe53
> src/main/java/org/apache/aurora/GuiceUtils.java 5d0af1ebdccd0d0782f0fbc53d046a338c450167
> src/main/java/org/apache/aurora/Protobufs.java 7f1789531bc7fc83dba6e19b39e74f3333a7232a
> src/main/java/org/apache/aurora/auth/UnsecureAuthModule.java 912e465793b42c81040a8cfe507c2e341932c205
> src/main/java/org/apache/aurora/auth/UnsecureSessionContext.java PRE-CREATION
> src/main/java/org/apache/aurora/codec/ThriftBinaryCodec.java 32a8a810ee55e83a2100a63021fbd773f636f2ca
> src/main/java/org/apache/aurora/scheduler/HostOffer.java 5056b60c32ab1c8f1b8d682be4827c489da2a104
> src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 1a158b4e0be94762ad0480e8ce74b19bacc90c97
> src/main/java/org/apache/aurora/scheduler/ResourceType.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/Resources.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/SchedulerLifecycle.java 32751896b43e7b0e92b1b9166f7606ab816425ef
> src/main/java/org/apache/aurora/scheduler/SchedulerModule.java 6edec22aba135136c4ce4066b9535f23de077db7
> src/main/java/org/apache/aurora/scheduler/SchedulerServicesModule.java ad2284ffb628eb9cc307ea6d9d2998f196cb162e
> src/main/java/org/apache/aurora/scheduler/TaskIdGenerator.java bcff437a5835c1a8002fab1d3ec2439ff6660af4
> src/main/java/org/apache/aurora/scheduler/TaskLauncher.java cd55a6ee7424873c3e615d95422c9ecab6442f46
> src/main/java/org/apache/aurora/scheduler/TaskStatusHandler.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/TaskVars.java 54d893e4bf02d4a49b445a0894b015e62deaf893
> src/main/java/org/apache/aurora/scheduler/TierInfo.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/TierManager.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/UserTaskLauncher.java 6bfbf0c76399d569ac762a7b433a6c576bf0ee87
> src/main/java/org/apache/aurora/scheduler/app/AppModule.java d2c172074092cd6e625a02646c9f12e37538b40b
> src/main/java/org/apache/aurora/scheduler/app/LifecycleModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/app/Log4jConfigurator.java 6df497e53d621964fdf163b4e2efe2674992b1a3
> src/main/java/org/apache/aurora/scheduler/app/Modules.java cad7c38ca62767093c23923ff83d7aefda6dc54f
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java c31446c447c3385a4763b8a516827988e46cc480
> src/main/java/org/apache/aurora/scheduler/app/ServiceDiscoveryModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/app/VolumeParser.java 2b7613741a729e7065bbe74690d543b45802c400
> src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java 8bcac6c4f93eaca7250d43ab81a72102b91bf836
> src/main/java/org/apache/aurora/scheduler/async/DelayExecutor.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/GatedWorkQueue.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/GatingDelayExecutor.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java f2ef70ddc5a859811f0e6c2ade62e115639c1654
> src/main/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPruner.java b4163435ea337a9976fae2f84850af0320ab9884
> src/main/java/org/apache/aurora/scheduler/async/KillRetry.java b125c1c81309c521219f856c21a590f6e574c31c
> src/main/java/org/apache/aurora/scheduler/async/OfferManager.java e60d01ed2de2a51dea3adb2682fe480fc623e002
> src/main/java/org/apache/aurora/scheduler/async/RandomJitterReturnDelay.java 6a8c96791b21e1d8a91d5f4ada71486f81d3e275
> src/main/java/org/apache/aurora/scheduler/async/RescheduleCalculator.java 6a0c0a9b933546c1581d3e29ba53eb86c58fbc39
> src/main/java/org/apache/aurora/scheduler/async/TaskGroup.java 635419b2558f646cd46650013d9582e9f404582c
> src/main/java/org/apache/aurora/scheduler/async/TaskGroups.java 1580404a36bbab6b612b8188c4a649d4a77118e0
> src/main/java/org/apache/aurora/scheduler/async/TaskHistoryPruner.java 7b6c0639fba6a322529f17c9a69c3b28ea7d2432
> src/main/java/org/apache/aurora/scheduler/async/TaskReconciler.java ccdbc029f3ce2ef51b777bfc06ee31ba78254d8a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java a500e55cb52fe89917d8499a42e2a2eb5032c109
> src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java f0dea48b3df69dedf620a55573731ed9451369c3
> src/main/java/org/apache/aurora/scheduler/async/TaskTimeout.java e250f3330504c3c3737564174ecc2943cb7dd93b
> src/main/java/org/apache/aurora/scheduler/async/preemptor/BiCache.java f5a18338748da7c443b2fd2ec3a72adf75e7387f
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterState.java 38610b21c5404dfcd938a2541c0f407a8fd259d2
> src/main/java/org/apache/aurora/scheduler/async/preemptor/ClusterStateImpl.java cd016af5621be76190ead81921096aae837b59c0
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PendingTaskProcessor.java c1114a411c32c25785adc93e594ee8e291025969
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptionProposal.java 7a03168ab6997a9ad13ec3692249a2ac75f89bd6
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptionVictim.java f196b212d2b2562e941100d17364b2442bce058f
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptionVictimFilter.java 75e23706df954b0cec77e7afe6c3f7dbfc258162
> src/main/java/org/apache/aurora/scheduler/async/preemptor/Preemptor.java 41591b80ee6d078e1c2beb5d98c7a7584ce173e2
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorMetrics.java 22a15330970a3f52b0c0b5307a2b90339f1c1d2e
> src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorModule.java 3d9e27b673c10ccbb78a3f535d991a3f5e5433fa
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java 7f166e2ec24f66a9ab0c35e3c65af45e461943f9
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java a5ffa5e95b301e536a84acf02817ea0c080559d0
> src/main/java/org/apache/aurora/scheduler/base/Query.java 458530fb216fd37048d3e398c07b147a1a71cf57
> src/main/java/org/apache/aurora/scheduler/base/ResourceAggregates.java 5ae27f9338c1e77c8a901a329c84d4a1433e682f
> src/main/java/org/apache/aurora/scheduler/base/TaskGroupKey.java 47e4d48ab4c82b2770a6817666dbd24c3e387b48
> src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 2f428513305e147b1b33fd5868b9ede8ae72ebb8
> src/main/java/org/apache/aurora/scheduler/base/Tasks.java 52d37e255edbabc9874591b7a4cb3e096ac51ecc
> src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java b777777b0ebbf303778e528b16ff3db1aa4e76f1
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/configuration/Resources.java 673a6909ec1bccbc43b0a3cd913aa48b9b1d90c1
> src/main/java/org/apache/aurora/scheduler/cron/CrontabEntry.java 904dd72dbb9036d8c06353a953bf35b8f04cfdbc
> src/main/java/org/apache/aurora/scheduler/cron/SanitizedCronJob.java 46e89cd51750b92a1e4afcdb03cba0ff22a24109
> src/main/java/org/apache/aurora/scheduler/cron/quartz/AuroraCronJob.java df180a4f9dff75a04ae104462c219444ef605d9a
> src/main/java/org/apache/aurora/scheduler/cron/quartz/CronJobManagerImpl.java 770672c85c06aaf4356b3b3580428b0323edb9d3
> src/main/java/org/apache/aurora/scheduler/cron/quartz/CronLifecycle.java ec524375ba11d13e57a49e1e4d7c7e4f7a009c94
> src/main/java/org/apache/aurora/scheduler/cron/quartz/CronModule.java 89e525898d4e412ecfc2f7ee45f3cc26e4e1a709
> src/main/java/org/apache/aurora/scheduler/cron/quartz/CronPredictorImpl.java d362a1e2dc87b94b3b7ca140eb6a144b48a8f9de
> src/main/java/org/apache/aurora/scheduler/cron/quartz/CronSchedulerImpl.java 1ec8ddf68751697f957ea0b3c7f4ff6034e0e58a
> src/main/java/org/apache/aurora/scheduler/cron/quartz/Quartz.java 45e062d3acbd7b8565bd3773c1e994aae96378e0
> src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java 0063aea16ff39e95c8d32848fdb3eeec6ab1bee3
> src/main/java/org/apache/aurora/scheduler/events/PubsubEventModule.java 82d479e142afe798cd84135b981075035d9ca6dc
> src/main/java/org/apache/aurora/scheduler/filter/AttributeAggregate.java bd74f892dde65e957f125a17979ee6c582b3043f
> src/main/java/org/apache/aurora/scheduler/filter/ConstraintMatcher.java ecba276f1882162d9578e0829b2139040876c7d0
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 260e11e910e97c1057351c206f0f763c04d4502a
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 54ffd8e3e76527e39c55ac735c3d1e71b317cbf4
> src/main/java/org/apache/aurora/scheduler/http/CorsFilter.java e413ad9bdccc329777b3d0764ba8684539956679
> src/main/java/org/apache/aurora/scheduler/http/Cron.java fd658e1ac5e73ef0c61c82685f6373c5f0eb6640
> src/main/java/org/apache/aurora/scheduler/http/H2ConsoleModule.java a44ea08950b9d4b33503f4cfc6e28a32c619b3c8
> src/main/java/org/apache/aurora/scheduler/http/HttpStatsFilter.java 4c2b62b895338034089be18bc7f3a746dd82a48b
> src/main/java/org/apache/aurora/scheduler/http/JerseyTemplateServlet.java 96aec089e6f1ece699e23f3b230ef3ceee32bb7e
> src/main/java/org/apache/aurora/scheduler/http/JettyServerModule.java b4701295922ddad44d9d5c9fa29eb95e195e1604
> src/main/java/org/apache/aurora/scheduler/http/LeaderRedirect.java fcf5e4419b9908281858e591c916c586c707c6cd
> src/main/java/org/apache/aurora/scheduler/http/LeaderRedirectFilter.java 1c2b08de1e2c470d32baeb845ae0e0a7ce2b75aa
> src/main/java/org/apache/aurora/scheduler/http/Maintenance.java 303f05cbec7a7134b39ad1d15af3ee5e7a793e0c
> src/main/java/org/apache/aurora/scheduler/http/Mname.java d735cbc4169dbf3f7fb8dd1c13bf683ddcce548c
> src/main/java/org/apache/aurora/scheduler/http/Offers.java b991616a5921fdfa42cc47935d2a6219e14041ed
> src/main/java/org/apache/aurora/scheduler/http/PendingTasks.java 82b6f5043146bc47927e4b9aefe314ce5c9371cf
> src/main/java/org/apache/aurora/scheduler/http/QuitCallback.java 2971871386b930d77dae44def3b212b4b3492703
> src/main/java/org/apache/aurora/scheduler/http/RequestLogger.java 8a0429c2ce16a477cf61d373e1a033a173030e4e
> src/main/java/org/apache/aurora/scheduler/http/Slaves.java b64e18c4d5278e6cdb2e480043a45e6ec4f87484
> src/main/java/org/apache/aurora/scheduler/http/StructDump.java 05434f8bfdd8165ab8a333b15301e1f0f46578dd
> src/main/java/org/apache/aurora/scheduler/http/Utilization.java 10ded0a655295fa5dea491acc080f36f952e1370
> src/main/java/org/apache/aurora/scheduler/http/api/ApiModule.java 63c31eeb293771b6cc0a3e4cc62dd3a94853e727
> src/main/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityModule.java 1e9b1c3b6e256fd994e94ad30d9340d12ce15f99
> src/main/java/org/apache/aurora/scheduler/http/api/security/IniShiroRealmModule.java f4decde73b29e6fbf24c8a44549af469ce348be5
> src/main/java/org/apache/aurora/scheduler/http/api/security/Kerberos5Realm.java b224983159c4979800fa384bbb082adc7b330b4c
> src/main/java/org/apache/aurora/scheduler/http/api/security/Kerberos5ShiroRealmModule.java 4ec531d3db371a7f259fdc35ebfa569292e652c0
> src/main/java/org/apache/aurora/scheduler/http/api/security/KerberosPrincipalParser.java e6e2f1e09ec57edace75956945e04d8a430b7a10
> src/main/java/org/apache/aurora/scheduler/http/api/security/ModuleParser.java 9077827a9bf9dd9118c0cea1bd4e8e5a9d6cc18e
> src/main/java/org/apache/aurora/scheduler/http/api/security/ShiroAuthorizingInterceptor.java 98bc4341eb7005c52714aa115515d4a11c480e27
> src/main/java/org/apache/aurora/scheduler/http/api/security/ShiroAuthorizingParamInterceptor.java 7425b93f476ca1228a233a56363136f9e586a5e5
> src/main/java/org/apache/aurora/scheduler/http/api/security/ShiroIniParser.java ff8063c050e13b4bffda2661a817fdc023b80867
> src/main/java/org/apache/aurora/scheduler/log/mesos/MesosLog.java 5a24cc5f566f245e286e8386900f9bbe8b2b7f2d
> src/main/java/org/apache/aurora/scheduler/log/mesos/MesosLogStreamModule.java 7232f603e21bbc9dbb5e05aedd5e493de519158e
> src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java 54608c8f819b65186ea0d000fc01b9538d5cb4ee
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java 325f55640648151ae19e0c18c6961aeff10bfac3
> src/main/java/org/apache/aurora/scheduler/mesos/MesosSchedulerImpl.java f233d5a181bb1f43fbbfe657dbda2cf975aa6895
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java e934f570e4a728470408970485abe0809487d312
> src/main/java/org/apache/aurora/scheduler/mesos/SchedulerDriverService.java 5567fe0c7c94cc91fecf87a23e910be49e4ad746
> src/main/java/org/apache/aurora/scheduler/mesos/TaskStatusStats.java 4d31b62e9cd201ece6f586e1e3b4b0ef1cece459
> src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java 2b194e5a761cb193a2b72419db1a1120fb52c9fe
> src/main/java/org/apache/aurora/scheduler/offers/OffersModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictim.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/pruning/PruningModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/quota/QuotaCheckResult.java 5bf4d9a659942c65290442fdcd4749bda046bfd8
> src/main/java/org/apache/aurora/scheduler/quota/QuotaInfo.java 3e258127f452070214d5e9df9da25f5b38d0d886
> src/main/java/org/apache/aurora/scheduler/quota/QuotaManager.java 7453680af5a518012f9779f82d0349e897c04994
> src/main/java/org/apache/aurora/scheduler/reconciliation/ReconciliationModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/scheduling/SchedulingModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/sla/MetricCalculator.java 82f36d5ca15df18bdc8ebbbd868d3394db38e603
> src/main/java/org/apache/aurora/scheduler/sla/SlaAlgorithm.java ff73ca6265bd0699791da5e5b6ed4aab9156d9e4
> src/main/java/org/apache/aurora/scheduler/sla/SlaGroup.java a1d6c9d6857d2bacfcc7e4719ef2b42b66a9b470
> src/main/java/org/apache/aurora/scheduler/sla/SlaModule.java 64e986fb2e0f955dd4a9c7824eac9494728bf24e
> src/main/java/org/apache/aurora/scheduler/spi/Permissions.java b9dff4471fe6bc6b793f7fb15de3a5c2e2d39209
> src/main/java/org/apache/aurora/scheduler/state/LockManager.java e8303f914466ebe93518ec2970e1173dc39d9adb
> src/main/java/org/apache/aurora/scheduler/state/LockManagerImpl.java 6aa281cb7b07e3e82c578c553eee8c7066cf2761
> src/main/java/org/apache/aurora/scheduler/state/MaintenanceController.java a6d7ab748aeb050f93f817e3b084b03d34a58d9d
> src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java 1b8733bff69aef5a7db6bc7d137932a69717275a
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 3acb45a9ffd88ccff0817acde602e133bf56ac4e
> src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java 48d0ff667cf97002795b97c235b9e9e34f8e5018
> src/main/java/org/apache/aurora/scheduler/state/TransitionResult.java 6928c663d9f3f46ab81816d3d0ef4e7d0485e368
> src/main/java/org/apache/aurora/scheduler/stats/AsyncStatsModule.java 825e7727d965d7d9e023da09d38e71586cc2b77f
> src/main/java/org/apache/aurora/scheduler/stats/CachedCounters.java 4a3bed331e43ed4430d4833c2f3b72000edfd859
> src/main/java/org/apache/aurora/scheduler/stats/ResourceCounter.java e5c0322007c8d9424b96016c9075790c69cf2cf5
> src/main/java/org/apache/aurora/scheduler/stats/SlotSizeCounter.java 5de34b2fb2d6fa6926021092d2fc4b7a06c2ccbc
> src/main/java/org/apache/aurora/scheduler/stats/StatsModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/AttributeStore.java 97f9ee81ed18a961e6df4916d9ede1d89f90ded1
> src/main/java/org/apache/aurora/scheduler/storage/CallOrderEnforcingStorage.java 64aa10d15fb2caa8d5be08ed2b57cd2f6e7e1aa2
> src/main/java/org/apache/aurora/scheduler/storage/ForwardingStore.java 5242eba2e1a5b35ba3d321f96e2c6670afed4c11
> src/main/java/org/apache/aurora/scheduler/storage/LockStore.java 596a3787369616d30a2763a37bcb7b7cc39ad386
> src/main/java/org/apache/aurora/scheduler/storage/Storage.java 21f6a642df48b66614e210be4a9a99d3cd823501
> src/main/java/org/apache/aurora/scheduler/storage/StorageBackfill.java a82fbb5f3e28a8e3a757268ac7e768403b29eda7
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 2768e6eafa51f6df78073715a7caf10bb7adba25
> src/main/java/org/apache/aurora/scheduler/storage/backup/BackupModule.java a6b187d888726b921733a36fcaecdab97bdef094
> src/main/java/org/apache/aurora/scheduler/storage/backup/Recovery.java fb0dbaec0f4cbd1ad12b808d795299b736552c6b
> src/main/java/org/apache/aurora/scheduler/storage/backup/StorageBackup.java 569648aea2ccdb57663d16b7c837fd2677694419
> src/main/java/org/apache/aurora/scheduler/storage/backup/TemporaryStorage.java d0156d330deb5d862fa54513fdbb6561d1c91548
> src/main/java/org/apache/aurora/scheduler/storage/db/AttributeMapper.java 3763f4dbd1b83b83c4b19a89784343ef9de100f1
> src/main/java/org/apache/aurora/scheduler/storage/db/CronJobMapper.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/db/DbAttributeStore.java 5f1cd2b7daec40a8a57095e16390698dbb0bcd24
> src/main/java/org/apache/aurora/scheduler/storage/db/DbCronJobStore.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStore.java 11c9c4ada400d51fc83e9e0de03108456be15fdf
> src/main/java/org/apache/aurora/scheduler/storage/db/DbLockStore.java 335d7a95e797fe940e71b10da44cbd97edea69ac
> src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java 2dc3034a5e4389588b5f796ff8dfb06dbc9939b8
> src/main/java/org/apache/aurora/scheduler/storage/db/DbQuotaStore.java 9371907374ffaf1082d8e9d224260d9a67ae1fb7
> src/main/java/org/apache/aurora/scheduler/storage/db/DbSchedulerStore.java d7696cd3956fe297490dd8a46ebd5731e32eb96f
> src/main/java/org/apache/aurora/scheduler/storage/db/DbStorage.java bb61542f4043847b1c8c92ff1b4a0ecfcfc88973
> src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 76f65dab5a5cb8d97ada962a06a7d0e191739119
> src/main/java/org/apache/aurora/scheduler/storage/db/DbUtil.java 3a86614767cf0435add237901b12d8108be09970
> src/main/java/org/apache/aurora/scheduler/storage/db/GarbageCollectedTableMapper.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/db/InsertResult.java 44dc8f5e3dcc91e80a03d980c5d8ae0db65c8b89
> src/main/java/org/apache/aurora/scheduler/storage/db/JobKeyMapper.java afdaa49d6cd5c135f6e4ddda2b6a45d189560e09
> src/main/java/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.java 02ea355539d6f56c1c861ac83293b649285f43f4
> src/main/java/org/apache/aurora/scheduler/storage/db/LockKeyMapper.java 5e9ba823d08d2b46342e7722c9df5f2a349c97cf
> src/main/java/org/apache/aurora/scheduler/storage/db/RowGarbageCollector.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/db/TaskConfigManager.java 3ada6286e6ef6e3302802b74eec6c46dd582dc10
> src/main/java/org/apache/aurora/scheduler/storage/db/TaskConfigMapper.java 7ee001f9c019a1e7b669ae5cec6088bf974a3746
> src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 9903675c9d5ee34aebbc52c85fbf02929422a3bf
> src/main/java/org/apache/aurora/scheduler/storage/db/shims/ContainerShim.java 07a991d34811256d5ef5504c432fc2b0973ca53f
> src/main/java/org/apache/aurora/scheduler/storage/db/shims/TaskConstraintShim.java 4990af7ce409eb5de20af1a9535bb30abe39ed45
> src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/CronCollisionPolicyTypeHandler.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TaskConfigTypeHandler.java 2f58357dd63c46eb69de6a6daedcf027712d7b2e
> src/main/java/org/apache/aurora/scheduler/storage/db/typehandlers/TypeHandlers.java 0a519be65f90cb730f6bda1e6d7b019f0f15252b
> src/main/java/org/apache/aurora/scheduler/storage/db/views/AssignedPort.java 0c6442c90289741aa4c59925a6d7a9e4050bf657
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbAssignedTask.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbConstraint.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobConfiguration.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdate.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdateDetails.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbJobUpdateInstructions.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbScheduledTask.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbStoredJobUpdateDetails.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/db/views/DbTaskConfig.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/db/views/Pairs.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/storage/db/views/ScheduledTaskWrapper.java b89e7b5463cdb9ff9d1f9106dda0c0c4908225ca
> src/main/java/org/apache/aurora/scheduler/storage/db/views/TaskConfigRow.java 0160ae3049db555cae9e3d3705c1f0daaeef53d4
> src/main/java/org/apache/aurora/scheduler/storage/db/views/TaskLink.java 52b09a905006bed2a9f01b8beb72772273b5630f
> src/main/java/org/apache/aurora/scheduler/storage/log/EntrySerializer.java 40c510a308ef06b211b3d4348309e325b650ab4e
> src/main/java/org/apache/aurora/scheduler/storage/log/LogStorage.java 82be3670338bfd976cf4ea0d93632c93703da016
> src/main/java/org/apache/aurora/scheduler/storage/log/LogStorageModule.java e5157709149b629f3576f1baaf536f62da338704
> src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotDeduplicator.java 620b6b6f6a99f5f77feaf610fdb8df30e019e3d4
> src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java 64b5b9548506b25fecce5113341f25b35b0f0e47
> src/main/java/org/apache/aurora/scheduler/storage/log/StreamManager.java 9047682e633613008f74a889688c5905e5e984d6
> src/main/java/org/apache/aurora/scheduler/storage/log/StreamManagerImpl.java 12da352a749fe5d974b0cd69eda0b694b0a90bf4
> src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java 0b5530478b6b43b87dbf095a3b4c53dc33eaddcf
> src/main/java/org/apache/aurora/scheduler/storage/mem/InMemStoresModule.java 35c83b9b8838a00693c8ebc96e496886ca249ed1
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemJobStore.java f9e9e89a52c9bce3dd7e5a727498a7c87c26a56c
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 3a9de60d7e743634815b83587bd61814d883bf86
> src/main/java/org/apache/aurora/scheduler/testing/FakeStatsProvider.java 8c0a5e65b5733b0c87fa6683a68860f3f0954b7c
> src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java 30e579c46168b64bdc4ba2416d49cf34b4e5c5f6
> src/main/java/org/apache/aurora/scheduler/thrift/Responses.java ffca661e20a9aedb2699a9dd64267375283500ac
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 0670b2b4c3817becefb38f0294f111768386817b
> src/main/java/org/apache/aurora/scheduler/thrift/aop/AopModule.java 3490394731fe46ccd6691530c03d6b6f9b817573
> src/main/java/org/apache/aurora/scheduler/thrift/aop/LoggingInterceptor.java d29cbaecdfd9c05214d5ea66f9ff63a06fbffe86
> src/main/java/org/apache/aurora/scheduler/thrift/aop/ThriftStatsExporterInterceptor.java 9b82e4fdcfc4439d06c6b698a60cdf48cb3958c0
> src/main/java/org/apache/aurora/scheduler/thrift/auth/ThriftAuthModule.java 3893e3dd397aa64252179e83fd38f2f388d406fe
> src/main/java/org/apache/aurora/scheduler/updater/InstanceAction.java b553f97662119b42835ad646b5d187c5550ee516
> src/main/java/org/apache/aurora/scheduler/updater/InstanceActionHandler.java 3b9919d635af3304960c8d510ffcfe764a692e07
> src/main/java/org/apache/aurora/scheduler/updater/InstanceUpdater.java 722dba4f3dc98efeed7128bfa1f2ae02f4670972
> src/main/java/org/apache/aurora/scheduler/updater/JobDiff.java 88a25e67baaa04449bc06e43624747b47616e0be
> src/main/java/org/apache/aurora/scheduler/updater/JobUpdateControllerImpl.java dc085879d14395e6de76dfe03fbc296756f5d880
> src/main/java/org/apache/aurora/scheduler/updater/JobUpdateEventSubscriber.java 134cd5429512338d0a464b1f3ffa557ca99d5713
> src/main/java/org/apache/aurora/scheduler/updater/OneWayJobUpdater.java fc88adb543154f1bad9a47c97657e2dfa29904e9
> src/main/java/org/apache/aurora/scheduler/updater/SideEffect.java 3f4924266e863587fcc8c44df9cb73b6e1e29259
> src/main/java/org/apache/aurora/scheduler/updater/StateEvaluator.java 6815f67ade1446b1ca0d7a13df4ef2d014d59a3f
> src/main/java/org/apache/aurora/scheduler/updater/UpdateFactory.java 927a7d131a385702e25947b6a87170f6302aa983
> src/main/java/org/apache/aurora/scheduler/updater/strategy/BatchStrategy.java 67d595bd81c2e90178b5b9179974190a8ae17322
> src/main/java/org/apache/aurora/scheduler/zookeeper/guice/client/ZooKeeperClientModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/zookeeper/guice/client/flagged/FlaggedClientConfig.java PRE-CREATION
> src/main/python/apache/aurora/admin/BUILD 22bf3f9943ea11258ba681bdb80feb00206bb926
> src/main/python/apache/aurora/admin/admin.py d2f7bd13ef335bdca7dc02177c031ac36574658d
> src/main/python/apache/aurora/admin/aurora_admin.py 1c39717b656465bb1966073340f9fe80be01a085
> src/main/python/apache/aurora/admin/host_maintenance.py c31e2a6fb0348c5e7074e46d61c2d2723afaf14d
> src/main/python/apache/aurora/admin/maintenance.py 27d63f2340b9ee7e66e0f9ba8ce17de529be923b
> src/main/python/apache/aurora/client/BUILD e73cd52289209bb9658b16bb77dc0b0a9c811a1a
> src/main/python/apache/aurora/client/api/BUILD d71cc31b2bb86d1a7321e78cd7716a32ff47e47d
> src/main/python/apache/aurora/client/api/command_runner.py c7238e274e53138187c2fe6fe5f14b7ae5f43ba2
> src/main/python/apache/aurora/client/api/quota_check.py 75406ac1d3ec9ba655daf7c6125f208f74290cfb
> src/main/python/apache/aurora/client/api/scheduler_client.py 2b8047c70d2d5bbe6d607437e2f4c9ba161e0cab
> src/main/python/apache/aurora/client/base.py df53b9036e609726bc2d2e17fe2e66a2e1c7f001
> src/main/python/apache/aurora/client/cli/BUILD 973d05971bd940c7e38c48ce6bfbf5c8e1654c5a
> src/main/python/apache/aurora/client/cli/client.py c0974f3bef59f8f7c2320398d367cb4dd9048d2b
> src/main/python/apache/aurora/client/cli/context.py c850223588df9d9be4a644aa51adc6419a751af0
> src/main/python/apache/aurora/client/cli/jobs.py 5f608419e1235f14836865b46be05d13963deb37
> src/main/python/apache/aurora/client/cli/options.py f8ac25287e296bcec89d3701738c7dd06ffae133
> src/main/python/apache/aurora/client/cli/quota.py e8aa010f5ecce0464a8ad6b072bccba589fe16d7
> src/main/python/apache/aurora/client/cli/sla.py 0274988d31dbe1386e8a62a005a1408a42043e45
> src/main/python/apache/aurora/client/cli/task.py b541d03d6e34dce7f4f1e38e2d970c71c8d94e08
> src/main/python/apache/aurora/client/cli/update.py 70f6c69aa9226981c66c0f986fd53840c56d210f
> src/main/python/apache/aurora/client/config.py 59703ef18c61dbed635954e05a38385ac364b679
> src/main/python/apache/aurora/client/hooks/BUILD ddf813b5d2e7d63507a8e08745ebf6cb3dbac8e1
> src/main/python/apache/aurora/common/BUILD abc122b0775bb17b1df67bdb946c472010219b9b
> src/main/python/apache/aurora/common/auth/BUILD 0abac94eeeb71e6af43ed191ea690e5f96a6be23
> src/main/python/apache/aurora/common/auth/kerberos.py 2d782b63b611a9d2604ef0ab1116d3e68fb86dc7
> src/main/python/apache/aurora/common/http_signaler.py 531f1fecbcbf8015175c7f1cb1e1c4d3e7d1268a
> src/main/python/apache/aurora/config/BUILD 0a3a93fa6bb785903f71bc067aecc79c0e45a0b5
> src/main/python/apache/aurora/config/__init__.py dd2f89014a3da730364b14e01c499ac0f2c288c1
> src/main/python/apache/aurora/config/loader.py c8b045e9a668199feee0ac3cee9fac55fe09cf08
> src/main/python/apache/aurora/config/schema/BUILD 171f42a24ec7ddc2846eb68b6a60e7d8dec4383b
> src/main/python/apache/aurora/config/schema/base.py ec9f983564516afe542ab277d987d4d391f87e45
> src/main/python/apache/aurora/config/thrift.py 810febb637d168b07c4aea77984e1d1451a39af2
> src/main/python/apache/aurora/executor/BUILD cbb2f5f7b5daa936db71cf8c0aac8ddb2002060b
> src/main/python/apache/aurora/executor/aurora_executor.py 7ad179e632a164df98109ca7cc3098751533e70d
> src/main/python/apache/aurora/executor/bin/BUILD 0fbb0f88881ee63499d9ce36150ae5e68fcc8a9e
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py 8093717266f8620ebc6ef4c028ac8c87ab8d22be
> src/main/python/apache/aurora/executor/bin/thermos_executor_main.py b3e8bf1e924999306e0b8a1314273b22c51028e7
> src/main/python/apache/aurora/executor/common/BUILD 3d94792b9e3c2d0f7bb7ce42616f08947f5119ee
> src/main/python/apache/aurora/executor/common/path_detector.py ba85d5c969d3f1f62d3f898bed83d930e2994bd2
> src/main/python/apache/aurora/executor/common/task_info.py d110faf08135d94d9af95ad74613950c56248c09
> src/main/python/apache/aurora/executor/gc_executor.py d4392faf50f8c72f08f951962913248045d7fcb5
> src/main/python/apache/aurora/executor/http_lifecycle.py PRE-CREATION
> src/main/python/apache/aurora/executor/thermos_task_runner.py 837ad5c55b4a02215a8d509c211e80529536f705
> src/main/python/apache/aurora/kerberos/BUILD PRE-CREATION
> src/main/python/apache/aurora/tools/BUILD 9dfeb6c7e5a3796c76250aa9ff6fd4230d00cfb0
> src/main/python/apache/aurora/tools/java/thrift_wrapper_codegen.py 7d8f848cc8a6fa9669a8a0377590f05b91985064
> src/main/python/apache/aurora/tools/thermos_observer.py 4b534d3b0ccb78ddbe2f8d66eba0c2560ea48474
> src/main/python/apache/thermos/BUILD 8221aa0bd4efe5f519550cba716d6a564ba9ae44
> src/main/python/apache/thermos/bin/BUILD 4f0cb9b67b1c3c52ec3278ce92ff0d765257796c
> src/main/python/apache/thermos/bin/__init__.py
> src/main/python/apache/thermos/bin/thermos_ckpt.py 8ed69e0440b6aec85c5fa9e138215b592e9adcb1
> src/main/python/apache/thermos/bin/thermos_runner.py
> src/main/python/apache/thermos/cli/BUILD 2b9bc6245718397e4cb1ba104194b167434a3112
> src/main/python/apache/thermos/cli/bin/BUILD f33c7f838a6a0932abc737d0ecf7ca3a59580e2e
> src/main/python/apache/thermos/cli/bin/thermos.py fcef38d33e7ca2005f35c3bdb90ffee6aeade3af
> src/main/python/apache/thermos/cli/commands/BUILD 1dae8c981bd750807ddd1b6071e232ff2697537d
> src/main/python/apache/thermos/cli/commands/gc.py 23d9ff4d2048b4f2d80ea62c54e58e8d768e11c0
> src/main/python/apache/thermos/cli/main.py f20f612790550b77ee3dc969c37317b014a64972
> src/main/python/apache/thermos/common/BUILD 942fc15f5a5d6a5ff58385b10f3783ac476a4f82
> src/main/python/apache/thermos/config/BUILD d9099c5806eadb1190d2028142a7ec711023d39f
> src/main/python/apache/thermos/config/bin/config_load.py d6e1f820ac6b0fa4e47e26f99df1602a2d3d021e
> src/main/python/apache/thermos/config/bin/config_repl.py ae9ca3b2812f719b0a6185081434d557492ac825
> src/main/python/apache/thermos/config/dsl.py
> src/main/python/apache/thermos/config/loader.py d77ab9a52b16e9d65acdb95f01fd251ae8ab2b6e
> src/main/python/apache/thermos/config/schema_helpers.py d88792a67928e23297db918cc62b9c7ee5497432
> src/main/python/apache/thermos/core/BUILD efb68e84cf547cb9505a8caf5b47be394dee5145
> src/main/python/apache/thermos/core/helper.py 8cd32948663a5d5a1e975e1661b78de701710436
> src/main/python/apache/thermos/core/inspector.py 4fe8aa31215a12b9a53e885697b4dd4e78c1f35f
> src/main/python/apache/thermos/core/process.py 5ce138dab161d880c0bd58b87a6f5a54d4ca2f99
> src/main/python/apache/thermos/monitoring/BUILD 633dd95f9d193b1f377ab5d6cdfcdca7bdaa610f
> src/main/python/apache/thermos/monitoring/garbage.py aa5a2729ae6c94b6a270d97425767ccee121e588
> src/main/python/apache/thermos/monitoring/resource.py c1d7804fe0896d8b92aa565d6dd5295fa58efd3f
> src/main/python/apache/thermos/observer/BUILD 41db2cc2e11234c434f58f55abec0b9f308096be
> src/main/python/apache/thermos/observer/bin/BUILD 0abe2ccfe9c5ccb509ad731125385900114ba352
> src/main/python/apache/thermos/observer/bin/__init__.py
> src/main/python/apache/thermos/observer/bin/thermos_observer.py 39d3994a6163746e853cd21fc4c3585edc2b54cb
> src/main/python/apache/thermos/observer/http/BUILD 0bd770453dc78b043c9e6171dc7439da19c5872e
> src/main/python/apache/thermos/observer/http/file_browser.py 1750f5bd0937f8ce411f976db488bf5d14930577
> src/main/python/apache/thermos/observer/task_observer.py b9a37de0f6260dc8c0a6ba92119c57e16f29d82c
> src/main/python/apache/thermos/runner/__init__.py PRE-CREATION
> src/main/python/apache/thermos/testing/BUILD 8b5f6dc93c95e2f69d1b755e93b5f24dec0ead30
> src/main/resources/apache/aurora/client/cli/.auroraversion 5ea35de7ada4edef3189e0b01b322d50e8f8a638
> src/main/resources/org/apache/aurora/scheduler/storage/db/AttributeMapper.xml d36b42a22ebf9bbe52e60fea9b8a2677c41bd598
> src/main/resources/org/apache/aurora/scheduler/storage/db/CronJobMapper.xml PRE-CREATION
> src/main/resources/org/apache/aurora/scheduler/storage/db/JobKeyMapper.xml f5829ac063272123995193caef5151e0d52d435b
> src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 6ffb54f5beef332097fd5c2390b56873a82f0aa9
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskConfigMapper.xml 8258fb102b7f5fca9635143ebaed542d43abeb9f
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml c3ab3eb9bd6289f73db13e7a7978b5d47621bb52
> src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql 24cf52680b69e23f5ccbbcada0606975b0405d5b
> src/main/resources/scheduler/assets/breadcrumb.html 13147935c3c28680d5decd1e03e40d9cf04aaed3
> src/main/resources/scheduler/assets/css/app.css ba26accceb1da7dcc89739ceb4524ab6b5c4a11d
> src/main/resources/scheduler/assets/home.html fadf00d81b8f9b62597cca24396b27c42fb9c89b
> src/main/resources/scheduler/assets/instance.html PRE-CREATION
> src/main/resources/scheduler/assets/job.html 0c8fd54d7d11318fc0dd2bdd56769dfb3ead9849
> src/main/resources/scheduler/assets/js/app.js b66409f628046c67b62c92a75cbeed20c09b4ec1
> src/main/resources/scheduler/assets/js/controllers.js 04ea1cb6998af7c238eb2fd1a7d05a9663abc56c
> src/main/resources/scheduler/assets/js/directives.js 408ee21416621d5d1c195752e1b1c5c2757f0664
> src/main/resources/scheduler/assets/js/services.js a514fa7da3c049e1f5a3e49263a82cf2c28d1b2d
> src/main/resources/scheduler/assets/latestUpdates.html bcdb70f8680ebd79b424580a048cfdb611a56f0d
> src/main/resources/scheduler/assets/loadingFooter.html PRE-CREATION
> src/main/resources/scheduler/assets/role.html 2572c4b4497aa78d3fa31457a5dc7c57e6565027
> src/main/resources/scheduler/assets/taskInstance.html PRE-CREATION
> src/main/resources/scheduler/assets/update.html e95053fdb499cae1c734f955d90f54cffbce68d3
> src/test/java/org/apache/aurora/GuavaUtilsTest.java PRE-CREATION
> src/test/java/org/apache/aurora/ProtobufsTest.java PRE-CREATION
> src/test/java/org/apache/aurora/auth/UnsecureSessionContextTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java b4dbab77ec0f8e843dd5248655b10fc7b7794398
> src/test/java/org/apache/aurora/scheduler/ResourcesTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/SchedulerLifecycleTest.java 8476b3409297c50262bfb62142393b3390d2d0c4
> src/test/java/org/apache/aurora/scheduler/SchedulerModuleTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 256ee7700a539575b143cbe8de4803e4c99b000e
> src/test/java/org/apache/aurora/scheduler/TierManagerTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/UserTaskLauncherTest.java 126001a90c819f9b3bfd448a060102bcef8b2a35
> src/test/java/org/apache/aurora/scheduler/app/ModulesTest.java 01137e9e6862f64382de83a07af197d1b9e2f09d
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 975920af695f1d4590c3e12ab204b55928dfd91a
> src/test/java/org/apache/aurora/scheduler/app/local/FakeMaster.java 1a0f14c0945c2510685d9e2667e3c3ba509f4bbb
> src/test/java/org/apache/aurora/scheduler/app/local/LocalSchedulerMain.java 7dc03b6bbad348c5d8c0efed7e7f61a3764f0755
> src/test/java/org/apache/aurora/scheduler/app/local/simulator/ClusterSimulatorModule.java 9ee4fe2e76d36e17f8de2ab3eb714a6aae52c09c
> src/test/java/org/apache/aurora/scheduler/async/AsyncModuleTest.java 4ed6b159afda3f118e8ae28d03fdf796cbd98149
> src/test/java/org/apache/aurora/scheduler/async/GatingDelayExecutorTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/async/GcExecutorLauncherTest.java d2ec944ca12456c23eb54c9be8b1e6756f35e7f1
> src/test/java/org/apache/aurora/scheduler/async/JobUpdateHistoryPrunerTest.java f73b2c69ef99d3296e17703e1685a044c48b8587
> src/test/java/org/apache/aurora/scheduler/async/KillRetryTest.java 0faee9279bc39b7e71f3d4cd12f6b21dcc678356
> src/test/java/org/apache/aurora/scheduler/async/OfferManagerImplTest.java 874a12416c7d7a6f1b4955e923b75df23b0df2a8
> src/test/java/org/apache/aurora/scheduler/async/Offers.java 8293dd181b0d062e89776fdc1205c1c227d6bb6c
> src/test/java/org/apache/aurora/scheduler/async/RandomJitterReturnDelayTest.java 1aed40c4f2fa874a57e5eb279260383d745470c7
> src/test/java/org/apache/aurora/scheduler/async/RescheduleCalculatorImplTest.java 131bd826dfe47f40f3c27f29c095ed42953e316c
> src/test/java/org/apache/aurora/scheduler/async/TaskGroupsTest.java 51256f4c0be2616aba2e8aea2edb9e6235d5d26f
> src/test/java/org/apache/aurora/scheduler/async/TaskHistoryPrunerTest.java 6eaf3ce765c8e50b6724e40848ceb9105e1ab529
> src/test/java/org/apache/aurora/scheduler/async/TaskReconcilerTest.java 9ed1dba04667df52ba1c1538709a043308828763
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 53b21cbb706b110e8b529d34e436f5eeb10b6071
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java f3485414690458bd70c10e5e1ff5b31323c115e8
> src/test/java/org/apache/aurora/scheduler/async/TaskThrottlerTest.java 5772e15468a578d17e9323b57ceed47a638f2432
> src/test/java/org/apache/aurora/scheduler/async/TaskTimeoutTest.java b98a8d76b8651de85ac7e8bb2591ccd41038dfb9
> src/test/java/org/apache/aurora/scheduler/async/preemptor/BiCacheTest.java 47347762b2cdfc3e7d8108d3febe4c8152665059
> src/test/java/org/apache/aurora/scheduler/async/preemptor/ClusterStateImplTest.java 1572a08bc4737f6f35292b14c994432920f56e93
> src/test/java/org/apache/aurora/scheduler/async/preemptor/PendingTaskProcessorTest.java 0280a0b54f9a75c72fd079a6ff2b83d966e58a74
> src/test/java/org/apache/aurora/scheduler/async/preemptor/PreemptionVictimFilterTest.java 67dfb82d1a35e1981e8a90788a3821ea083e5298
> src/test/java/org/apache/aurora/scheduler/async/preemptor/PreemptionVictimTest.java bb93b638f6d63bb9f449ab1b4dc8a9c9bca14f40
> src/test/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImplTest.java 6ecdbd17898e2df4addd93fe03f536b14a65a638
> src/test/java/org/apache/aurora/scheduler/async/preemptor/PreemptorModuleTest.java 9d3820a72997a03b25701a2d4ac0691a15734a66
> src/test/java/org/apache/aurora/scheduler/base/AsyncUtilTest.java 23971868fc57d3f0fa416894fddf8ba140d77c68
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java ec43a44d9cbd0f303e4833953cdb65664d5ed569
> src/test/java/org/apache/aurora/scheduler/base/TasksTest.java ed9a2878fece0ffc75b333bdd850d6b59e913a41
> src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java abbd23dd3ee4382565ce846eb035e2aa502badae
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/configuration/ResourcesTest.java d6febb8998e05257cabe8d193cefa0b6c79f197e
> src/test/java/org/apache/aurora/scheduler/cron/quartz/AuroraCronJobTest.java b9e16578b27de2985d24c25aae507b3540fcd3ff
> src/test/java/org/apache/aurora/scheduler/cron/quartz/CronIT.java 863e9c998c97506759a5526135a33a161a8fb75f
> src/test/java/org/apache/aurora/scheduler/cron/quartz/CronJobManagerImplTest.java abb915d722235c889ce4a7dbd10238cf39affb6a
> src/test/java/org/apache/aurora/scheduler/cron/quartz/CronPredictorImplTest.java c68adc457c3e9c560e0e83c31d676c8f58ce6a4f
> src/test/java/org/apache/aurora/scheduler/cron/quartz/QuartzTestUtil.java 2d74b326831fbe22fa6045610ca8d714cd64779e
> src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java abbc8e34ed2a02d22c4c82cea6e1b69311d80754
> src/test/java/org/apache/aurora/scheduler/events/PubsubEventModuleTest.java 0e0fabb3d4cb97ad3a33b5904348eceab54778ee
> src/test/java/org/apache/aurora/scheduler/filter/AttributeAggregateTest.java 6b360626dbcf96b60e5a408d9d4b605e78d55d56
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 15bb13bf259d1fcd08e51ceda8dcd3e029a9eadd
> src/test/java/org/apache/aurora/scheduler/http/CorsFilterTest.java 2fa29ee50b8cc88be875b8d4ac1aa0fed26db631
> src/test/java/org/apache/aurora/scheduler/http/CronTest.java 6171daf4c29afd5dc8b65c613e002971b243992e
> src/test/java/org/apache/aurora/scheduler/http/H2ConsoleModuleIT.java 9536fe3c484329066d8939edc40fa32525880ef5
> src/test/java/org/apache/aurora/scheduler/http/JettyServerModuleTest.java 0d9aefff34b9a138e30747674218c2ba119b7627
> src/test/java/org/apache/aurora/scheduler/http/LeaderRedirectTest.java 703b02698191cbc0e5a911ed6cf4dbfc5de286cf
> src/test/java/org/apache/aurora/scheduler/http/LocksTest.java 55fbc55d5218b601472bce979714e33be9f88369
> src/test/java/org/apache/aurora/scheduler/http/MnameTest.java cf61dd1eee17c5bb44e96d31452d773165d3cce2
> src/test/java/org/apache/aurora/scheduler/http/QuitCallbackTest.java f4588504c7a56d4e71e27a64d9366b28fb3da5ca
> src/test/java/org/apache/aurora/scheduler/http/RequestLoggerTest.java 4999d3a68485eedec2785425dd97574c404db6a0
> src/test/java/org/apache/aurora/scheduler/http/ServicesTest.java 006c2ded9659f00bca61a400a5ef5ed7116bf916
> src/test/java/org/apache/aurora/scheduler/http/ServletFilterTest.java f8b134fa1f3fbbd7affef31acb077e7acd4f9761
> src/test/java/org/apache/aurora/scheduler/http/api/ApiBetaTest.java 08e1284ac1ef08b7649ed83df0a55be04cfeb88f
> src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java 53ba949691768078ac17846b70d1baf440c444d4
> src/test/java/org/apache/aurora/scheduler/http/api/security/Kerberos5ShiroRealmModuleTest.java fda16442b8eb1611cabb77f1c68619dfde1f83f2
> src/test/java/org/apache/aurora/scheduler/http/api/security/ShiroAuthenticatingThriftInterceptorTest.java 568cd8f6c312fa14d8bccc42a059179ec4b5064d
> src/test/java/org/apache/aurora/scheduler/http/api/security/ShiroAuthorizingInterceptorTest.java 39d2df0bc9df6ea98178c9829efa362126705e79
> src/test/java/org/apache/aurora/scheduler/http/api/security/ShiroAuthorizingParamInterceptorTest.java b2b74c62cc7a86693c8be15fb8872fe1bce81000
> src/test/java/org/apache/aurora/scheduler/log/mesos/MesosLogTest.java 5b6e0649ea07136817001eb23fdbdeb6f2a8473e
> src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java 9e176882246b48ac87bf35c77ac8c449a0a53352
> src/test/java/org/apache/aurora/scheduler/mesos/MesosSchedulerImplTest.java f0f9ac392973a276028aee3e06517a6e6d960bb6
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 1b2a7948ebb946a2e12b0eded6acf4ce3c8e20f9
> src/test/java/org/apache/aurora/scheduler/mesos/Offers.java 83eec5d3c8e493fcefbcb5b9cf67dae4e741b095
> src/test/java/org/apache/aurora/scheduler/mesos/SchedulerDriverServiceTest.java 09593b15c9bd711530ddcb5508ed85b58a2ebe02
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java ebd81775c5c9f0ef5c309869df1d12dca3ddbdd7
> src/test/java/org/apache/aurora/scheduler/mesos/TaskStatusStatsTest.java 0724278195bb47cd719f51568f112d533551e042
> src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 210bafd5911d03855758a8c2604feb7473176bbe
> src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/quota/QuotaManagerImplTest.java 58ee226a3819d6796a169156320c10677e35611a
> src/test/java/org/apache/aurora/scheduler/scheduling/TaskGroupsTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/sla/MetricCalculatorTest.java cb98834e925793fc116814371548a30470830164
> src/test/java/org/apache/aurora/scheduler/sla/SlaModuleTest.java 5ee123a03e3c8670e0c03b05c48a9f4c66f6af9d
> src/test/java/org/apache/aurora/scheduler/state/LockManagerImplTest.java 9c9cf1b45506a2c0e1a15e3e92f1bee83778a12a
> src/test/java/org/apache/aurora/scheduler/state/MaintenanceControllerImplTest.java 0d54049c822b6f24871745b7be2cf43eeca142f3
> src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java ff0ef021b3d0d7df5f37d14915a8f783672cde6a
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 702a01abaab5554319da6e6700b870fe6692fd46
> src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java b7326a629c471f319b8396aac1046ce8b8e9598c
> src/test/java/org/apache/aurora/scheduler/stats/AsyncStatsModuleTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/stats/ResourceCounterTest.java 7b12d642184e09923dd0a551ad38348cb7f2e2af
> src/test/java/org/apache/aurora/scheduler/stats/SlotSizeCounterTest.java 41f14f967074c75e996a19c90692a4d08e9db71a
> src/test/java/org/apache/aurora/scheduler/storage/AbstractCronJobStoreTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java 0f3a1a0ffde16f629422caf868fadb59be70e418
> src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 5ad0de7c6a648f5eb6408eea7bcd789c25d55f88
> src/test/java/org/apache/aurora/scheduler/storage/backup/RecoveryTest.java 09e4208afd5f811bae1531642fef63fd6e0d26bf
> src/test/java/org/apache/aurora/scheduler/storage/backup/StorageBackupTest.java ba56937bb749511ea7602cc4ff02487b8a211b7c
> src/test/java/org/apache/aurora/scheduler/storage/db/DbAttributeStoreTest.java db508cace922b20a19081a1d918cc082b255cd02
> src/test/java/org/apache/aurora/scheduler/storage/db/DbCronJobStoreTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStoreTest.java 550deae7020d0f547e81188e457e5cf2fe05caef
> src/test/java/org/apache/aurora/scheduler/storage/db/DbLockStoreTest.java ecbe1dd1eb1f0de07bea490b076c7a920f08bccb
> src/test/java/org/apache/aurora/scheduler/storage/db/DbQuotaStoreTest.java a0bd6e1360ee27c368f8a6f2fd5711d22feca14d
> src/test/java/org/apache/aurora/scheduler/storage/db/DbSchedulerStoreTest.java 0bfb4d4d3272c2da00bd9fe97c9bf9a950e9c89e
> src/test/java/org/apache/aurora/scheduler/storage/db/DbStorageTest.java b24fef95c98b32d162a8e68fc46349dded492c34
> src/test/java/org/apache/aurora/scheduler/storage/db/DbTaskStoreTest.java dda988d03634f8de582cf2b8ccdeb433c3e3de0c
> src/test/java/org/apache/aurora/scheduler/storage/db/RowGarbageCollectorTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/storage/entities/IHostAttributesTest.java 05b31c2f75da9df53ab66007a1e907900ed90cd0
> src/test/java/org/apache/aurora/scheduler/storage/log/LogManagerTest.java d4aaa670341a226ff3e51309843db1f3ff5e3d4e
> src/test/java/org/apache/aurora/scheduler/storage/log/LogStorageTest.java e45594665a29ade8adb4a735ccc176a52e3d49dc
> src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplTest.java c183acfbca0e5b87295730a77868cc37f43d164b
> src/test/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorageTest.java 4d447d348606e29ae2536974db98f82fa199a877
> src/test/java/org/apache/aurora/scheduler/storage/mem/InMemTaskStoreTest.java 2ed748383f269217860b80c831a3521facba83ba
> src/test/java/org/apache/aurora/scheduler/storage/mem/MemCronJobStoreTest.java 58256afafd12e5de234755969605861891af4daf
> src/test/java/org/apache/aurora/scheduler/storage/mem/StorageTransactionTest.java e5e889a54acf1344b31459abb19b9aff163ba9e9
> src/test/java/org/apache/aurora/scheduler/storage/testing/StorageEntityUtil.java 0bde4d8bab3a2ad09e53d024dd5c76b7fcbf3083
> src/test/java/org/apache/aurora/scheduler/storage/testing/StorageTestUtil.java 17957491ea1a02c89b4a39d47be34fbd3be2a423
> src/test/java/org/apache/aurora/scheduler/testing/FakeScheduledExecutor.java 916483b7b0876c671cdc67dc60734804b05b2317
> src/test/java/org/apache/aurora/scheduler/thrift/Fixtures.java 48179b843e3b2d4333230f0a9269f28dcd009a31
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java a854ed3d99af08fb46e436eb1bc00d8fb312278e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 38ef412a6d29dfef7b305e00cf44522818303965
> src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 2a2b499905a788eb59f5824bc66b487f91e838f1
> src/test/java/org/apache/aurora/scheduler/thrift/aop/AopModuleTest.java 5c853002ca02b1da5b0f7075e49627157abb57e4
> src/test/java/org/apache/aurora/scheduler/thrift/aop/FeatureToggleInterceptorTest.java 22a15460e4d16d79cb0ea873ad0e826035d9ead8
> src/test/java/org/apache/aurora/scheduler/thrift/aop/LoggingInterceptorTest.java 420f03fe510e9a804f457f03845b2159305cd807
> src/test/java/org/apache/aurora/scheduler/thrift/aop/ServerInfoInterceptorTest.java 793684c93b61db68798f2516926cca18591e1c05
> src/test/java/org/apache/aurora/scheduler/thrift/aop/ThriftStatsExporterInterceptorTest.java dac9af2ebbfe5f645f2e56330f3f0929713b54cf
> src/test/java/org/apache/aurora/scheduler/updater/AddTaskTest.java 2b255c48f19e3fa87351d24f061c64d726b9c9eb
> src/test/java/org/apache/aurora/scheduler/updater/InstanceUpdaterTest.java 11283031a2288aee8b41ff3c57e71315de0cd622
> src/test/java/org/apache/aurora/scheduler/updater/JobDiffTest.java ddec9e63b48a525780747bcb52f80abe1b200726
> src/test/java/org/apache/aurora/scheduler/updater/JobUpdateEventSubscriberTest.java da3986d37ed1ccafa3809862d6085b850f882a22
> src/test/java/org/apache/aurora/scheduler/updater/JobUpdateStateMachineTest.java a6a8885c09e6215970ad3b10a7f99940abd71633
> src/test/java/org/apache/aurora/scheduler/updater/JobUpdaterIT.java 33dd9f1c77c3778af69e766e3c0b91dfb5d17f46
> src/test/java/org/apache/aurora/scheduler/updater/OneWayJobUpdaterTest.java d32b40a9b374eeed280a552508c98efeef460ecb
> src/test/java/org/apache/aurora/scheduler/updater/UpdateFactoryImplTest.java d6e855b879e7909e8ba66c03ed34c845bf978a8f
> src/test/python/apache/aurora/BUILD 0701440ae47034c580b5baae40e5f6feec4bb4fd
> src/test/python/apache/aurora/admin/BUILD 69da2c97cb08025a27ca276bb2ad6fcc43db1b10
> src/test/python/apache/aurora/admin/test_admin.py 4e34ef14a0d2e536a62163377e36d52696de55a2
> src/test/python/apache/aurora/client/BUILD c55adfe9825b77f418e41fa9a4ba43926bd991ed
> src/test/python/apache/aurora/client/api/BUILD 65b378b0b4c5fa11f9899ef04a4a10a211f37245
> src/test/python/apache/aurora/client/api/test_quota_check.py ddd94667718f89e0a1f01f7798d73acd4a060909
> src/test/python/apache/aurora/client/api/test_sla.py 22b9f995eaa1deae9f5c5467e0939670c099f438
> src/test/python/apache/aurora/client/cli/BUILD 0d85f5fba9d19ae0e9c36546f130b93664b4f6de
> src/test/python/apache/aurora/client/cli/test_context.py 1a134a2ee7d88d2d12a2bcbc122ed0f447ba997f
> src/test/python/apache/aurora/client/cli/test_create.py 9f32ca3474ea51529217db29b88911010fa73602
> src/test/python/apache/aurora/client/cli/test_diff.py da52105b5c77419ff64e15a30a04f997f9216a7e
> src/test/python/apache/aurora/client/cli/test_kill.py 5eae70a3e830b01bfeab523c4dfc840447eb8f80
> src/test/python/apache/aurora/client/cli/test_quota.py 3573e4c4575577e2232d2b99ca781b06a03d48d7
> src/test/python/apache/aurora/client/cli/test_restart.py dc67cfeeeada1b226cf0fe237fde33ee0712a8cb
> src/test/python/apache/aurora/client/cli/test_supdate.py c6b6ccc8570bac9c742e6458a9d09e24aa0a3bcc
> src/test/python/apache/aurora/client/cli/test_task.py 3ad0b70a7d918055ffee34f593d108a28de6e9f9
> src/test/python/apache/aurora/client/cli/test_update.py 22b2ca71254a1d2254cc56095f19b07531212c10
> src/test/python/apache/aurora/client/cli/util.py 053c9a56248848478ca2fe6ab2c12e24c3d93c13
> src/test/python/apache/aurora/client/hooks/BUILD 0b924ccfbcdb4800f99b80067ff8a78252f99907
> src/test/python/apache/aurora/client/hooks/test_hooked_api.py f92e715ad2eb328170c998989644aba1437ca3cb
> src/test/python/apache/aurora/client/test_base.py 1a560088279ac945cce14d02454e50b8483771e4
> src/test/python/apache/aurora/client/test_config.py c56779712b91f621261358aa7ebd6c4fc65446a0
> src/test/python/apache/aurora/common/BUILD 9b1d4d876e10cb759202240931e4787ce673d897
> src/test/python/apache/aurora/common/test_http_signaler.py c6a21708eb997f213f4cca038891a84f6c5218d4
> src/test/python/apache/aurora/common/test_pex_version.py 3e6e2844231237f850e4e45cb66eb0cbdbb79029
> src/test/python/apache/aurora/common/test_transport.py f307e8dcc90a0d93ec90a90f15844bc5ce0b2c09
> src/test/python/apache/aurora/config/BUILD c85e998e66a989bb3e2ad8b9aab03702f33c0ef9
> src/test/python/apache/aurora/config/test_base.py 5a58dd0f072889a850153841f972dcca4d3c913b
> src/test/python/apache/aurora/config/test_loader.py 00b6eabadb8f4e40aba1191068e1ca04c78d1ff3
> src/test/python/apache/aurora/config/test_thrift.py 654c0b5ae82c98db163c7a44301ff6b23e19b211
> src/test/python/apache/aurora/executor/BUILD f415ecc77022b34f053c35272d004e133803d702
> src/test/python/apache/aurora/executor/bin/BUILD 2caab2aec136ede9b51ce3bdd0d139270024ba48
> src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py d4c1d572663039eb742f70de1e06d708eb0b769a
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py ec5b3f96b3decfd66c37301e9c2101562bd7f8d3
> src/test/python/apache/aurora/executor/common/BUILD b3da27bb83c44a228851b256cc09f093d1e2eb87
> src/test/python/apache/aurora/executor/common/fixtures.py 37d032beb66a67cfd3cfcea272747a2915a745ff
> src/test/python/apache/aurora/executor/common/test_announcer.py f614f2c47893e0e16877c1449cf90ca4237382a0
> src/test/python/apache/aurora/executor/common/test_path_detector.py 86460bb6499596964124bec85d74ca9dba7d2c7c
> src/test/python/apache/aurora/executor/common/test_task_info.py 102ba531aa6c28f2d74bd0d7f1668e5861e3a6b8
> src/test/python/apache/aurora/executor/test_gc_executor.py 17d359054d1fc33f79a7612162064abd335ccf81
> src/test/python/apache/aurora/executor/test_http_lifecycle.py PRE-CREATION
> src/test/python/apache/aurora/executor/test_thermos_executor.py 26b26f452fb7c718e4f99f470211dfe926ebe354
> src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1e21a112efcbd0a575fb8d90f78ff1e8c7f034cf
> src/test/python/apache/aurora/tools/test_thermos_observer_entry_point.py PRE-CREATION
> src/test/python/apache/thermos/cli/BUILD 087b1740de4d8a30c74c7776eb83f9784e0049cc
> src/test/python/apache/thermos/cli/commands/BUILD 5465b19be5ab8bdf7252e7b1fa7a4ef95063193d
> src/test/python/apache/thermos/cli/commands/test_import.py 74d9a32cf85a9e49cfbc596a7d6d44393df14e7a
> src/test/python/apache/thermos/common/BUILD bb70867e0e070ee9651f018f30990bd9ff0dd88f
> src/test/python/apache/thermos/config/BUILD a06af36c83fc7ec00ecd00df7e424faae3118131
> src/test/python/apache/thermos/config/test_schema.py 7e1867485593a8fadacb525c57264778831b24f6
> src/test/python/apache/thermos/core/BUILD d637c24f098468776fb8b0758f5990b83ec362c5
> src/test/python/apache/thermos/monitoring/BUILD 89030d0e25e8eb3f4d4eec6a0d0a0fc3dfd43481
> src/test/python/apache/thermos/monitoring/test_garbage.py 4309c46a3af5f12c8eb3192e3156348fa7c0db23
> src/test/python/apache/thermos/monitoring/test_resource.py a7ab360d7bc0022b115b9059b16d76b8923d5e23
> src/test/python/apache/thermos/observer/BUILD ff92a52fe7a88379e8aa1c1b2385c065bd375a68
> src/test/python/apache/thermos/observer/http/BUILD 0cdaafc3e71d21d48c35e0dac68910c53d003fae
> src/test/python/apache/thermos/observer/test_task_observer.py PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/tiers-example.json PRE-CREATION
> src/test/sh/org/apache/aurora/e2e/Dockerfile e7f7f541e340c426b4e2a0f7a2c326af4bf902c2
> src/test/sh/org/apache/aurora/e2e/http/http_example.aurora c1a10d8ea60be6aa56e4517fb34288d7d5ae1480
> src/test/sh/org/apache/aurora/e2e/http/http_example_docker.aurora 870b3e68035fdf86253cf9b92b606645134b3369
> src/test/sh/org/apache/aurora/e2e/http/http_example_docker_updated.aurora e55aad3a58d4e3c19332e06b70771f51f07aa9b7
> src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora 423dd4d4e8b03c2f852e25acd9340bd6288b7d24
> src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 018efbe43de0c829799bc2f789958c55b1cb61a4
> src/test/sh/org/apache/aurora/e2e/test_kerberos_end_to_end.sh 4d6043a402a761fe44239e33b83c1c8872fe7068
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> Currently fails end to end test. Investigating cause, but has to do with the job failing to announce in the serverset.
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Kevin Sweeney <ke...@apache.org>.
> On Sept. 2, 2015, 9:45 a.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, lines 4-7
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061919#file1061919line4>
> >
> > The code later converts this array into a single command string. I suggest we just make this a string here (and save some code down the line).
>
> Renan DelValle wrote:
> We can go ahead and do that. Kevin thought it should be the array way, but I'm not sure how strongly he feels about it. The extra code isn't too bad because we can use String.join() form Java 8. Either way, I'm fine with either. The single string may eliminate any suspicions that we're breaking something in a command when joining the array.
Bill, I prefer the array mode as it's more explicit for dealing with magic characters in shell commands (like " (){}$").
You can still explicitly do
```
["sh", "-c", "PATH=/opt/aurora/executor/bin:$PATH thermos"]
```
When you prefer the old form, but this form will not automatically expand environment variables or do other shell magic for you.
Renan - can you update CommandUtil to use the `arguments`[1] field instead of `shell` and `value` (or drop a TODO for a later patch)? Changing back to `String.join` defeats part of the purpose of the array form here.
[1] https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L408-L423
- Kevin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 1, 2015, 7:35 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 1, 2015, 7:35 p.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Bill Farner <wf...@apache.org>.
> On Sept. 2, 2015, 9:45 a.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, lines 4-7
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061919#file1061919line4>
> >
> > The code later converts this array into a single command string. I suggest we just make this a string here (and save some code down the line).
>
> Renan DelValle wrote:
> We can go ahead and do that. Kevin thought it should be the array way, but I'm not sure how strongly he feels about it. The extra code isn't too bad because we can use String.join() form Java 8. Either way, I'm fine with either. The single string may eliminate any suspicions that we're breaking something in a command when joining the array.
>
> Kevin Sweeney wrote:
> Bill, I prefer the array mode as it's more explicit for dealing with magic characters in shell commands (like " (){}$").
>
> You can still explicitly do
>
> ```
> ["sh", "-c", "PATH=/opt/aurora/executor/bin:$PATH thermos"]
> ```
>
> When you prefer the old form, but this form will not automatically expand environment variables or do other shell magic for you.
>
> Renan - can you update CommandUtil to use the `arguments`[1] field instead of `shell` and `value` (or drop a TODO for a later patch)? Changing back to `String.join` defeats part of the purpose of the array form here.
>
> [1] https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L408-L423
Aha - the `arguments` field in mesos.proto was the missing detail for me. Since mesos allows us to do the right thing here, we should definitely use an array.
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 1, 2015, 7:35 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 1, 2015, 7:35 p.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Bill Farner <wf...@apache.org>.
> On Sept. 2, 2015, 9:45 a.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java, line 87
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061926#file1061926line87>
> >
> > This code will be easier to maintain long-term with 2 goals in mind:
> > - keep json-related code within this class (no annotations on ExecutorSettings)
> > - avoid hand-rolling deserializers
> >
> > This sounds like more work, but it's not that bad. The major difference is that you'll create a class in this file that defines the JSON structure, and does so in a way that the json deserializer can work with.
> >
> > Maxim just did something very similar, which you can crib from:
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55
>
> Bill Farner wrote:
> Oh, the detail i neglected to mention - i suggest you then write code to translate your config schema object into ExecutorSettings. This can provides compile-time guarantees that we don't have with the current code, making it easier to maintain.
>
> Renan DelValle wrote:
> Can you expend on this? I don't have any experience with Jackson. I'm looking through what Maxim did and I'll definitely crib here and there but I can't find anything related to getting this done at compile time.
>
> Bill Farner wrote:
> Don't be thrown by the mention of compile time, i'm just referring to refactor safety if/when the structures changed. The concise way to put it is - use a different class for your config domain object. This way you don't need to change ExecutorSettings to compensate for the fact that some code in another part of the system is using a json mapper.
>
> Start by defining a class that matches your config schema:
> ```
> class ExecutorConfig {
> String command;
> List<Resources> resource;
> ...
> }
>
> class Resource {
> String value; // note - i would like to see this renamed to uri
> boolean executable;
> boolean extract;
> boolean cache;
> }
> ...
> ```
> (that's just a quick example, feel free to adjust as you see fit)
>
> Jackson is very similar to gson, there's just a behavior nuance that is desirable. You'll use an `ObjectMapper` to parse the json to an `ExecutorConfig`, then write a function to transform your `ExecutorConfig` into `ExecutorSettings`.
>
> ```
> private static ExecutorSettings configToSettings(ExecutorConfig config) {
> return new ExecutorSettings(...);
> }
> ```
I took a quick stab at illustrating this, take a look here: https://github.com/wfarner/aurora/commit/a53c215432b34ba95238121581375bde24b88a1e
The remaining work is to complete filling out constructor fields:
https://github.com/wfarner/aurora/blob/a53c215432b34ba95238121581375bde24b88a1e/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java#L53-L61
This is really a matter of writing functions to translate from types in `ExecutorConfig` to the corresponding types in `ExecutorSettings`.
Depending on your bandwidth over the next few days, i may take over this patch if you don't mind.
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 8, 2015, 9:32 a.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 8, 2015, 9:32 a.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Renan DelValle <rd...@binghamton.edu>.
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java, line 87
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061926#file1061926line87>
> >
> > This code will be easier to maintain long-term with 2 goals in mind:
> > - keep json-related code within this class (no annotations on ExecutorSettings)
> > - avoid hand-rolling deserializers
> >
> > This sounds like more work, but it's not that bad. The major difference is that you'll create a class in this file that defines the JSON structure, and does so in a way that the json deserializer can work with.
> >
> > Maxim just did something very similar, which you can crib from:
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55
>
> Bill Farner wrote:
> Oh, the detail i neglected to mention - i suggest you then write code to translate your config schema object into ExecutorSettings. This can provides compile-time guarantees that we don't have with the current code, making it easier to maintain.
>
> Renan DelValle wrote:
> Can you expend on this? I don't have any experience with Jackson. I'm looking through what Maxim did and I'll definitely crib here and there but I can't find anything related to getting this done at compile time.
>
> Bill Farner wrote:
> Don't be thrown by the mention of compile time, i'm just referring to refactor safety if/when the structures changed. The concise way to put it is - use a different class for your config domain object. This way you don't need to change ExecutorSettings to compensate for the fact that some code in another part of the system is using a json mapper.
>
> Start by defining a class that matches your config schema:
> ```
> class ExecutorConfig {
> String command;
> List<Resources> resource;
> ...
> }
>
> class Resource {
> String value; // note - i would like to see this renamed to uri
> boolean executable;
> boolean extract;
> boolean cache;
> }
> ...
> ```
> (that's just a quick example, feel free to adjust as you see fit)
>
> Jackson is very similar to gson, there's just a behavior nuance that is desirable. You'll use an `ObjectMapper` to parse the json to an `ExecutorConfig`, then write a function to transform your `ExecutorConfig` into `ExecutorSettings`.
>
> ```
> private static ExecutorSettings configToSettings(ExecutorConfig config) {
> return new ExecutorSettings(...);
> }
> ```
>
> Bill Farner wrote:
> I took a quick stab at illustrating this, take a look here: https://github.com/wfarner/aurora/commit/a53c215432b34ba95238121581375bde24b88a1e
>
> The remaining work is to complete filling out constructor fields:
> https://github.com/wfarner/aurora/blob/a53c215432b34ba95238121581375bde24b88a1e/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java#L53-L61
>
> This is really a matter of writing functions to translate from types in `ExecutorConfig` to the corresponding types in `ExecutorSettings`.
>
> Depending on your bandwidth over the next few days, i may take over this patch if you don't mind.
>
> Renan DelValle wrote:
> Thanks for the examples, in particular the entry transformer really helped speed things along.
>
> I've almost got this done, and I'm having an issue with using optionals, which in my opinion are necessary if we want to leave the defaults set on the URI resource configuration if nothing is specified. I.e: If executable is not in the schema for a URI, then its better to not set it than to set a default value, IMO.
>
> I've looked around and Jackson has support for them if we load a new module:
> https://github.com/FasterXML/jackson-datatype-jdk8
>
> However, I can't find where the jackson dependecy is. And the fact the jackson pacakges still point to codehaus lead me to believe were using an old version.
>
> Here are some relevant files:
> https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorConfiguration.java
> https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
>
> We're almost there.
>
> Bill Farner wrote:
> Perhaps we should go a step further and try to directly parse `ExecutorInfo`? That way we don't have to maintain parity between the protobuf schema and our own. Jackson can't directly deserialize protobuf-generated objects, but a quick search turned up this promising project: https://github.com/HubSpot/jackson-datatype-protobuf
>
> The deps are defined in build.gradle. You're right about the old jackson library (pulled in by an old jersey-json). However, the fact that they changed packages plays in our favor - we can add an explicit dep on the new jackson version and use the new one in this part of the code.
>
> Bill Farner wrote:
> I put together a proof of concept of this idea here: https://github.com/wfarner/aurora/commit/04e1ff6ad566755823b624303cfd66541076785b
>
> You'll notice that this makes the json more verbose, but it becomes a 1:1 association with the protobuf message.
>
> Renan DelValle wrote:
> Awesome, thanks for that. It's a WIP at the moment, but I just wanted to share what I have so far so I'm not going off on some trail I shouldn't be going on.
>
> I cribbed some of the code from your POC and came up with this. I'm still working on fixing all the tests that changing the ExecutorSettings broke, but after that it should be good to go.
>
> https://github.com/rdelval/aurora/commit/58e4524ab20698d6c665d7736dadbcac651d445e#diff-ca4de6b05130534619820a5f6a521f2d
> https://github.com/rdelval/aurora/commit/38958251833566589d448c9f3a19096201430605#diff-ca4de6b05130534619820a5f6a521f2d
>
> You'll notice I condensed the command list and URIs into a CommandBuilder and I changed the Volume from the Aurora one to the Mesos Protobuf one, I hope this last change isn't an issue, if it is, let me know.
>
> With these changes IMO, src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java is no longer needed, at least I haven't found anywhere where it would be a benefit to have it.
>
> Bill Farner wrote:
> The protobuf `Volume` object is the right thing to use. The most important thing i would suggest is that you remove `ExecutorSettings`, and let the protobuf go all the way to where it is used in `TaskInfo`.
>
> If CommandUtil is no longer used, it should indeed be removed.
The only thing that concerns me with doing that, is the amount of yak shaving involved in replacing ExecutorSettings everywhere that it's used in the tests. If you don't mind the patch ballooing in size a little bit, I also think this is the proper way to do it. Otherwise, we can leave a transformer for now and do this in a separate patch.
- Renan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 8, 2015, 4:32 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 8, 2015, 4:32 p.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Renan DelValle <rd...@binghamton.edu>.
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, lines 17-18
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061919#file1061919line17>
> >
> > Can you omit this since it's blank?
Yep, was just leaving it there for now so I don't forget to put it in the documaenttion for this feature.
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, lines 4-7
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061919#file1061919line4>
> >
> > The code later converts this array into a single command string. I suggest we just make this a string here (and save some code down the line).
We can go ahead and do that. Kevin thought it should be the array way, but I'm not sure how strongly he feels about it. The extra code isn't too bad because we can use String.join() form Java 8. Either way, I'm fine with either. The single string may eliminate any suspicions that we're breaking something in a command when joining the array.
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, line 32
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061919#file1061919line32>
> >
> > Use single quotes to avoid escaping:
> >
> > "'Hello World from Aurora!'"
Done
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java, lines 84-115
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061924#file1061924line84>
> >
> > Please add an entry in /NEWS under 0.10.0 to inform people what args were removed, and what the replacement is.
Done.
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java, lines 71-73
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061925#file1061925line71>
> >
> > remove
Done
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java, lines 107-108
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061925#file1061925line107>
> >
> > I suggest removing this for now. There's a lot of ways fetching the URIs and executing commands can fail. For the sake of simplicity, let's allow them to fail and focus on providing good visibility into the failure.
Done
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java, line 46
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061926#file1061926line46>
> >
> > please add a javadoc for this class
Done
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java, line 62
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061926#file1061926line62>
> >
> > Save a few lines:
> >
> > ```
> > String config = Files.toString(configFile, StandardCharsets.UTF_8);
> > ```
Awesome. Done
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java, line 79
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061926#file1061926line79>
> >
> > remove the ```\n```, same with the string below
Done and Done
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java, line 210
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061924#file1061924line210>
> >
> > throw Throwables.propagate(e);
> >
> > Otherwise the app will exit in a non-obvious way.
Thanks for pointing it out. I meant to come back to it but ended up forgetting about it.
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java, line 87
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061926#file1061926line87>
> >
> > This code will be easier to maintain long-term with 2 goals in mind:
> > - keep json-related code within this class (no annotations on ExecutorSettings)
> > - avoid hand-rolling deserializers
> >
> > This sounds like more work, but it's not that bad. The major difference is that you'll create a class in this file that defines the JSON structure, and does so in a way that the json deserializer can work with.
> >
> > Maxim just did something very similar, which you can crib from:
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55
>
> Bill Farner wrote:
> Oh, the detail i neglected to mention - i suggest you then write code to translate your config schema object into ExecutorSettings. This can provides compile-time guarantees that we don't have with the current code, making it easier to maintain.
Can you expend on this? I don't have any experience with Jackson. I'm looking through what Maxim did and I'll definitely crib here and there but I can't find anything related to getting this done at compile time.
- Renan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 2, 2015, 2:35 a.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 2, 2015, 2:35 a.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Bill Farner <wf...@apache.org>.
> On Sept. 2, 2015, 9:45 a.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java, line 87
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061926#file1061926line87>
> >
> > This code will be easier to maintain long-term with 2 goals in mind:
> > - keep json-related code within this class (no annotations on ExecutorSettings)
> > - avoid hand-rolling deserializers
> >
> > This sounds like more work, but it's not that bad. The major difference is that you'll create a class in this file that defines the JSON structure, and does so in a way that the json deserializer can work with.
> >
> > Maxim just did something very similar, which you can crib from:
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55
>
> Bill Farner wrote:
> Oh, the detail i neglected to mention - i suggest you then write code to translate your config schema object into ExecutorSettings. This can provides compile-time guarantees that we don't have with the current code, making it easier to maintain.
>
> Renan DelValle wrote:
> Can you expend on this? I don't have any experience with Jackson. I'm looking through what Maxim did and I'll definitely crib here and there but I can't find anything related to getting this done at compile time.
Don't be thrown by the mention of compile time, i'm just referring to refactor safety if/when the structures changed. The concise way to put it is - use a different class for your config domain object. This way you don't need to change ExecutorSettings to compensate for the fact that some code in another part of the system is using a json mapper.
Start by defining a class that matches your config schema:
```
class ExecutorConfig {
String command;
List<Resources> resource;
...
}
class Resource {
String value; // note - i would like to see this renamed to uri
boolean executable;
boolean extract;
boolean cache;
}
...
```
(that's just a quick example, feel free to adjust as you see fit)
Jackson is very similar to gson, there's just a behavior nuance that is desirable. You'll use an `ObjectMapper` to parse the json to an `ExecutorConfig`, then write a function to transform your `ExecutorConfig` into `ExecutorSettings`.
```
private static ExecutorSettings configToSettings(ExecutorConfig config) {
return new ExecutorSettings(...);
}
```
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 1, 2015, 7:35 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 1, 2015, 7:35 p.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Bill Farner <wf...@apache.org>.
> On Sept. 2, 2015, 9:45 a.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java, line 87
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061926#file1061926line87>
> >
> > This code will be easier to maintain long-term with 2 goals in mind:
> > - keep json-related code within this class (no annotations on ExecutorSettings)
> > - avoid hand-rolling deserializers
> >
> > This sounds like more work, but it's not that bad. The major difference is that you'll create a class in this file that defines the JSON structure, and does so in a way that the json deserializer can work with.
> >
> > Maxim just did something very similar, which you can crib from:
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55
>
> Bill Farner wrote:
> Oh, the detail i neglected to mention - i suggest you then write code to translate your config schema object into ExecutorSettings. This can provides compile-time guarantees that we don't have with the current code, making it easier to maintain.
>
> Renan DelValle wrote:
> Can you expend on this? I don't have any experience with Jackson. I'm looking through what Maxim did and I'll definitely crib here and there but I can't find anything related to getting this done at compile time.
>
> Bill Farner wrote:
> Don't be thrown by the mention of compile time, i'm just referring to refactor safety if/when the structures changed. The concise way to put it is - use a different class for your config domain object. This way you don't need to change ExecutorSettings to compensate for the fact that some code in another part of the system is using a json mapper.
>
> Start by defining a class that matches your config schema:
> ```
> class ExecutorConfig {
> String command;
> List<Resources> resource;
> ...
> }
>
> class Resource {
> String value; // note - i would like to see this renamed to uri
> boolean executable;
> boolean extract;
> boolean cache;
> }
> ...
> ```
> (that's just a quick example, feel free to adjust as you see fit)
>
> Jackson is very similar to gson, there's just a behavior nuance that is desirable. You'll use an `ObjectMapper` to parse the json to an `ExecutorConfig`, then write a function to transform your `ExecutorConfig` into `ExecutorSettings`.
>
> ```
> private static ExecutorSettings configToSettings(ExecutorConfig config) {
> return new ExecutorSettings(...);
> }
> ```
>
> Bill Farner wrote:
> I took a quick stab at illustrating this, take a look here: https://github.com/wfarner/aurora/commit/a53c215432b34ba95238121581375bde24b88a1e
>
> The remaining work is to complete filling out constructor fields:
> https://github.com/wfarner/aurora/blob/a53c215432b34ba95238121581375bde24b88a1e/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java#L53-L61
>
> This is really a matter of writing functions to translate from types in `ExecutorConfig` to the corresponding types in `ExecutorSettings`.
>
> Depending on your bandwidth over the next few days, i may take over this patch if you don't mind.
>
> Renan DelValle wrote:
> Thanks for the examples, in particular the entry transformer really helped speed things along.
>
> I've almost got this done, and I'm having an issue with using optionals, which in my opinion are necessary if we want to leave the defaults set on the URI resource configuration if nothing is specified. I.e: If executable is not in the schema for a URI, then its better to not set it than to set a default value, IMO.
>
> I've looked around and Jackson has support for them if we load a new module:
> https://github.com/FasterXML/jackson-datatype-jdk8
>
> However, I can't find where the jackson dependecy is. And the fact the jackson pacakges still point to codehaus lead me to believe were using an old version.
>
> Here are some relevant files:
> https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorConfiguration.java
> https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
>
> We're almost there.
>
> Bill Farner wrote:
> Perhaps we should go a step further and try to directly parse `ExecutorInfo`? That way we don't have to maintain parity between the protobuf schema and our own. Jackson can't directly deserialize protobuf-generated objects, but a quick search turned up this promising project: https://github.com/HubSpot/jackson-datatype-protobuf
>
> The deps are defined in build.gradle. You're right about the old jackson library (pulled in by an old jersey-json). However, the fact that they changed packages plays in our favor - we can add an explicit dep on the new jackson version and use the new one in this part of the code.
I put together a proof of concept of this idea here: https://github.com/wfarner/aurora/commit/04e1ff6ad566755823b624303cfd66541076785b
You'll notice that this makes the json more verbose, but it becomes a 1:1 association with the protobuf message.
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 8, 2015, 9:32 a.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 8, 2015, 9:32 a.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Bill Farner <wf...@apache.org>.
> On Sept. 2, 2015, 9:45 a.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java, line 87
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061926#file1061926line87>
> >
> > This code will be easier to maintain long-term with 2 goals in mind:
> > - keep json-related code within this class (no annotations on ExecutorSettings)
> > - avoid hand-rolling deserializers
> >
> > This sounds like more work, but it's not that bad. The major difference is that you'll create a class in this file that defines the JSON structure, and does so in a way that the json deserializer can work with.
> >
> > Maxim just did something very similar, which you can crib from:
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55
>
> Bill Farner wrote:
> Oh, the detail i neglected to mention - i suggest you then write code to translate your config schema object into ExecutorSettings. This can provides compile-time guarantees that we don't have with the current code, making it easier to maintain.
>
> Renan DelValle wrote:
> Can you expend on this? I don't have any experience with Jackson. I'm looking through what Maxim did and I'll definitely crib here and there but I can't find anything related to getting this done at compile time.
>
> Bill Farner wrote:
> Don't be thrown by the mention of compile time, i'm just referring to refactor safety if/when the structures changed. The concise way to put it is - use a different class for your config domain object. This way you don't need to change ExecutorSettings to compensate for the fact that some code in another part of the system is using a json mapper.
>
> Start by defining a class that matches your config schema:
> ```
> class ExecutorConfig {
> String command;
> List<Resources> resource;
> ...
> }
>
> class Resource {
> String value; // note - i would like to see this renamed to uri
> boolean executable;
> boolean extract;
> boolean cache;
> }
> ...
> ```
> (that's just a quick example, feel free to adjust as you see fit)
>
> Jackson is very similar to gson, there's just a behavior nuance that is desirable. You'll use an `ObjectMapper` to parse the json to an `ExecutorConfig`, then write a function to transform your `ExecutorConfig` into `ExecutorSettings`.
>
> ```
> private static ExecutorSettings configToSettings(ExecutorConfig config) {
> return new ExecutorSettings(...);
> }
> ```
>
> Bill Farner wrote:
> I took a quick stab at illustrating this, take a look here: https://github.com/wfarner/aurora/commit/a53c215432b34ba95238121581375bde24b88a1e
>
> The remaining work is to complete filling out constructor fields:
> https://github.com/wfarner/aurora/blob/a53c215432b34ba95238121581375bde24b88a1e/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java#L53-L61
>
> This is really a matter of writing functions to translate from types in `ExecutorConfig` to the corresponding types in `ExecutorSettings`.
>
> Depending on your bandwidth over the next few days, i may take over this patch if you don't mind.
>
> Renan DelValle wrote:
> Thanks for the examples, in particular the entry transformer really helped speed things along.
>
> I've almost got this done, and I'm having an issue with using optionals, which in my opinion are necessary if we want to leave the defaults set on the URI resource configuration if nothing is specified. I.e: If executable is not in the schema for a URI, then its better to not set it than to set a default value, IMO.
>
> I've looked around and Jackson has support for them if we load a new module:
> https://github.com/FasterXML/jackson-datatype-jdk8
>
> However, I can't find where the jackson dependecy is. And the fact the jackson pacakges still point to codehaus lead me to believe were using an old version.
>
> Here are some relevant files:
> https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorConfiguration.java
> https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
>
> We're almost there.
Perhaps we should go a step further and try to directly parse `ExecutorInfo`? That way we don't have to maintain parity between the protobuf schema and our own. Jackson can't directly deserialize protobuf-generated objects, but a quick search turned up this promising project: https://github.com/HubSpot/jackson-datatype-protobuf
The deps are defined in build.gradle. You're right about the old jackson library (pulled in by an old jersey-json). However, the fact that they changed packages plays in our favor - we can add an explicit dep on the new jackson version and use the new one in this part of the code.
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 8, 2015, 9:32 a.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 8, 2015, 9:32 a.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Bill Farner <wf...@apache.org>.
> On Sept. 2, 2015, 9:45 a.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java, line 87
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061926#file1061926line87>
> >
> > This code will be easier to maintain long-term with 2 goals in mind:
> > - keep json-related code within this class (no annotations on ExecutorSettings)
> > - avoid hand-rolling deserializers
> >
> > This sounds like more work, but it's not that bad. The major difference is that you'll create a class in this file that defines the JSON structure, and does so in a way that the json deserializer can work with.
> >
> > Maxim just did something very similar, which you can crib from:
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55
>
> Bill Farner wrote:
> Oh, the detail i neglected to mention - i suggest you then write code to translate your config schema object into ExecutorSettings. This can provides compile-time guarantees that we don't have with the current code, making it easier to maintain.
>
> Renan DelValle wrote:
> Can you expend on this? I don't have any experience with Jackson. I'm looking through what Maxim did and I'll definitely crib here and there but I can't find anything related to getting this done at compile time.
>
> Bill Farner wrote:
> Don't be thrown by the mention of compile time, i'm just referring to refactor safety if/when the structures changed. The concise way to put it is - use a different class for your config domain object. This way you don't need to change ExecutorSettings to compensate for the fact that some code in another part of the system is using a json mapper.
>
> Start by defining a class that matches your config schema:
> ```
> class ExecutorConfig {
> String command;
> List<Resources> resource;
> ...
> }
>
> class Resource {
> String value; // note - i would like to see this renamed to uri
> boolean executable;
> boolean extract;
> boolean cache;
> }
> ...
> ```
> (that's just a quick example, feel free to adjust as you see fit)
>
> Jackson is very similar to gson, there's just a behavior nuance that is desirable. You'll use an `ObjectMapper` to parse the json to an `ExecutorConfig`, then write a function to transform your `ExecutorConfig` into `ExecutorSettings`.
>
> ```
> private static ExecutorSettings configToSettings(ExecutorConfig config) {
> return new ExecutorSettings(...);
> }
> ```
>
> Bill Farner wrote:
> I took a quick stab at illustrating this, take a look here: https://github.com/wfarner/aurora/commit/a53c215432b34ba95238121581375bde24b88a1e
>
> The remaining work is to complete filling out constructor fields:
> https://github.com/wfarner/aurora/blob/a53c215432b34ba95238121581375bde24b88a1e/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java#L53-L61
>
> This is really a matter of writing functions to translate from types in `ExecutorConfig` to the corresponding types in `ExecutorSettings`.
>
> Depending on your bandwidth over the next few days, i may take over this patch if you don't mind.
>
> Renan DelValle wrote:
> Thanks for the examples, in particular the entry transformer really helped speed things along.
>
> I've almost got this done, and I'm having an issue with using optionals, which in my opinion are necessary if we want to leave the defaults set on the URI resource configuration if nothing is specified. I.e: If executable is not in the schema for a URI, then its better to not set it than to set a default value, IMO.
>
> I've looked around and Jackson has support for them if we load a new module:
> https://github.com/FasterXML/jackson-datatype-jdk8
>
> However, I can't find where the jackson dependecy is. And the fact the jackson pacakges still point to codehaus lead me to believe were using an old version.
>
> Here are some relevant files:
> https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorConfiguration.java
> https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
>
> We're almost there.
>
> Bill Farner wrote:
> Perhaps we should go a step further and try to directly parse `ExecutorInfo`? That way we don't have to maintain parity between the protobuf schema and our own. Jackson can't directly deserialize protobuf-generated objects, but a quick search turned up this promising project: https://github.com/HubSpot/jackson-datatype-protobuf
>
> The deps are defined in build.gradle. You're right about the old jackson library (pulled in by an old jersey-json). However, the fact that they changed packages plays in our favor - we can add an explicit dep on the new jackson version and use the new one in this part of the code.
>
> Bill Farner wrote:
> I put together a proof of concept of this idea here: https://github.com/wfarner/aurora/commit/04e1ff6ad566755823b624303cfd66541076785b
>
> You'll notice that this makes the json more verbose, but it becomes a 1:1 association with the protobuf message.
>
> Renan DelValle wrote:
> Awesome, thanks for that. It's a WIP at the moment, but I just wanted to share what I have so far so I'm not going off on some trail I shouldn't be going on.
>
> I cribbed some of the code from your POC and came up with this. I'm still working on fixing all the tests that changing the ExecutorSettings broke, but after that it should be good to go.
>
> https://github.com/rdelval/aurora/commit/58e4524ab20698d6c665d7736dadbcac651d445e#diff-ca4de6b05130534619820a5f6a521f2d
> https://github.com/rdelval/aurora/commit/38958251833566589d448c9f3a19096201430605#diff-ca4de6b05130534619820a5f6a521f2d
>
> You'll notice I condensed the command list and URIs into a CommandBuilder and I changed the Volume from the Aurora one to the Mesos Protobuf one, I hope this last change isn't an issue, if it is, let me know.
>
> With these changes IMO, src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java is no longer needed, at least I haven't found anywhere where it would be a benefit to have it.
>
> Bill Farner wrote:
> The protobuf `Volume` object is the right thing to use. The most important thing i would suggest is that you remove `ExecutorSettings`, and let the protobuf go all the way to where it is used in `TaskInfo`.
>
> If CommandUtil is no longer used, it should indeed be removed.
>
> Renan DelValle wrote:
> The only thing that concerns me with doing that, is the amount of yak shaving involved in replacing ExecutorSettings everywhere that it's used in the tests. If you don't mind the patch ballooing in size a little bit, I also think this is the proper way to do it. Otherwise, we can leave a transformer for now and do this in a separate patch.
To insulate somewhat from that, you could keep `ExecutorSettings`, but keep `ExecutorInfo.Builder` as a field inside it.
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 8, 2015, 9:32 a.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 8, 2015, 9:32 a.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Renan DelValle <rd...@binghamton.edu>.
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java, line 87
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061926#file1061926line87>
> >
> > This code will be easier to maintain long-term with 2 goals in mind:
> > - keep json-related code within this class (no annotations on ExecutorSettings)
> > - avoid hand-rolling deserializers
> >
> > This sounds like more work, but it's not that bad. The major difference is that you'll create a class in this file that defines the JSON structure, and does so in a way that the json deserializer can work with.
> >
> > Maxim just did something very similar, which you can crib from:
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55
>
> Bill Farner wrote:
> Oh, the detail i neglected to mention - i suggest you then write code to translate your config schema object into ExecutorSettings. This can provides compile-time guarantees that we don't have with the current code, making it easier to maintain.
>
> Renan DelValle wrote:
> Can you expend on this? I don't have any experience with Jackson. I'm looking through what Maxim did and I'll definitely crib here and there but I can't find anything related to getting this done at compile time.
>
> Bill Farner wrote:
> Don't be thrown by the mention of compile time, i'm just referring to refactor safety if/when the structures changed. The concise way to put it is - use a different class for your config domain object. This way you don't need to change ExecutorSettings to compensate for the fact that some code in another part of the system is using a json mapper.
>
> Start by defining a class that matches your config schema:
> ```
> class ExecutorConfig {
> String command;
> List<Resources> resource;
> ...
> }
>
> class Resource {
> String value; // note - i would like to see this renamed to uri
> boolean executable;
> boolean extract;
> boolean cache;
> }
> ...
> ```
> (that's just a quick example, feel free to adjust as you see fit)
>
> Jackson is very similar to gson, there's just a behavior nuance that is desirable. You'll use an `ObjectMapper` to parse the json to an `ExecutorConfig`, then write a function to transform your `ExecutorConfig` into `ExecutorSettings`.
>
> ```
> private static ExecutorSettings configToSettings(ExecutorConfig config) {
> return new ExecutorSettings(...);
> }
> ```
>
> Bill Farner wrote:
> I took a quick stab at illustrating this, take a look here: https://github.com/wfarner/aurora/commit/a53c215432b34ba95238121581375bde24b88a1e
>
> The remaining work is to complete filling out constructor fields:
> https://github.com/wfarner/aurora/blob/a53c215432b34ba95238121581375bde24b88a1e/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java#L53-L61
>
> This is really a matter of writing functions to translate from types in `ExecutorConfig` to the corresponding types in `ExecutorSettings`.
>
> Depending on your bandwidth over the next few days, i may take over this patch if you don't mind.
>
> Renan DelValle wrote:
> Thanks for the examples, in particular the entry transformer really helped speed things along.
>
> I've almost got this done, and I'm having an issue with using optionals, which in my opinion are necessary if we want to leave the defaults set on the URI resource configuration if nothing is specified. I.e: If executable is not in the schema for a URI, then its better to not set it than to set a default value, IMO.
>
> I've looked around and Jackson has support for them if we load a new module:
> https://github.com/FasterXML/jackson-datatype-jdk8
>
> However, I can't find where the jackson dependecy is. And the fact the jackson pacakges still point to codehaus lead me to believe were using an old version.
>
> Here are some relevant files:
> https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorConfiguration.java
> https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
>
> We're almost there.
>
> Bill Farner wrote:
> Perhaps we should go a step further and try to directly parse `ExecutorInfo`? That way we don't have to maintain parity between the protobuf schema and our own. Jackson can't directly deserialize protobuf-generated objects, but a quick search turned up this promising project: https://github.com/HubSpot/jackson-datatype-protobuf
>
> The deps are defined in build.gradle. You're right about the old jackson library (pulled in by an old jersey-json). However, the fact that they changed packages plays in our favor - we can add an explicit dep on the new jackson version and use the new one in this part of the code.
>
> Bill Farner wrote:
> I put together a proof of concept of this idea here: https://github.com/wfarner/aurora/commit/04e1ff6ad566755823b624303cfd66541076785b
>
> You'll notice that this makes the json more verbose, but it becomes a 1:1 association with the protobuf message.
Awesome, thanks for that. It's a WIP at the moment, but I just wanted to share what I have so far so I'm not going off on some trail I shouldn't be going on.
I cribbed some of the code from your POC and came up with this. I'm still working on fixing all the tests that changing the ExecutorSettings broke, but after that it should be good to go.
https://github.com/rdelval/aurora/commit/58e4524ab20698d6c665d7736dadbcac651d445e#diff-ca4de6b05130534619820a5f6a521f2d
https://github.com/rdelval/aurora/commit/38958251833566589d448c9f3a19096201430605#diff-ca4de6b05130534619820a5f6a521f2d
You'll notice I condensed the command list and URIs into a CommandBuilder and I changed the Volume from the Aurora one to the Mesos Protobuf one, I hope this last change isn't an issue, if it is, let me know.
With these changes IMO, src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java is no longer needed, at least I haven't found anywhere where it would be a benefit to have it.
- Renan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 8, 2015, 4:32 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 8, 2015, 4:32 p.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Bill Farner <wf...@apache.org>.
> On Sept. 2, 2015, 9:45 a.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java, line 87
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061926#file1061926line87>
> >
> > This code will be easier to maintain long-term with 2 goals in mind:
> > - keep json-related code within this class (no annotations on ExecutorSettings)
> > - avoid hand-rolling deserializers
> >
> > This sounds like more work, but it's not that bad. The major difference is that you'll create a class in this file that defines the JSON structure, and does so in a way that the json deserializer can work with.
> >
> > Maxim just did something very similar, which you can crib from:
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55
Oh, the detail i neglected to mention - i suggest you then write code to translate your config schema object into ExecutorSettings. This can provides compile-time guarantees that we don't have with the current code, making it easier to maintain.
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 1, 2015, 7:35 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 1, 2015, 7:35 p.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Renan DelValle <rd...@binghamton.edu>.
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, lines 4-7
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061919#file1061919line4>
> >
> > The code later converts this array into a single command string. I suggest we just make this a string here (and save some code down the line).
>
> Renan DelValle wrote:
> We can go ahead and do that. Kevin thought it should be the array way, but I'm not sure how strongly he feels about it. The extra code isn't too bad because we can use String.join() form Java 8. Either way, I'm fine with either. The single string may eliminate any suspicions that we're breaking something in a command when joining the array.
>
> Kevin Sweeney wrote:
> Bill, I prefer the array mode as it's more explicit for dealing with magic characters in shell commands (like " (){}$").
>
> You can still explicitly do
>
> ```
> ["sh", "-c", "PATH=/opt/aurora/executor/bin:$PATH thermos"]
> ```
>
> When you prefer the old form, but this form will not automatically expand environment variables or do other shell magic for you.
>
> Renan - can you update CommandUtil to use the `arguments`[1] field instead of `shell` and `value` (or drop a TODO for a later patch)? Changing back to `String.join` defeats part of the purpose of the array form here.
>
> [1] https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L408-L423
>
> Bill Farner wrote:
> Aha - the `arguments` field in mesos.proto was the missing detail for me. Since mesos allows us to do the right thing here, we should definitely use an array.
I gave this a shot, and it works fine for no container tasks. But, when running a Docker container job, since the $MESOS_SANDBOX string is no longer automatically expanding, it breaks. Should we just create a new CommandInfo builder for docker that sets "sh" as the value and "-c" as the first argument or is this a bad idea?
- Renan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 2, 2015, 2:35 a.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 2, 2015, 2:35 a.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Bill Farner <wf...@apache.org>.
> On Sept. 2, 2015, 9:45 a.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java, line 87
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061926#file1061926line87>
> >
> > This code will be easier to maintain long-term with 2 goals in mind:
> > - keep json-related code within this class (no annotations on ExecutorSettings)
> > - avoid hand-rolling deserializers
> >
> > This sounds like more work, but it's not that bad. The major difference is that you'll create a class in this file that defines the JSON structure, and does so in a way that the json deserializer can work with.
> >
> > Maxim just did something very similar, which you can crib from:
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55
>
> Bill Farner wrote:
> Oh, the detail i neglected to mention - i suggest you then write code to translate your config schema object into ExecutorSettings. This can provides compile-time guarantees that we don't have with the current code, making it easier to maintain.
>
> Renan DelValle wrote:
> Can you expend on this? I don't have any experience with Jackson. I'm looking through what Maxim did and I'll definitely crib here and there but I can't find anything related to getting this done at compile time.
>
> Bill Farner wrote:
> Don't be thrown by the mention of compile time, i'm just referring to refactor safety if/when the structures changed. The concise way to put it is - use a different class for your config domain object. This way you don't need to change ExecutorSettings to compensate for the fact that some code in another part of the system is using a json mapper.
>
> Start by defining a class that matches your config schema:
> ```
> class ExecutorConfig {
> String command;
> List<Resources> resource;
> ...
> }
>
> class Resource {
> String value; // note - i would like to see this renamed to uri
> boolean executable;
> boolean extract;
> boolean cache;
> }
> ...
> ```
> (that's just a quick example, feel free to adjust as you see fit)
>
> Jackson is very similar to gson, there's just a behavior nuance that is desirable. You'll use an `ObjectMapper` to parse the json to an `ExecutorConfig`, then write a function to transform your `ExecutorConfig` into `ExecutorSettings`.
>
> ```
> private static ExecutorSettings configToSettings(ExecutorConfig config) {
> return new ExecutorSettings(...);
> }
> ```
>
> Bill Farner wrote:
> I took a quick stab at illustrating this, take a look here: https://github.com/wfarner/aurora/commit/a53c215432b34ba95238121581375bde24b88a1e
>
> The remaining work is to complete filling out constructor fields:
> https://github.com/wfarner/aurora/blob/a53c215432b34ba95238121581375bde24b88a1e/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java#L53-L61
>
> This is really a matter of writing functions to translate from types in `ExecutorConfig` to the corresponding types in `ExecutorSettings`.
>
> Depending on your bandwidth over the next few days, i may take over this patch if you don't mind.
>
> Renan DelValle wrote:
> Thanks for the examples, in particular the entry transformer really helped speed things along.
>
> I've almost got this done, and I'm having an issue with using optionals, which in my opinion are necessary if we want to leave the defaults set on the URI resource configuration if nothing is specified. I.e: If executable is not in the schema for a URI, then its better to not set it than to set a default value, IMO.
>
> I've looked around and Jackson has support for them if we load a new module:
> https://github.com/FasterXML/jackson-datatype-jdk8
>
> However, I can't find where the jackson dependecy is. And the fact the jackson pacakges still point to codehaus lead me to believe were using an old version.
>
> Here are some relevant files:
> https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorConfiguration.java
> https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
>
> We're almost there.
>
> Bill Farner wrote:
> Perhaps we should go a step further and try to directly parse `ExecutorInfo`? That way we don't have to maintain parity between the protobuf schema and our own. Jackson can't directly deserialize protobuf-generated objects, but a quick search turned up this promising project: https://github.com/HubSpot/jackson-datatype-protobuf
>
> The deps are defined in build.gradle. You're right about the old jackson library (pulled in by an old jersey-json). However, the fact that they changed packages plays in our favor - we can add an explicit dep on the new jackson version and use the new one in this part of the code.
>
> Bill Farner wrote:
> I put together a proof of concept of this idea here: https://github.com/wfarner/aurora/commit/04e1ff6ad566755823b624303cfd66541076785b
>
> You'll notice that this makes the json more verbose, but it becomes a 1:1 association with the protobuf message.
>
> Renan DelValle wrote:
> Awesome, thanks for that. It's a WIP at the moment, but I just wanted to share what I have so far so I'm not going off on some trail I shouldn't be going on.
>
> I cribbed some of the code from your POC and came up with this. I'm still working on fixing all the tests that changing the ExecutorSettings broke, but after that it should be good to go.
>
> https://github.com/rdelval/aurora/commit/58e4524ab20698d6c665d7736dadbcac651d445e#diff-ca4de6b05130534619820a5f6a521f2d
> https://github.com/rdelval/aurora/commit/38958251833566589d448c9f3a19096201430605#diff-ca4de6b05130534619820a5f6a521f2d
>
> You'll notice I condensed the command list and URIs into a CommandBuilder and I changed the Volume from the Aurora one to the Mesos Protobuf one, I hope this last change isn't an issue, if it is, let me know.
>
> With these changes IMO, src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java is no longer needed, at least I haven't found anywhere where it would be a benefit to have it.
The protobuf `Volume` object is the right thing to use. The most important thing i would suggest is that you remove `ExecutorSettings`, and let the protobuf go all the way to where it is used in `TaskInfo`.
If CommandUtil is no longer used, it should indeed be removed.
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 8, 2015, 9:32 a.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 8, 2015, 9:32 a.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Bill Farner <wf...@apache.org>.
> On Sept. 2, 2015, 9:45 a.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, lines 4-7
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061919#file1061919line4>
> >
> > The code later converts this array into a single command string. I suggest we just make this a string here (and save some code down the line).
>
> Renan DelValle wrote:
> We can go ahead and do that. Kevin thought it should be the array way, but I'm not sure how strongly he feels about it. The extra code isn't too bad because we can use String.join() form Java 8. Either way, I'm fine with either. The single string may eliminate any suspicions that we're breaking something in a command when joining the array.
>
> Kevin Sweeney wrote:
> Bill, I prefer the array mode as it's more explicit for dealing with magic characters in shell commands (like " (){}$").
>
> You can still explicitly do
>
> ```
> ["sh", "-c", "PATH=/opt/aurora/executor/bin:$PATH thermos"]
> ```
>
> When you prefer the old form, but this form will not automatically expand environment variables or do other shell magic for you.
>
> Renan - can you update CommandUtil to use the `arguments`[1] field instead of `shell` and `value` (or drop a TODO for a later patch)? Changing back to `String.join` defeats part of the purpose of the array form here.
>
> [1] https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L408-L423
>
> Bill Farner wrote:
> Aha - the `arguments` field in mesos.proto was the missing detail for me. Since mesos allows us to do the right thing here, we should definitely use an array.
>
> Renan DelValle wrote:
> I gave this a shot, and it works fine for no container tasks. But, when running a Docker container job, since the $MESOS_SANDBOX string is no longer automatically expanding, it breaks. Should we just create a new CommandInfo builder for docker that sets "sh" as the value and "-c" as the first argument or is this a bad idea?
Kevin - any opinion on this?
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 8, 2015, 9:32 a.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 8, 2015, 9:32 a.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Renan DelValle <rd...@binghamton.edu>.
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java, line 87
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061926#file1061926line87>
> >
> > This code will be easier to maintain long-term with 2 goals in mind:
> > - keep json-related code within this class (no annotations on ExecutorSettings)
> > - avoid hand-rolling deserializers
> >
> > This sounds like more work, but it's not that bad. The major difference is that you'll create a class in this file that defines the JSON structure, and does so in a way that the json deserializer can work with.
> >
> > Maxim just did something very similar, which you can crib from:
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
> > https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55
>
> Bill Farner wrote:
> Oh, the detail i neglected to mention - i suggest you then write code to translate your config schema object into ExecutorSettings. This can provides compile-time guarantees that we don't have with the current code, making it easier to maintain.
>
> Renan DelValle wrote:
> Can you expend on this? I don't have any experience with Jackson. I'm looking through what Maxim did and I'll definitely crib here and there but I can't find anything related to getting this done at compile time.
>
> Bill Farner wrote:
> Don't be thrown by the mention of compile time, i'm just referring to refactor safety if/when the structures changed. The concise way to put it is - use a different class for your config domain object. This way you don't need to change ExecutorSettings to compensate for the fact that some code in another part of the system is using a json mapper.
>
> Start by defining a class that matches your config schema:
> ```
> class ExecutorConfig {
> String command;
> List<Resources> resource;
> ...
> }
>
> class Resource {
> String value; // note - i would like to see this renamed to uri
> boolean executable;
> boolean extract;
> boolean cache;
> }
> ...
> ```
> (that's just a quick example, feel free to adjust as you see fit)
>
> Jackson is very similar to gson, there's just a behavior nuance that is desirable. You'll use an `ObjectMapper` to parse the json to an `ExecutorConfig`, then write a function to transform your `ExecutorConfig` into `ExecutorSettings`.
>
> ```
> private static ExecutorSettings configToSettings(ExecutorConfig config) {
> return new ExecutorSettings(...);
> }
> ```
>
> Bill Farner wrote:
> I took a quick stab at illustrating this, take a look here: https://github.com/wfarner/aurora/commit/a53c215432b34ba95238121581375bde24b88a1e
>
> The remaining work is to complete filling out constructor fields:
> https://github.com/wfarner/aurora/blob/a53c215432b34ba95238121581375bde24b88a1e/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java#L53-L61
>
> This is really a matter of writing functions to translate from types in `ExecutorConfig` to the corresponding types in `ExecutorSettings`.
>
> Depending on your bandwidth over the next few days, i may take over this patch if you don't mind.
Thanks for the examples, in particular the entry transformer really helped speed things along.
I've almost got this done, and I'm having an issue with using optionals, which in my opinion are necessary if we want to leave the defaults set on the URI resource configuration if nothing is specified. I.e: If executable is not in the schema for a URI, then its better to not set it than to set a default value, IMO.
I've looked around and Jackson has support for them if we load a new module:
https://github.com/FasterXML/jackson-datatype-jdk8
However, I can't find where the jackson dependecy is. And the fact the jackson pacakges still point to codehaus lead me to believe were using an old version.
Here are some relevant files:
https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorConfiguration.java
https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java
We're almost there.
> On Sept. 2, 2015, 4:45 p.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, lines 4-7
> > <https://reviews.apache.org/r/37818/diff/3/?file=1061919#file1061919line4>
> >
> > The code later converts this array into a single command string. I suggest we just make this a string here (and save some code down the line).
>
> Renan DelValle wrote:
> We can go ahead and do that. Kevin thought it should be the array way, but I'm not sure how strongly he feels about it. The extra code isn't too bad because we can use String.join() form Java 8. Either way, I'm fine with either. The single string may eliminate any suspicions that we're breaking something in a command when joining the array.
>
> Kevin Sweeney wrote:
> Bill, I prefer the array mode as it's more explicit for dealing with magic characters in shell commands (like " (){}$").
>
> You can still explicitly do
>
> ```
> ["sh", "-c", "PATH=/opt/aurora/executor/bin:$PATH thermos"]
> ```
>
> When you prefer the old form, but this form will not automatically expand environment variables or do other shell magic for you.
>
> Renan - can you update CommandUtil to use the `arguments`[1] field instead of `shell` and `value` (or drop a TODO for a later patch)? Changing back to `String.join` defeats part of the purpose of the array form here.
>
> [1] https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L408-L423
>
> Bill Farner wrote:
> Aha - the `arguments` field in mesos.proto was the missing detail for me. Since mesos allows us to do the right thing here, we should definitely use an array.
>
> Renan DelValle wrote:
> I gave this a shot, and it works fine for no container tasks. But, when running a Docker container job, since the $MESOS_SANDBOX string is no longer automatically expanding, it breaks. Should we just create a new CommandInfo builder for docker that sets "sh" as the value and "-c" as the first argument or is this a bad idea?
>
> Bill Farner wrote:
> Kevin - any opinion on this?
This is what I came up with, not ideal by any means, but I think it's something we can address later:
https://github.com/rdelval/aurora/blob/AURORA-1376-1d/src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java
- Renan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
On Sept. 8, 2015, 4:32 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 8, 2015, 4:32 p.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97468
-----------------------------------------------------------
examples/vagrant/executors-config.json (lines 4 - 7)
<https://reviews.apache.org/r/37818/#comment153348>
The code later converts this array into a single command string. I suggest we just make this a string here (and save some code down the line).
examples/vagrant/executors-config.json (lines 17 - 18)
<https://reviews.apache.org/r/37818/#comment153342>
Can you omit this since it's blank?
examples/vagrant/executors-config.json (line 32)
<https://reviews.apache.org/r/37818/#comment153343>
Use single quotes to avoid escaping:
"'Hello World from Aurora!'"
src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java
<https://reviews.apache.org/r/37818/#comment153344>
Please add an entry in /NEWS under 0.10.0 to inform people what args were removed, and what the replacement is.
src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java (line 166)
<https://reviews.apache.org/r/37818/#comment153345>
throw Throwables.propagate(e);
Otherwise the app will exit in a non-obvious way.
src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java (lines 61 - 63)
<https://reviews.apache.org/r/37818/#comment153346>
remove
src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java (lines 95 - 96)
<https://reviews.apache.org/r/37818/#comment153347>
I suggest removing this for now. There's a lot of ways fetching the URIs and executing commands can fail. For the sake of simplicity, let's allow them to fail and focus on providing good visibility into the failure.
src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java (line 46)
<https://reviews.apache.org/r/37818/#comment153349>
please add a javadoc for this class
src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java (line 62)
<https://reviews.apache.org/r/37818/#comment153351>
Save a few lines:
```
String config = Files.toString(configFile, StandardCharsets.UTF_8);
```
src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java (lines 66 - 71)
<https://reviews.apache.org/r/37818/#comment153352>
You might want to consider using jackson instead of json. We recently learned that gson will silently ignore unexpected json properties, which is not behavior we want for a config file. This doesn't change much for you, see pointers in comment below for an example.
src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java (line 79)
<https://reviews.apache.org/r/37818/#comment153350>
remove the ```\n```, same with the string below
src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java (line 87)
<https://reviews.apache.org/r/37818/#comment153360>
This code will be easier to maintain long-term with 2 goals in mind:
- keep json-related code within this class (no annotations on ExecutorSettings)
- avoid hand-rolling deserializers
This sounds like more work, but it's not that bad. The major difference is that you'll create a class in this file that defines the JSON structure, and does so in a way that the json deserializer can work with.
Maxim just did something very similar, which you can crib from:
https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/SchedulerModule.java#L132-L147
https://github.com/apache/aurora/blob/89da936f3d28743e307c7c4fed0bff6fead7ceca/src/main/java/org/apache/aurora/scheduler/TierManager.java#L47-L55
- Bill Farner
On Sept. 1, 2015, 7:35 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 1, 2015, 7:35 p.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Renan DelValle <rd...@binghamton.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/
-----------------------------------------------------------
(Updated Sept. 2, 2015, 2:35 a.m.)
Review request for Aurora.
Changes
-------
Added support for loading map directly from JSON file, modified JSON schema to look like a Map, changed config object in ExecutorSettings to contain a JsonElement for increased flexibility, added more robust testing for ExecutorSettingsLoader. Included an example for the Mesos Command executor. However, mesos command executor won't be functional until another patch is submitted and commited.
@ReviewBot retry
Repository: aurora
Description
-------
This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
Diffs (updated)
-----
examples/vagrant/executors-config.json PRE-CREATION
examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
Diff: https://reviews.apache.org/r/37818/diff/
Testing
-------
./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
Thanks,
Renan DelValle
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Renan DelValle <rd...@binghamton.edu>.
> On Sept. 1, 2015, 2:35 a.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, line 1
> > <https://reviews.apache.org/r/37818/diff/2/?file=1060851#file1060851line1>
> >
> > Can you expand this example to include the command executor? That's likely to be the first non-default executor folks will want to try.
Sure thing. Although, be advised that the functionality for custom executors is not included in this patch. This patch only lays the ground work. The next patch will include the ability to actually select them and run them.
- Renan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97231
-----------------------------------------------------------
On Sept. 1, 2015, 12:37 a.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 1, 2015, 12:37 a.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 0c440b5cd5b939872c1ee05d048bf739bfa977cb
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/thermos-settings-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Bill Farner <wf...@apache.org>.
> On Aug. 31, 2015, 7:35 p.m., Bill Farner wrote:
> > examples/vagrant/executors-config.json, line 1
> > <https://reviews.apache.org/r/37818/diff/2/?file=1060851#file1060851line1>
> >
> > Can you expand this example to include the command executor? That's likely to be the first non-default executor folks will want to try.
>
> Renan DelValle wrote:
> Sure thing. Although, be advised that the functionality for custom executors is not included in this patch. This patch only lays the ground work. The next patch will include the ability to actually select them and run them.
Cool, that's actually what i wanted to see - a format change. If we want the format to support multiple executors, we should use that format now so it works once multiple executors _are_ supported.
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97231
-----------------------------------------------------------
On Sept. 1, 2015, 7:35 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 1, 2015, 7:35 p.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 8047622e206c9827e5cd8e40152a278d495bd0ff
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/multiple-executor-example.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/single-executor-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97231
-----------------------------------------------------------
examples/vagrant/executors-config.json (line 1)
<https://reviews.apache.org/r/37818/#comment153053>
Can you expand this example to include the command executor? That's likely to be the first non-default executor folks will want to try.
- Bill Farner
On Aug. 31, 2015, 5:37 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Aug. 31, 2015, 5:37 p.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 0c440b5cd5b939872c1ee05d048bf739bfa977cb
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/thermos-settings-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review97220
-----------------------------------------------------------
Ship it!
Master (c8e65d3) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Sept. 1, 2015, 12:37 a.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Sept. 1, 2015, 12:37 a.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
> examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
> src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 0c440b5cd5b939872c1ee05d048bf739bfa977cb
> src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
> src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
> src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
> src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/thermos-settings-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Renan DelValle <rd...@binghamton.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/
-----------------------------------------------------------
(Updated Sept. 1, 2015, 12:37 a.m.)
Review request for Aurora.
Changes
-------
New schema implemented with support for URIs as outlined by the Mesos spec.
Separation of command to be executed from resources to be fetched in preparation for support for cusotm executors.
Deserializers changed to support URI objects in schema.
Empty or non-existent globalContainerMounts is represented as an empty list in ExecutorSettings.
@ReviewBot retry
Repository: aurora
Description
-------
This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
Diffs (updated)
-----
examples/vagrant/executors-config.json PRE-CREATION
examples/vagrant/upstart/aurora-scheduler-kerberos.conf 4f43892723db4744db205ea7dd107e9e9ce9d5db
examples/vagrant/upstart/aurora-scheduler.conf e909451892f117e9e6eb80994079661827a0914c
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java c210c0db07bb1f4b3f76668178dcd7e2de56a4ac
src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java 197184b6edc0768d677636341b5737f262abdf7d
src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 0c440b5cd5b939872c1ee05d048bf739bfa977cb
src/main/java/org/apache/aurora/scheduler/base/CommandUtil.java aa5ce8b2f14c7dbd0eae120018ee41387c26059f
src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java f6ba2c40aea555d3e0ab774218bfe08d7e1c984b
src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java 6fad3344042dc6a75cdf74ce79d388fcd4fc9861
src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 1a25924d789295c5950947f5e302e1d1fbec68f2
src/test/java/org/apache/aurora/scheduler/base/CommandUtilTest.java cd0295780d41bc4e914583f195b37eaed28a46dc
src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java dddf7952d3f0e508cd736d5fb95e573267708d43
src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java d0987251b058988fcbfab16c1b138c37e0c5b8c6
src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
src/test/resources/org/apache/aurora/scheduler/configuration/no-value-URI.json PRE-CREATION
src/test/resources/org/apache/aurora/scheduler/configuration/thermos-settings-example.json PRE-CREATION
Diff: https://reviews.apache.org/r/37818/diff/
Testing
-------
./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
Thanks,
Renan DelValle
Re: Review Request 37818: Moved executor settings configuration to
loadable JSON
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37818/#review96733
-----------------------------------------------------------
Ship it!
Master (06ddaad) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Aug. 26, 2015, 10:19 p.m., Renan DelValle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37818/
> -----------------------------------------------------------
>
> (Updated Aug. 26, 2015, 10:19 p.m.)
>
>
> Review request for Aurora.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This is the first stage in a series of patches to create support for custom executors. In an effort to expedite the review process, I have decided to break down my patch into multiple pieces that when/if commited won't break the trunk.
>
> This patch includes the ability to load configuration from a JSON file. A JSON example file is included in examples/vagrant/executors-config.json
>
> Command line arguments have been eliminated and moved over to the JSON file. GSON is leveraged and does most of the work with the aid of a few custom deserializers that were needed.
>
> Note that right now a global container mount that does not follow specification will cause the scheduler to detect the error an exit early. It is up for discussion if this is the desired behavior or if we should just ignore said mount.
>
>
> Diffs
> -----
>
> examples/vagrant/executors-config-new.json PRE-CREATION
> examples/vagrant/executors-config.json PRE-CREATION
> examples/vagrant/upstart/aurora-scheduler-kerberos.conf 744b4a35c61e749734e222b3d4cbd296927665aa
> examples/vagrant/upstart/aurora-scheduler.conf 789a3a0315e8530880999432aa9b1e7d0f57d1ff
> src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 0c440b5cd5b939872c1ee05d048bf739bfa977cb
> src/main/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoader.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java b3c913892248e4a9a8111412307463985f5ca97f
> src/test/java/org/apache/aurora/scheduler/configuration/ExecutorSettingsLoaderTest.java PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/executor-settings-thermos-no-observer.json PRE-CREATION
> src/test/resources/org/apache/aurora/scheduler/configuration/thermos-settings-example.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37818/diff/
>
>
> Testing
> -------
>
> ./build-support/jenkins/build.sh: directory sandbox failed but it may be a flaky test
> bash src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Renan DelValle
>
>