You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@amaterasu.apache.org by ya...@apache.org on 2018/03/27 05:43:17 UTC
[incubator-amaterasu] branch master updated (0d8664c -> 4409529)
This is an automated email from the ASF dual-hosted git repository.
yaniv pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-amaterasu.git.
from 0d8664c Merge pull request #12 from annoymousGiraf/master
add 3c6cf0f added logic to read ama-workers / spark drivers(am) from configuration. Spark is first, second amaterasu configs, third is defaults. check against maximum cluster resources is also in place.
add b108c7a added driver configuration to the SDK
add 3028980 removed .gitignore-ception refactored execData in sparkrunnersprovider added future comment but mainly added logic in appmaster to handle spark-specific configurations
add 279d6c7 added spark framework providers support and moved logic there.
add f8e8999 driverConfiguration not cupled to spark
add 3d51596 get cpus return cpus now :)
add dad66bc fixed some bugs
add 2d7e603 handling memory formats
new 4409529 Merge pull request #13 from eyalbenivri/AMATERASU-18
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.gitignore | 2 +-
.../common/configuration/ClusterConfig.scala | 4 +-
.../amaterasu/common/dataobjects/TaskData.scala | 4 ++
.../runners/spark/SparkRunnersProvider.scala | 26 +++++------
.../org/apache/amaterasu/leader/yarn/Client.java | 17 +++----
.../frameworks/FrameworkProvidersFactory.scala | 8 ++--
.../frameworks/spark/SparkSetupProvider.scala | 53 ++++++++++++++++++++--
.../leader/mesos/schedulers/JobScheduler.scala | 9 +++-
.../leader/utilities/MemoryFormatParser.scala | 18 ++++++++
.../amaterasu/leader/yarn/ApplicationMaster.scala | 28 ++++++++----
.../{ => frameworks}/FrameworkSetupProvider.java | 7 ++-
.../configuration/DriverConfiguration.java | 21 +++++++++
12 files changed, 148 insertions(+), 49 deletions(-)
create mode 100644 leader/src/main/scala/org/apache/amaterasu/leader/utilities/MemoryFormatParser.scala
rename sdk/src/main/java/org/apache/amaterasu/sdk/{ => frameworks}/FrameworkSetupProvider.java (52%)
create mode 100644 sdk/src/main/java/org/apache/amaterasu/sdk/frameworks/configuration/DriverConfiguration.java
--
To stop receiving notification emails like this one, please contact
yaniv@apache.org.
[incubator-amaterasu] 01/01: Merge pull request #13 from
eyalbenivri/AMATERASU-18
Posted by ya...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
yaniv pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-amaterasu.git
commit 4409529309fe60313087007f49b474d3b0fefbc6
Merge: 0d8664c 2d7e603
Author: Yaniv Rodenski <ro...@gmail.com>
AuthorDate: Tue Mar 27 16:43:15 2018 +1100
Merge pull request #13 from eyalbenivri/AMATERASU-18
AMATERASU-18
.gitignore | 2 +-
.../common/configuration/ClusterConfig.scala | 4 +-
.../amaterasu/common/dataobjects/TaskData.scala | 4 ++
.../runners/spark/SparkRunnersProvider.scala | 26 +++++------
.../org/apache/amaterasu/leader/yarn/Client.java | 17 +++----
.../frameworks/FrameworkProvidersFactory.scala | 8 ++--
.../frameworks/spark/SparkSetupProvider.scala | 53 ++++++++++++++++++++--
.../leader/mesos/schedulers/JobScheduler.scala | 9 +++-
.../leader/utilities/MemoryFormatParser.scala | 18 ++++++++
.../amaterasu/leader/yarn/ApplicationMaster.scala | 28 ++++++++----
.../{ => frameworks}/FrameworkSetupProvider.java | 7 ++-
.../configuration/DriverConfiguration.java | 21 +++++++++
12 files changed, 148 insertions(+), 49 deletions(-)
--
To stop receiving notification emails like this one, please contact
yaniv@apache.org.