You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ambari.apache.org by ol...@apache.org on 2017/05/31 18:07:33 UTC
ambari git commit: AMBARI-21155. Design Ambari Infra Component &
workflows (oleewere)
Repository: ambari
Updated Branches:
refs/heads/trunk 753f8aacc -> 23e23afd8
AMBARI-21155. Design Ambari Infra Component & workflows (oleewere)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/23e23afd
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/23e23afd
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/23e23afd
Branch: refs/heads/trunk
Commit: 23e23afd884449812d05c490804dc4845b01ac21
Parents: 753f8aa
Author: oleewere <ol...@gmail.com>
Authored: Wed May 31 12:21:34 2017 +0200
Committer: oleewere <ol...@gmail.com>
Committed: Wed May 31 20:03:27 2017 +0200
----------------------------------------------------------------------
ambari-infra/ambari-infra-manager/README.md | 92 ++-
.../ambari-infra-manager/docs/api/swagger.yaml | 784 +++++++++++++++++++
.../docs/images/batch-1.png | Bin 0 -> 20521 bytes
.../docs/images/batch-2.png | Bin 0 -> 29388 bytes
.../docs/images/batch-3.png | Bin 0 -> 14105 bytes
.../docs/images/batch-4.png | Bin 0 -> 23277 bytes
.../infra/common/InfraManagerConstants.java | 2 +-
.../infra/conf/InfraManagerApiDocConfig.java | 35 +-
.../conf/batch/InfraManagerBatchConfig.java | 8 +-
.../ambari/infra/job/dummy/DummyItemWriter.java | 13 +
.../infra/job/dummy/DummyJobListener.java | 39 +
.../infra/job/dummy/DummyStepListener.java | 41 +
.../apache/ambari/infra/rest/JobResource.java | 2 +-
13 files changed, 1002 insertions(+), 14 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/README.md
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/README.md b/ambari-infra/ambari-infra-manager/README.md
index d3527c4..dd2854c 100644
--- a/ambari-infra/ambari-infra-manager/README.md
+++ b/ambari-infra/ambari-infra-manager/README.md
@@ -18,13 +18,99 @@ limitations under the License.
-->
# Ambari Infra Manager
-TODO
-## Build & Run Application
+
+## Overview
+
+Ambari Infra Manager is a REST based management application for Ambari Infra services (like Infra Solr). The API is built on top of [Spring Batch] (http://docs.spring.io/spring-batch/reference/html/)
+
+### Architecture
+
+
+### Job execution overview
+
+
+### Job workflow
+
+
+### Step workflow
+
+
+(images originally from [here] (http://docs.spring.io/spring-batch/reference/html/))
+
+## API documentation
+
+Infra Manager uses [Swagger] (http://swagger.io/), generated yaml file can be downloaded from [here] (docs/api/swagger.yaml)
+
+
+## Development guide
+
+### Adding a new custom job
+
+As Infra Manager is a Spring based application and using Java configurations, if it is needed to add a new custom Job, the Jobs/Steps/Configurations are need to be on the classpath. Spring beans are registered only in a specific package, so for writing a plugin, all the added Java classes needs to be added inside "org.apache.ambari.infra" package.
+
+For the plugin it will be needed to add all Spring & Spring batch dependencies. For adding a new Job you will need to define a new Configuration object. There you can define your own jobs/steps/writers/readers/processors, as you can see in that example:
+```java
+@Configuration
+@EnableBatchProcessing
+public class MyJobConfig {
+
+ @Inject
+ private StepBuilderFactory steps;
+
+ @Inject
+ private JobBuilderFactory jobs;
+
+
+ @Bean(name = "dummyStep")
+ protected Step dummyStep(ItemReader<DummyObject> reader,
+ ItemProcessor<DummyObject, String> processor,
+ ItemWriter<String> writer) {
+ return steps.get("dummyStep").listener(new DummyStepListener()).<DummyObject, String> chunk(2)
+ .reader(reader).processor(processor).writer(writer).build();
+ }
+
+ @Bean(name = "dummyJob")
+ public Job job(@Qualifier("dummyStep") Step dummyStep) {
+ return jobs.get("dummyJob").listener(new DummyJobListener()).start(dummyStep).build();
+ }
+
+}
+```
+As you can see it will require to implement [ItemWriter] (https://docs.spring.io/spring-batch/apidocs/org/springframework/batch/item/ItemWriter.html), [ItemReader] (http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/item/ItemReader.html) and [ItemProcessor] (https://docs.spring.io/spring-batch/apidocs/org/springframework/batch/item/ItemProcessor.html)
+
+### Schedule custom jobs
+
+It can be needed based on business requirements to schedule jobs (e.g. daily) instead of run manually through the REST API. It can be done with adding a custom bean to "org.apache.ambari.infra" package with using [@Scheduled] (http://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/annotation/Scheduled.html):
+```java
+@Named
+public class MySchedulerObject {
+
+ @Inject
+ private JobService jobService; // or JobOperator jobOperator if spring-batch-admin manager dependecy is not included
+
+ @Value("${infra-manager.batch.my.param:defaultString}")
+ private String myParamFromLogSearchProperties;
+
+ @Scheduled(cron = "*/5 * * * * MON-FRI")
+ public void doSomething() {
+ // setup job params
+ jobService.launch(jobName, jobParameters, TimeZone.getDefault());
+ }
+
+ @Scheduled(cron = "${infra.manager.my.prop}")
+ public void doSomethingBasedOnInfraProperty() {
+ // do something ...
+ }
+}
+```
+
+You can put your cron expression inside infra-manager.properties file just make it configuratble.
+### Build & Run Application
```bash
mvn clean package exec:java
```
-## Build & Run Application in docker container
+### Build & Run Application in docker container
```bash
cd docker
./infra-manager-docker.sh
http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/docs/api/swagger.yaml
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/docs/api/swagger.yaml b/ambari-infra/ambari-infra-manager/docs/api/swagger.yaml
new file mode 100644
index 0000000..824629f
--- /dev/null
+++ b/ambari-infra/ambari-infra-manager/docs/api/swagger.yaml
@@ -0,0 +1,784 @@
+---
+swagger: "2.0"
+info:
+ description: "Manager component for Ambari Infra"
+ version: "1.0.0"
+ title: "Infra Manager REST API"
+ license:
+ name: "Apache 2.0"
+ url: "http://www.apache.org/licenses/LICENSE-2.0.html"
+basePath: "/api/v1"
+tags:
+- name: "jobs"
+schemes:
+- "http"
+- "https"
+paths:
+ /jobs:
+ get:
+ tags:
+ - "jobs"
+ summary: "Get all jobs"
+ description: ""
+ operationId: "getAllJobs"
+ produces:
+ - "application/json"
+ parameters:
+ - name: "page"
+ in: "query"
+ required: false
+ type: "integer"
+ default: 0
+ format: "int32"
+ - name: "size"
+ in: "query"
+ required: false
+ type: "integer"
+ default: 20
+ format: "int32"
+ responses:
+ 200:
+ description: "successful operation"
+ schema:
+ type: "array"
+ items:
+ $ref: "#/definitions/JobInfo"
+ /jobs/executions:
+ delete:
+ tags:
+ - "jobs"
+ summary: "Stop all job executions."
+ description: ""
+ operationId: "stopAll"
+ produces:
+ - "application/json"
+ parameters: []
+ responses:
+ 200:
+ description: "successful operation"
+ schema:
+ type: "integer"
+ format: "int32"
+ /jobs/executions/{jobExecutionId}:
+ get:
+ tags:
+ - "jobs"
+ summary: "Get job and step details for job execution instance."
+ description: ""
+ operationId: "getExectionInfo"
+ produces:
+ - "application/json"
+ parameters:
+ - name: "jobExecutionId"
+ in: "path"
+ required: true
+ type: "integer"
+ format: "int64"
+ responses:
+ 200:
+ description: "successful operation"
+ schema:
+ $ref: "#/definitions/JobExecutionDetailsResponse"
+ delete:
+ tags:
+ - "jobs"
+ summary: "Stop or abandon a running job execution."
+ description: ""
+ operationId: "stopOrAbandonJobExecution"
+ produces:
+ - "application/json"
+ parameters:
+ - name: "jobExecutionId"
+ in: "path"
+ required: true
+ type: "integer"
+ format: "int64"
+ - name: "operation"
+ in: "query"
+ required: true
+ type: "string"
+ enum:
+ - "STOP"
+ - "ABANDON"
+ responses:
+ 200:
+ description: "successful operation"
+ schema:
+ $ref: "#/definitions/JobExecutionInfoResponse"
+ /jobs/executions/{jobExecutionId}/context:
+ get:
+ tags:
+ - "jobs"
+ summary: "Get execution context for specific job."
+ description: ""
+ operationId: "getExecutionContextByJobExecId"
+ produces:
+ - "application/json"
+ parameters:
+ - name: "jobExecutionId"
+ in: "path"
+ required: true
+ type: "integer"
+ format: "int64"
+ responses:
+ 200:
+ description: "successful operation"
+ schema:
+ $ref: "#/definitions/ExecutionContextResponse"
+ /jobs/executions/{jobExecutionId}/steps/{stepExecutionId}:
+ get:
+ tags:
+ - "jobs"
+ summary: "Get step execution details."
+ description: ""
+ operationId: "getStepExecution"
+ produces:
+ - "application/json"
+ parameters:
+ - name: "jobExecutionId"
+ in: "path"
+ required: true
+ type: "integer"
+ format: "int64"
+ - name: "stepExecutionId"
+ in: "path"
+ required: true
+ type: "integer"
+ format: "int64"
+ responses:
+ 200:
+ description: "successful operation"
+ schema:
+ $ref: "#/definitions/StepExecutionInfoResponse"
+ /jobs/executions/{jobExecutionId}/steps/{stepExecutionId}/execution-context:
+ get:
+ tags:
+ - "jobs"
+ summary: "Get the execution context of step execution."
+ description: ""
+ operationId: "getStepExecutionContext"
+ produces:
+ - "application/json"
+ parameters:
+ - name: "jobExecutionId"
+ in: "path"
+ required: true
+ type: "integer"
+ format: "int64"
+ - name: "stepExecutionId"
+ in: "path"
+ required: true
+ type: "integer"
+ format: "int64"
+ responses:
+ 200:
+ description: "successful operation"
+ schema:
+ $ref: "#/definitions/StepExecutionContextResponse"
+ /jobs/executions/{jobExecutionId}/steps/{stepExecutionId}/progress:
+ get:
+ tags:
+ - "jobs"
+ summary: "Get progress of step execution."
+ description: ""
+ operationId: "getStepExecutionProgress"
+ produces:
+ - "application/json"
+ parameters:
+ - name: "jobExecutionId"
+ in: "path"
+ required: true
+ type: "integer"
+ format: "int64"
+ - name: "stepExecutionId"
+ in: "path"
+ required: true
+ type: "integer"
+ format: "int64"
+ responses:
+ 200:
+ description: "successful operation"
+ schema:
+ $ref: "#/definitions/StepExecutionProgressResponse"
+ /jobs/info/names:
+ get:
+ tags:
+ - "jobs"
+ summary: "Get all job names"
+ description: ""
+ operationId: "getAllJobNames"
+ produces:
+ - "application/json"
+ parameters: []
+ responses:
+ 200:
+ description: "successful operation"
+ schema:
+ type: "array"
+ uniqueItems: true
+ items:
+ type: "string"
+ /jobs/{jobName}:
+ post:
+ tags:
+ - "jobs"
+ summary: "Start a new job instance by job name."
+ description: ""
+ operationId: "startJob"
+ produces:
+ - "application/json"
+ parameters:
+ - name: "jobName"
+ in: "path"
+ required: true
+ type: "string"
+ - name: "params"
+ in: "query"
+ required: false
+ type: "string"
+ responses:
+ 200:
+ description: "successful operation"
+ schema:
+ $ref: "#/definitions/JobExecutionInfoResponse"
+ /jobs/{jobName}/executions:
+ get:
+ tags:
+ - "jobs"
+ summary: "Get the id values of all the running job instances."
+ description: ""
+ operationId: "getExecutionIdsByJobName"
+ produces:
+ - "application/json"
+ parameters:
+ - name: "jobName"
+ in: "path"
+ required: true
+ type: "string"
+ responses:
+ 200:
+ description: "successful operation"
+ schema:
+ type: "array"
+ uniqueItems: true
+ items:
+ type: "integer"
+ format: "int64"
+ /jobs/{jobName}/info:
+ get:
+ tags:
+ - "jobs"
+ summary: "Get job details by job name."
+ description: ""
+ operationId: "getJobDetails"
+ produces:
+ - "application/json"
+ parameters:
+ - name: "page"
+ in: "query"
+ required: false
+ type: "integer"
+ default: 0
+ format: "int32"
+ - name: "size"
+ in: "query"
+ required: false
+ type: "integer"
+ default: 20
+ format: "int32"
+ - name: "jobName"
+ in: "path"
+ required: true
+ type: "string"
+ responses:
+ 200:
+ description: "successful operation"
+ schema:
+ $ref: "#/definitions/JobDetailsResponse"
+ /jobs/{jobName}/{jobInstanceId}/executions:
+ get:
+ tags:
+ - "jobs"
+ summary: "Get execution for job instance."
+ description: ""
+ operationId: "getExecutionsForInstance"
+ produces:
+ - "application/json"
+ parameters:
+ - name: "jobName"
+ in: "path"
+ required: true
+ type: "string"
+ - name: "jobInstanceId"
+ in: "path"
+ required: true
+ type: "integer"
+ format: "int64"
+ responses:
+ 200:
+ description: "successful operation"
+ schema:
+ type: "array"
+ items:
+ $ref: "#/definitions/JobExecutionInfoResponse"
+ post:
+ tags:
+ - "jobs"
+ summary: "Restart job instance."
+ description: ""
+ operationId: "restartJobInstance"
+ produces:
+ - "application/json"
+ parameters:
+ - in: "body"
+ name: "body"
+ required: false
+ schema:
+ $ref: "#/definitions/JobExecutionRestartRequest"
+ responses:
+ 200:
+ description: "successful operation"
+ schema:
+ $ref: "#/definitions/JobExecutionInfoResponse"
+definitions:
+ JobExecutionData:
+ type: "object"
+ properties:
+ id:
+ type: "integer"
+ format: "int64"
+ executionContext:
+ $ref: "#/definitions/ExecutionContext"
+ jobInstance:
+ $ref: "#/definitions/JobInstance"
+ jobId:
+ type: "integer"
+ format: "int64"
+ jobParameters:
+ $ref: "#/definitions/JobParameters"
+ failureExceptions:
+ type: "array"
+ items:
+ $ref: "#/definitions/Throwable"
+ endTime:
+ type: "string"
+ format: "date-time"
+ exitStatus:
+ $ref: "#/definitions/ExitStatus"
+ createTime:
+ type: "string"
+ format: "date-time"
+ lastUpdated:
+ type: "string"
+ format: "date-time"
+ jobConfigurationName:
+ type: "string"
+ startTime:
+ type: "string"
+ format: "date-time"
+ status:
+ type: "string"
+ enum:
+ - "COMPLETED"
+ - "STARTING"
+ - "STARTED"
+ - "STOPPING"
+ - "STOPPED"
+ - "FAILED"
+ - "ABANDONED"
+ - "UNKNOWN"
+ stepExecutionDataList:
+ type: "array"
+ items:
+ $ref: "#/definitions/StepExecutionData"
+ JobInstance:
+ type: "object"
+ properties:
+ id:
+ type: "integer"
+ format: "int64"
+ version:
+ type: "integer"
+ format: "int32"
+ jobName:
+ type: "string"
+ instanceId:
+ type: "integer"
+ format: "int64"
+ StepExecutionData:
+ type: "object"
+ properties:
+ id:
+ type: "integer"
+ format: "int64"
+ jobExecutionId:
+ type: "integer"
+ format: "int64"
+ executionContext:
+ $ref: "#/definitions/ExecutionContext"
+ stepName:
+ type: "string"
+ terminateOnly:
+ type: "boolean"
+ default: false
+ failureExceptions:
+ type: "array"
+ items:
+ $ref: "#/definitions/Throwable"
+ endTime:
+ type: "string"
+ format: "date-time"
+ exitStatus:
+ $ref: "#/definitions/ExitStatus"
+ lastUpdated:
+ type: "string"
+ format: "date-time"
+ commitCount:
+ type: "integer"
+ format: "int32"
+ readCount:
+ type: "integer"
+ format: "int32"
+ filterCount:
+ type: "integer"
+ format: "int32"
+ writeCount:
+ type: "integer"
+ format: "int32"
+ readSkipCount:
+ type: "integer"
+ format: "int32"
+ writeSkipCount:
+ type: "integer"
+ format: "int32"
+ processSkipCount:
+ type: "integer"
+ format: "int32"
+ rollbackCount:
+ type: "integer"
+ format: "int32"
+ startTime:
+ type: "string"
+ format: "date-time"
+ status:
+ type: "string"
+ enum:
+ - "COMPLETED"
+ - "STARTING"
+ - "STARTED"
+ - "STOPPING"
+ - "STOPPED"
+ - "FAILED"
+ - "ABANDONED"
+ - "UNKNOWN"
+ StackTraceElement:
+ type: "object"
+ properties:
+ methodName:
+ type: "string"
+ fileName:
+ type: "string"
+ lineNumber:
+ type: "integer"
+ format: "int32"
+ className:
+ type: "string"
+ nativeMethod:
+ type: "boolean"
+ default: false
+ JobExecutionDetailsResponse:
+ type: "object"
+ properties:
+ jobExecutionInfoResponse:
+ $ref: "#/definitions/JobExecutionInfoResponse"
+ stepExecutionInfoList:
+ type: "array"
+ items:
+ $ref: "#/definitions/StepExecutionInfoResponse"
+ StepExecutionContextResponse:
+ type: "object"
+ properties:
+ executionContextMap:
+ type: "object"
+ additionalProperties:
+ type: "object"
+ jobExecutionId:
+ type: "integer"
+ format: "int64"
+ stepExecutionId:
+ type: "integer"
+ format: "int64"
+ stepName:
+ type: "string"
+ StepExecutionProgress:
+ type: "object"
+ properties:
+ estimatedPercentCompleteMessage:
+ $ref: "#/definitions/MessageSourceResolvable"
+ estimatedPercentComplete:
+ type: "number"
+ format: "double"
+ ExitStatus:
+ type: "object"
+ properties:
+ exitCode:
+ type: "string"
+ exitDescription:
+ type: "string"
+ running:
+ type: "boolean"
+ default: false
+ ExecutionContextResponse:
+ type: "object"
+ properties:
+ jobExecutionId:
+ type: "integer"
+ format: "int64"
+ executionContextMap:
+ type: "object"
+ additionalProperties:
+ type: "object"
+ StepExecutionHistory:
+ type: "object"
+ properties:
+ stepName:
+ type: "string"
+ count:
+ type: "integer"
+ format: "int32"
+ commitCount:
+ $ref: "#/definitions/CumulativeHistory"
+ rollbackCount:
+ $ref: "#/definitions/CumulativeHistory"
+ readCount:
+ $ref: "#/definitions/CumulativeHistory"
+ writeCount:
+ $ref: "#/definitions/CumulativeHistory"
+ filterCount:
+ $ref: "#/definitions/CumulativeHistory"
+ readSkipCount:
+ $ref: "#/definitions/CumulativeHistory"
+ writeSkipCount:
+ $ref: "#/definitions/CumulativeHistory"
+ processSkipCount:
+ $ref: "#/definitions/CumulativeHistory"
+ duration:
+ $ref: "#/definitions/CumulativeHistory"
+ durationPerRead:
+ $ref: "#/definitions/CumulativeHistory"
+ TimeZone:
+ type: "object"
+ properties:
+ displayName:
+ type: "string"
+ id:
+ type: "string"
+ dstsavings:
+ type: "integer"
+ format: "int32"
+ rawOffset:
+ type: "integer"
+ format: "int32"
+ MessageSourceResolvable:
+ type: "object"
+ properties:
+ arguments:
+ type: "array"
+ items:
+ type: "object"
+ codes:
+ type: "array"
+ items:
+ type: "string"
+ defaultMessage:
+ type: "string"
+ ExecutionContext:
+ type: "object"
+ properties:
+ dirty:
+ type: "boolean"
+ default: false
+ empty:
+ type: "boolean"
+ default: false
+ StepExecutionInfoResponse:
+ type: "object"
+ properties:
+ id:
+ type: "integer"
+ format: "int64"
+ jobExecutionId:
+ type: "integer"
+ format: "int64"
+ jobName:
+ type: "string"
+ name:
+ type: "string"
+ startDate:
+ type: "string"
+ startTime:
+ type: "string"
+ duration:
+ type: "string"
+ durationMillis:
+ type: "integer"
+ format: "int64"
+ exitCode:
+ type: "string"
+ status:
+ type: "string"
+ JobExecutionInfoResponse:
+ type: "object"
+ properties:
+ id:
+ type: "integer"
+ format: "int64"
+ stepExecutionCount:
+ type: "integer"
+ format: "int32"
+ jobId:
+ type: "integer"
+ format: "int64"
+ jobName:
+ type: "string"
+ startDate:
+ type: "string"
+ startTime:
+ type: "string"
+ duration:
+ type: "string"
+ jobExecutionData:
+ $ref: "#/definitions/JobExecutionData"
+ jobParameters:
+ type: "object"
+ additionalProperties:
+ type: "object"
+ jobParametersString:
+ type: "string"
+ restartable:
+ type: "boolean"
+ default: false
+ abandonable:
+ type: "boolean"
+ default: false
+ stoppable:
+ type: "boolean"
+ default: false
+ timeZone:
+ $ref: "#/definitions/TimeZone"
+ JobInfo:
+ type: "object"
+ properties:
+ name:
+ type: "string"
+ executionCount:
+ type: "integer"
+ format: "int32"
+ launchable:
+ type: "boolean"
+ default: false
+ incrementable:
+ type: "boolean"
+ default: false
+ jobInstanceId:
+ type: "integer"
+ format: "int64"
+ JobExecutionRestartRequest:
+ type: "object"
+ properties:
+ jobName:
+ type: "string"
+ jobInstanceId:
+ type: "integer"
+ format: "int64"
+ operation:
+ type: "string"
+ enum:
+ - "RESTART"
+ Throwable:
+ type: "object"
+ properties:
+ cause:
+ $ref: "#/definitions/Throwable"
+ stackTrace:
+ type: "array"
+ items:
+ $ref: "#/definitions/StackTraceElement"
+ message:
+ type: "string"
+ localizedMessage:
+ type: "string"
+ suppressed:
+ type: "array"
+ items:
+ $ref: "#/definitions/Throwable"
+ JobParameters:
+ type: "object"
+ properties:
+ parameters:
+ type: "object"
+ additionalProperties:
+ $ref: "#/definitions/JobParameter"
+ empty:
+ type: "boolean"
+ default: false
+ CumulativeHistory:
+ type: "object"
+ properties:
+ count:
+ type: "integer"
+ format: "int32"
+ min:
+ type: "number"
+ format: "double"
+ max:
+ type: "number"
+ format: "double"
+ standardDeviation:
+ type: "number"
+ format: "double"
+ mean:
+ type: "number"
+ format: "double"
+ JobInstanceDetailsResponse:
+ type: "object"
+ properties:
+ jobInstance:
+ $ref: "#/definitions/JobInstance"
+ jobExecutionInfoResponseList:
+ type: "array"
+ items:
+ $ref: "#/definitions/JobExecutionInfoResponse"
+ JobParameter:
+ type: "object"
+ properties:
+ identifying:
+ type: "boolean"
+ default: false
+ value:
+ type: "object"
+ type:
+ type: "string"
+ enum:
+ - "STRING"
+ - "DATE"
+ - "LONG"
+ - "DOUBLE"
+ StepExecutionProgressResponse:
+ type: "object"
+ properties:
+ stepExecutionProgress:
+ $ref: "#/definitions/StepExecutionProgress"
+ stepExecutionHistory:
+ $ref: "#/definitions/StepExecutionHistory"
+ stepExecutionInfoResponse:
+ $ref: "#/definitions/StepExecutionInfoResponse"
+ JobDetailsResponse:
+ type: "object"
+ properties:
+ jobInfo:
+ $ref: "#/definitions/JobInfo"
+ jobInstanceDetailsResponseList:
+ type: "array"
+ items:
+ $ref: "#/definitions/JobInstanceDetailsResponse"
http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/docs/images/batch-1.png
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/docs/images/batch-1.png b/ambari-infra/ambari-infra-manager/docs/images/batch-1.png
new file mode 100644
index 0000000..d763852
Binary files /dev/null and b/ambari-infra/ambari-infra-manager/docs/images/batch-1.png differ
http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/docs/images/batch-2.png
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/docs/images/batch-2.png b/ambari-infra/ambari-infra-manager/docs/images/batch-2.png
new file mode 100644
index 0000000..1de3479
Binary files /dev/null and b/ambari-infra/ambari-infra-manager/docs/images/batch-2.png differ
http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/docs/images/batch-3.png
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/docs/images/batch-3.png b/ambari-infra/ambari-infra-manager/docs/images/batch-3.png
new file mode 100644
index 0000000..7f1123c
Binary files /dev/null and b/ambari-infra/ambari-infra-manager/docs/images/batch-3.png differ
http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/docs/images/batch-4.png
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/docs/images/batch-4.png b/ambari-infra/ambari-infra-manager/docs/images/batch-4.png
new file mode 100644
index 0000000..beb610a
Binary files /dev/null and b/ambari-infra/ambari-infra-manager/docs/images/batch-4.png differ
http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/common/InfraManagerConstants.java
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/common/InfraManagerConstants.java b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/common/InfraManagerConstants.java
index 77f7008..105f20e 100644
--- a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/common/InfraManagerConstants.java
+++ b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/common/InfraManagerConstants.java
@@ -25,7 +25,7 @@ public final class InfraManagerConstants {
public static final String PROTOCOL_SSL = "https";
public static final String ROOT_CONTEXT = "/";
public static final String WEB_RESOURCE_FOLDER = "webapp";
- public static final String DEFAULT_DATA_FOLDER_LOCATION = "/usr/ambari-infra-manager/data";
+ public static final String DEFAULT_DATA_FOLDER_LOCATION = "/opt/ambari-infra-manager/data";
public static final String DATA_FOLDER_LOCATION_PARAM = "dataFolderLocation";
public static final Integer SESSION_TIMEOUT = 60 * 30;
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/InfraManagerApiDocConfig.java
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/InfraManagerApiDocConfig.java b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/InfraManagerApiDocConfig.java
index 22e2263..4c76742 100644
--- a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/InfraManagerApiDocConfig.java
+++ b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/InfraManagerApiDocConfig.java
@@ -21,12 +21,22 @@ package org.apache.ambari.infra.conf;
import io.swagger.jaxrs.config.BeanConfig;
import io.swagger.jaxrs.listing.ApiListingResource;
import io.swagger.jaxrs.listing.SwaggerSerializers;
+import io.swagger.models.Info;
+import io.swagger.models.License;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class InfraManagerApiDocConfig {
+ private static final String DESCRIPTION = "Manager component for Ambari Infra";
+ private static final String VERSION = "1.0.0";
+ private static final String TITLE = "Infra Manager REST API";
+ private static final String LICENSE = "Apache 2.0";
+ private static final String LICENSE_URL = "http://www.apache.org/licenses/LICENSE-2.0.html";
+ private static final String RESOURCE_PACKAGE = "org.apache.ambari.infra.rest";
+ private static final String BASE_PATH = "/api/v1";
+
@Bean
public ApiListingResource apiListingResource() {
return new ApiListingResource();
@@ -41,14 +51,25 @@ public class InfraManagerApiDocConfig {
public BeanConfig swaggerConfig() {
BeanConfig beanConfig = new BeanConfig();
beanConfig.setSchemes(new String[]{"http", "https"});
- beanConfig.setBasePath("/api/v1");
- beanConfig.setTitle("Infra Manager REST API");
- beanConfig.setDescription("Manager component for Ambari Infra");
- beanConfig.setLicense("Apache 2.0");
- beanConfig.setLicenseUrl("http://www.apache.org/licenses/LICENSE-2.0.html");
+ beanConfig.setBasePath(BASE_PATH);
+ beanConfig.setTitle(TITLE);
+ beanConfig.setDescription(DESCRIPTION);
+ beanConfig.setLicense(LICENSE);
+ beanConfig.setLicenseUrl(LICENSE_URL);
beanConfig.setScan(true);
- beanConfig.setVersion("1.0.0");
- beanConfig.setResourcePackage("org.apache.ambari.infra.rest");
+ beanConfig.setVersion(VERSION);
+ beanConfig.setResourcePackage(RESOURCE_PACKAGE);
+
+ License license = new License();
+ license.setName(LICENSE);
+ license.setUrl(LICENSE_URL);
+
+ Info info = new Info();
+ info.setDescription(DESCRIPTION);
+ info.setTitle(TITLE);
+ info.setVersion(VERSION);
+ info.setLicense(license);
+ beanConfig.setInfo(info);
return beanConfig;
}
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/batch/InfraManagerBatchConfig.java
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/batch/InfraManagerBatchConfig.java b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/batch/InfraManagerBatchConfig.java
index c3d8db6..95f87f5 100644
--- a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/batch/InfraManagerBatchConfig.java
+++ b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/batch/InfraManagerBatchConfig.java
@@ -20,7 +20,9 @@ package org.apache.ambari.infra.conf.batch;
import org.apache.ambari.infra.job.dummy.DummyItemProcessor;
import org.apache.ambari.infra.job.dummy.DummyItemWriter;
+import org.apache.ambari.infra.job.dummy.DummyJobListener;
import org.apache.ambari.infra.job.dummy.DummyObject;
+import org.apache.ambari.infra.job.dummy.DummyStepListener;
import org.springframework.batch.admin.service.JdbcSearchableJobExecutionDao;
import org.springframework.batch.admin.service.JdbcSearchableJobInstanceDao;
import org.springframework.batch.admin.service.JdbcSearchableStepExecutionDao;
@@ -68,6 +70,7 @@ import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.datasource.DriverManagerDataSource;
import org.springframework.jdbc.datasource.init.DataSourceInitializer;
import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator;
+import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.transaction.PlatformTransactionManager;
@@ -78,6 +81,7 @@ import java.net.MalformedURLException;
@Configuration
@EnableBatchProcessing
@EnableScheduling
+@EnableAsync
public class InfraManagerBatchConfig {
@Value("classpath:org/springframework/batch/core/schema-drop-sqlite.sql")
@@ -225,13 +229,13 @@ public class InfraManagerBatchConfig {
protected Step dummyStep(ItemReader<DummyObject> reader,
ItemProcessor<DummyObject, String> processor,
ItemWriter<String> writer) {
- return steps.get("dummyStep").<DummyObject, String> chunk(2)
+ return steps.get("dummyStep").listener(new DummyStepListener()).<DummyObject, String> chunk(2)
.reader(reader).processor(processor).writer(writer).build();
}
@Bean(name = "dummyJob")
public Job job(@Qualifier("dummyStep") Step dummyStep) {
- return jobs.get("dummyJob").start(dummyStep).build();
+ return jobs.get("dummyJob").listener(new DummyJobListener()).start(dummyStep).build();
}
@Bean
http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyItemWriter.java
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyItemWriter.java b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyItemWriter.java
index f495795..9a78706 100644
--- a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyItemWriter.java
+++ b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyItemWriter.java
@@ -22,8 +22,15 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.item.ItemWriter;
+import java.io.File;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Date;
import java.util.List;
+import static org.apache.ambari.infra.common.InfraManagerConstants.DATA_FOLDER_LOCATION_PARAM;
+
public class DummyItemWriter implements ItemWriter<String> {
private static final Logger LOG = LoggerFactory.getLogger(DummyItemWriter.class);
@@ -32,5 +39,11 @@ public class DummyItemWriter implements ItemWriter<String> {
public void write(List<? extends String> values) throws Exception {
LOG.info("DummyItem writer called (values: {})... wait 1 seconds", values.toString());
Thread.sleep(1000);
+ String outputDirectoryLocation = String.format("%s%s%s%s", System.getProperty(DATA_FOLDER_LOCATION_PARAM), File.separator, "dummyOutput-", new Date().getTime());
+ Path pathToDirectory = Paths.get(outputDirectoryLocation);
+ Path pathToFile = Paths.get(String.format("%s%s%s", outputDirectoryLocation, File.separator, "dummyOutput.txt"));
+ Files.createDirectories(pathToDirectory);
+ LOG.info("Write to file: ", pathToFile.getFileName().toAbsolutePath());
+ Files.write(pathToFile, values.toString().getBytes());
}
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyJobListener.java
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyJobListener.java b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyJobListener.java
new file mode 100644
index 0000000..0bbfb55
--- /dev/null
+++ b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyJobListener.java
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.ambari.infra.job.dummy;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.batch.core.JobExecution;
+import org.springframework.batch.core.JobExecutionListener;
+
+public class DummyJobListener implements JobExecutionListener {
+
+ private static final Logger LOG = LoggerFactory.getLogger(DummyJobListener.class);
+
+ @Override
+ public void beforeJob(JobExecution jobExecution) {
+ LOG.info("Dummy - before job execution");
+ }
+
+ @Override
+ public void afterJob(JobExecution jobExecution) {
+ LOG.info("Dummy - after job execution");
+ }
+}
http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyStepListener.java
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyStepListener.java b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyStepListener.java
new file mode 100644
index 0000000..548e650
--- /dev/null
+++ b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyStepListener.java
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.ambari.infra.job.dummy;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.batch.core.ExitStatus;
+import org.springframework.batch.core.StepExecution;
+import org.springframework.batch.core.StepExecutionListener;
+
+public class DummyStepListener implements StepExecutionListener {
+
+ private static final Logger LOG = LoggerFactory.getLogger(DummyStepListener.class);
+
+ @Override
+ public void beforeStep(StepExecution stepExecution) {
+ LOG.info("Dummy step - before step execution");
+ }
+
+ @Override
+ public ExitStatus afterStep(StepExecution stepExecution) {
+ LOG.info("Dummy step - after step execution");
+ return stepExecution.getExitStatus();
+ }
+}
http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/rest/JobResource.java
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/rest/JobResource.java b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/rest/JobResource.java
index 7023957..0e20b54 100644
--- a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/rest/JobResource.java
+++ b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/rest/JobResource.java
@@ -98,7 +98,7 @@ public class JobResource {
@GET
@Produces({"application/json"})
- @Path("/info/{jobName}")
+ @Path("{jobName}/info")
@ApiOperation("Get job details by job name.")
public JobDetailsResponse getJobDetails(@BeanParam @Valid JobRequest jobRequest) throws NoSuchJobException {
return jobManager.getJobDetails(jobRequest.getJobName(), jobRequest.getPage(), jobRequest.getSize());