You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by Purshotam Shah <pu...@yahoo-inc.com> on 2016/09/29 00:53:58 UTC
Review Request 52373: OOZIE-2688 YAML representation of Oozie unit
testcase
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52373/
-----------------------------------------------------------
Review request for oozie.
Bugs: OOZIE-2688
https://issues.apache.org/jira/browse/OOZIE-2688
Repository: oozie-git
Description
-------
Oozie YAML Unit test
Maven configuration
YAML configuration
Declaration
Import
setup
Initialization
Job Definition
HDFS operation
HCat operation
Job stubbing
Usage Guideline
validation
output validation
job validation
Usage Guideline
TearUp
Tag
Examples
Bundle Test
Coordinator test
Coordinator test with HCat
Workflow Test
Workflow Test with skip Job submission
API
Maven configuration
To enable oozie unit yaml, one need add oozie-yaml plugin to maven.
<plugin>
<groupId>org.apache.oozie</groupId>
<artifactId>oozie-yaml</artifactId>
<version>0.1-SNAPSHOT</version>
<executions>
<execution>
<goals>
<goal>oozieyaml</goal>
</goals>
</execution>
</executions>
</plugin>
Oozie-YAML plugin to Maven will search for any oozie YAML file and if it's there, it will convert it into respective JUnit test case and will execute them. There will be a surefire report.
YAML configuration
Whole YAML configuration is divided into following sections
Declaration
Import
Setup
Initialization
Job Definition
Stubbing
HDFS operation
HCat operation
Validation
TearUp
Tag
Declaration
This section can be used to defined test case properties like test case name, package name, etc.
name: String, required
Name of the test cases. Java file will be created with the same name.
package: String, Optional
The Java package of the generated test class.
skip_test : boolean, Optional
A boolean that indicates whether or not to execute this test. This parameter defaults to false.
copy_dependencies_files_hdfs: List of Strings, Optional
List of HDFS of paths where all Maven dependent jar will be copied. This is useful if for pig/hive jobs.
Examples:
package: org.apache.oozie.yamltest
name: TestWorkflowActionFromBundle
copy_dependencies_files_hdfs:
- /tmp/workflow-test/lib
Import
imports: List of Strings, Optional
A list of Java classes to import in the generated Oozie unit Java file. * Examples: *
import:
- org.apache.oozie.util.DateUtils
- java.util.Date
setup
setup: List of Strings, Optional
List of java statement that can be used for JUnit setup method.
setup can also include HDFS and HCat operation.
Initialization
initialization: List of Strings, Optional
List of java statement that can be used for initialization. This statement will be added first in test case.
Examples:
initialization:
- "Date date = new Date()"
- "String currentTime = DateUtils.formatDateOozieTZ(date)"
Job Definition
This section can be used to define job properties
job_property: String, Optional
job_property can be a Java Properties file (.properties) or a Hadoop XML Configuration file (.xml).
properties: String, Optional
Overridden key/value configuration use in job submission. This value overrides the value of job_property.
Examples:
job:
properties:
raw_logs_path: "/tmp/test"
aggregated_logs_path: "/user/test"
workflowPath: "/testDirPath"
start: "2010-02-01T10:00Z"
job:
job_property : src/test/resources/job-1.properties
properties:
oozie.wf.application.path: "/tmp/workflow-test/"
jobTracker: !func getJobTrackerUri()
nameNode: !func getNameNodeUri()
HDFS operation
Set of HDFS operation. Can be used to create dummy input dataset for coordinator or input file for workflow. Supported operation are
mkdir: List, Optional
The mkdir command creates the specified directory. If the directory already exists, it does a no-op.
touchz: List, Optional
Touchz creates a zero length file in the specified path if none exists. If one already exists, then touchz will perform a touch operation.
delete: List, Optional
List of file/dir to be deleted.
delete: List, Optional
Deletes the specified path.
chmod: List, Optional
The chmod command changes the permissions for the specified path.
permission: String, Optional
Permission of path. Permissions can be specified using the Unix Symbolic representation (e.g. -rwxrw-rw-) or an octal representation (755).
path: String, Optional
HDFS path of file/dir whose permission need to changed.
copyFromLocal: List, Optional
List of file to be copied to HDFS
source: String, mandatory
destination: String, mandatory
move: List, Optional
source: String, mandatory
destination: String, mandatory
Examples:
hdfs:
touchz:
- /tmp/test/2010/02/01/09/_SUCCESS
copyFromLocal:
- source: coordinator-test-3.xml
destination: /tmp/workflow-test/coordinator-test-3.xml
HCat operation
Set of HCat operation. Can be used to create HCat partition for coordinator/workflow. HCat operation can be configured in nested form. At parent level, at has two operation delete and create
create: Can be used to create HCat partition.
delete: Can be used to delete HCat partition.
Both delete and create have following structure.
databases:
- database: db_test
tables:
- table: table_test
columns: "dt,country"
partitions:
- dt=20100201;country=usa
database: List, Optional
List of databases to be created/deleted.
table: List, Optional
List of tables to create/delete for the database.
columns: String, Optional
Table columns definition.
partitions: List, Optional
partition to be created.
Examples:
hcat:
create:
databases:
- database: db_test
tables:
- table: table_test
columns: "dt,country"
partitions:
- dt=20100201;country=usa
- database: db_test_1
Job stubbing
List of coordinator job, workflow, workflow action, pig script, hive script, etc. to be skipped while running test It's a nested configuration.
skip_all_job_submission: boolean, Optional
All job submission will be skipped.
skip_pig_job_submission: boolean, Optional
All pig job submission will be skipped.
skip_hive_job_submission: boolean, Opt.ional
All hive job submission will be skipped.
skip_spark_job_submission: boolean, Optional
All spark job submission will be skipped.
skip_java_job_submission: boolean, Optional
All java job submission will be skipped.
skip_shell_job_submission: boolean, Optional
All shell job submission will be skipped.
skip_distcp_job_submission: boolean, Optional
All shell job submission will be skipped.
coordinator: List, Optional
List of coordinator to be skipped.
workflow_action:List, Optional
List of workflow action to be skipped.
skip_job_submission:boolean, Optional
Indicate that action should be executed, but only Hadoop job submission is ignored.
coordinators: List, Optional
List of coordinators to be skipped.
coordinator: String, Optional
Name of the coordinator which has to be skipped.
workflow:
just a marked that workflow of a coordinator will be skipped.
workflow_actions:
List of workflow action that will be skipped.
workflow_action: String, Optional
Name of the workflow action.
skip_job_submission:boolean, Optional
If true, job submission will be skipped.
status: String, Optional
Status of skipped action. It will be useful if somebody wants to test different execution path of workflow. The default value is OK. Supported status are OK and ERROR.
output: List, Optional
Copy output to for skipped action
source: String, Optional
destination: String, Optional
Usage Guideline
Skip is a nested configuration, if submitted job is bundle, to skip workflow action one need to do a nested configuration.
skip:
coordinators:
- coordinator: coord-1
- coordinator: coord-2
workflow:
- coordinator: coord-3
workflow:
workflow_actions:
- workflow_action : pig
skip_job_submission: true
status: OK
output :
- path:
source: /tmp/resources/action-1-output
destination: /tmp/test/output/
- path:
source: src/test/resources/action-2-output
destination: /tmp/test/output/
Above example will skip coord-1 submission. Coord-2 will be submitted, but there won't be any workflow submission. For coord-3, job submission of action "pig" will be skipped. File /tmp/resources/action-1-output and /tmp/resources/action-2-output will copied to path /tmp/test/output/.
If submitted job is coordinator below configuration can be used for skipping jobs.
Examples:
skip:
workflow:
workflow_actions:
- status: OK
output:
- source: src/test/resources/action-1-output
destination: "/tmp/test/output/"
- source: src/test/resources/action-2-output
destination: "/tmp/test/output/"
workflow_action: pig
skip_job_submission: true
- status: FAILED
workflow_action: hive-action
skip:
skip_all_job: true
validation
Validation is dived into two section.
output validation
job validation
output validation
TODO
job validation
This section can be used to validate job status and attributes. Job validation syntax is very similar to skip syntax. One need to use nested configuration to validate job.
sleep : integer, Optional
Thread sleep.
wait_for_status: integer, Optional
Testcase will wait for status.
status: String, Optional
Assert job status
not_status: String, Optional
Assert job status as false
nominal_time: String, Optional
Assert coord action nominal time
coordinators: List, Optional
List of coordinators for validation
coordinator: String, Optional
Name of the coordinator which will be used for validation
workflow_action: String, Optional
Name of the workflow action which will be used for validation
job_conf: List, Optional
Validate job conf.
key: String, Optional
expected_value: String, Optional
Usage Guideline
Like stubbing, validate_job can have nested configuration. If the submitted job is bundle then to validate workflow action, one need to use nested configuration.
Validating a Bundle Job
validate_job:
sleep: 60000
status: RUNNING
coordinators:
- coordinator: coord-3
not_status: PREP
coordinator_actions:
- coordinator_action : "@2"
sleep: 6000
wait_for_status: RUNNING
nominal_time: 2010-02-01T11:00Z
workflow:
workflow_actions:
- workflow_action : pig
status: OK
Validating a Coord Job
validations:
validate_job:
sleep: 6000
job_conf:
- key: queueName
expected_value: defult
coordinator_actions:
- coordinator_action : "@2"
not_status: WAITING
nominal_time: 2010-02-01T11:00Z
Validating a workflow Job
validations:
validate_job:
sleep: 60000
status: SUCCEEDED
workflow_actions:
- workflow_action : pig
status: OK
TearUp
tearUp: List of Strings, Optional
List of java statement that can be used for JUnit tearUp method.
tearUp can also include HDFS and HCat operation.
Tag
Oozie yaml also support tagging. Supported tags are
!func : This indicate that value is function, not string.
jobTracker: !func getJobTrackerUri(), this will be interpreted as jobTracker = getJobTrackerUri() . If !func tag is missing then it will be interpreted as jobTracker = "getJobTrackerUri()" .
!var : This indicate that value is variable, not string.
Examples:
initialization:
- "Date date = new Date()"
- "String currentTime = DateUtils.formatDateOozieTZ(date)"
job:
properties:
start: !var currentTime
!exp : This indicate that value is java statement, not string.
Examples:
touchz:
- !exp \"/tmp/test/\" + formatDate(currentTime, \"yyyy/MM/dd/HH\") + \"/_SUCCESS\"
Examples
Bundle Test
name: TestBundleDisableCoord
output_path: ./target/oozieyaml-sourcesjob:
properties:
raw_logs_path: "/tmp/test"
aggregated_logs_path: "/user/test"
workflowPath: "/testDirPath"
start: "2010-02-01T10:00Z"
end: "2012-02-03T23:59Z"
oozie.bundle.application.path: "/tmp/workflow-test/bundle-multiple-coord.xml"
queueName: "defult"
coord_path_1: "/tmp/workflow-test/coordinator-test-1.xml"
coord_path_2: "/tmp/workflow-test/coordinator-test-2.xml"
jobTracker: !func getJobTrackerUri()
nameNode: !func getNameNodeUri()
hdfs:
touchz:
- /tmp/test/2010/02/01/09/_SUCCESS
- /tmp/test/2010/02/01/10/_SUCCESS
- /tmp/test/2010/02/01/11/_SUCCESS
- /tmp/test/2010/02/01/12/_SUCCESS
copyFromLocal:
- source: bundle-multiple-coord.xml
destination: "/tmp/workflow-test/bundle-multiple-coord.xml"
- source: coordinator-test-1.xml
destination: /tmp/workflow-test/coordinator-test-1.xml
- source: coordinator-test-2.xml
skip:
coordinators:
- coordinator: coord_1
validations:
validate_job:
sleep: 6000
status: RUNNING
coordinators:
- coordinator: coord_2
not_status: PREP
coordinator_actions:
- coordinator_action : "@2"
not_status: WAITING
nominal_time: 2010-02-01T11:00Z
Coordinator test
name: TestCoordinator
output_path: ./target/oozieyaml-sourcesjob:
properties:
raw_logs_path: "/tmp/test"
aggregated_logs_path: "/user/test"
workflowPath: "/testDirPath"
start: "2010-02-01T10:00Z"
end: "2012-02-03T23:59Z"
oozie.coord.application.path: "/tmp/workflow-test/coordinator.xml"
queueName: "defult"
jobTracker: !func getJobTrackerUri()
nameNode: !func getNameNodeUri()
hdfs:
touchz:
- /tmp/test/2010/02/01/09/_SUCCESS
- /tmp/test/2010/02/01/10/_SUCCESS
- /tmp/test/2010/02/01/11/_SUCCESS
- /tmp/test/2010/02/01/12/_SUCCESS
copyFromLocal:
- source: coordinator-test.xml
destination: "/tmp/workflow-test/coordinator.xml"
validations:
validate_job:
sleep: 6000
job_conf:
- key: queueName
expected_value: defult
coordinator_actions:
- coordinator_action : "@2"
not_status: WAITING
nominal_time: 2010-02-01T11:00Z
Coordinator test with HCat
name: TestCoordinatorWithHcat
output_path: ./target/oozieyaml-sourcesjob:
properties:
raw_logs_path: "/tmp/test"
aggregated_logs_path: "/user/test"
workflowAppUri: "/testDirPath"
start: "2010-02-01T10:00Z"
end: "2012-02-03T23:59Z"
oozie.coord.application.path: "/tmp/workflow-test/coordinator.xml"
queueName: "defult"
db: db_test
table: table_test
dataOut: dataOut
jobTracker: !func getJobTrackerUri()
nameNode: !func getNameNodeUri()
hcatNode: !func \"hcat://\" + getMetastoreAuthority()
hdfs:
mkdir:
- "/tmp/workflow-test/"
copyFromLocal:
- source: coordinator-hcat-test.xml
destination: "/tmp/workflow-test/coordinator.xml"
hcat:
create:
databases:
- database: db_test
tables:
- table: table_test
columns: "dt,country"
partitions:
- dt=20100201;country=usa
validations:
validate_job:
sleep: 6000
coordinator_actions:
- coordinator_action : "@1"
not_status: WAITING
nominal_time: 2010-02-01T10:00Z
Workflow Test
name: TestWorkflowWithPig
output_path: ./target/oozieyaml-sources
copy_dependencies_files_hdfs:
- /tmp/workflow-test/libjob:
properties:
INPUT: "/tmp/input/dummp.txt"
oozie.wf.application.path: "/tmp/workflow-test/"
queueName: "defult"
OUTPUT: "/tmp/output"
jobTracker: !func getJobTrackerUri()
nameNode: !func getNameNodeUri()
hdfs:
mkdir:
- "/tmp/workflow-test/"
touchz:
- /tmp/input/dummp.txt
copyFromLocal:
- source: echo.pig
destination: "/tmp/workflow-test/echo.pig"
- source: workflow-test-with-pig.xml
destination: "/tmp/workflow-test/workflow.xml"
validations:
validate_job:
sleep: 60000
status: SUCCEEDED
workflow_actions:
- workflow_action : pig
status: OK
Workflow Test with skip Job submission
name: TestWorkflowJobStatus
output_path: ./target/oozieyaml-sources
copy_dependencies_files_hdfs:
- /tmp/workflow-test/libjob:
properties:
INPUT: "/tmp/input/dummp.txt"
oozie.wf.application.path: "/tmp/workflow-test/"
queueName: "defult"
OUTPUT: "/tmp/output"
jobTracker: !func getJobTrackerUri()
nameNode: !func getNameNodeUri()
hdfs:
mkdir:
- "/tmp/workflow-test/"
touchz:
- /tmp/input/dummp.txt
copyFromLocal:
- source: echo.pig
destination: "/tmp/workflow-test/echo.pig"
- source: workflow-test-with-output.xml
destination: "/tmp/workflow-test/workflow.xml"
skip:
workflow_actions:
- workflow_action : pig
skip_job_submission: true
status: OK
output:
- source: /tmp/workflow-test/echo.pig
destination: "/tmp/workflow-test/dummy.pig"
- workflow_action : pig_2
skip_job_submission: true
status: ERROR
validations:
validate_job:
wait_for_status: KILLED
workflow_actions:
- workflow_action : pig
status: OK
- workflow_action : pig_1
status: OK
- workflow_action : pig_2
status: ERROR
API
String getJobTrackerUri():
Returns job tracker URI.
String getNameNodeUri():
Returns name node URI.
String getMetastoreAuthority()
Returns hcat URI ( without hcat:// prefix).
FileSystem getFileSystem():
Returns filesystem.
Date addDays(Date date, int amount)
Return new date after adding days.
Date addHours(Date date, int amount)
Return new date after adding hours.
Date addMonths(Date date, int amount)
Return new date after adding months.
Date addMinutes(Date date, int amount)
Return new date after adding minutes.
String addDays(String date, int amount)
Return new date after adding days. Input date is in TZ format.
String addHours(String date, int amount)
Return new date after adding hours. Input date is in TZ format.
String addMonths(String date, int amount)
Return new date after adding months. Input date is in TZ format.
String addMinutes(String date, int amount)
Return new date after adding minutes. Input date is in TZ format.
String formatDate(String date, String format)
Format date. Input date is in TZ format.
String formatDate(Date date, String format)
Diffs
-----
client/src/main/java/org/apache/oozie/client/WorkflowJob.java 131d9b2336d7b643927af1cb65aceac95c6dc171
core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc
core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797
core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d
core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 65996d97145e85b1e4d1de87fe9891921df8f0db
core/src/main/java/org/apache/oozie/command/bundle/BundleStartXCommand.java cc98a6db803cebbefc68d7f8dd264e24d933c9eb
core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java 45abe5a5e88f482c6b0e8c6385d7ea054d242c7d
core/src/main/java/org/apache/oozie/command/wf/ActionEndXCommand.java 740b8d3f101352e951001af689e541ffbf606b52
core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java e95a60ad49f503695cc5fce24a67924dc6ec60e5
core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java 3c032678d06641ae7bb2f2a9c0631948190ea662
core/src/main/java/org/apache/oozie/mini/MiniOozie.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieDistcpActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieEmailActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieExecutorUtil.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieFsActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieHive2ActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieHiveActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieJavaActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieLocalClient.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieMapReduceActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOoziePigActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieShellActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieSparkActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieSubWorkflowActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/service/LiteWorkflowStoreService.java ffc29af2f834da6d0890ea7215c5a62fd7cd693e
core/src/main/java/org/apache/oozie/store/WorkflowStore.java c565e74893b863caef6c93015cfe38fe520d04ec
core/src/main/java/org/apache/oozie/util/JobUtils.java a7a53b355d194b55959ede20c85d38e02294853d
core/src/main/resources/localoozie-log4j.properties 906397fe0f284c187a8f05ece7a1a0677572e845
core/src/main/resources/oozie-log4j.properties c86b30143f5f58135493d9e6bdba29f402a0b1fd
core/src/test/java/org/apache/oozie/test/MiniHCatServer.java 8699ff881b2f27d8369b1a66083fcf785a56ef0f
core/src/test/java/org/apache/oozie/test/MiniOozieTestCase.java 12ede028655f2a30613a9f802e48cd3a27af6ed9
core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7
core/src/test/java/org/apache/oozie/test/XTestCase.java 70f77be191bd844759ab1506a8d4c7aa4cb74f4c
core/src/test/resources/test-oozie-log4j.properties 3040d4f621b785d98cdccae2fa039e19f7069484
docs/src/site/twiki/YamlUnit.twiki e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
docs/src/site/twiki/index.twiki 8591530a2418b9d2bd4cdf5f25597bdd18ee4a75
minitest/pom.xml 21aa57534c3d89cd1c349ec81b788b9694b59bb9
minitest/src/main/java/org/apache/oozie/test/MiniOozieTestCase.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/java/org/apache/oozie/test/TestBundle.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/java/org/apache/oozie/test/TestCoordinator.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/java/org/apache/oozie/test/TestCoordinatorWithHCat.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/java/org/apache/oozie/test/TestWorkflow.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/java/org/apache/oozie/test/TestWorkflowWithCert.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/java/org/apache/oozie/test/TestWorkflowWithHive.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/java/org/apache/oozie/test/TestWorkflowWithPig.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/bundle-test.properties e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/bundle-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/coordinator-test-withhcat.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/coordinator-test.properties e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/coordinator-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/echo.pig e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/hive.script e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/oozie-log4j.properties c142d725140930bfa89cd2b163d0768a4c3a750a
minitest/src/test/resources/wf-hive-job.properties e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/wf-pig-job.properties e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/wf-test-with-cert.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/wf-test-with-hive.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/wf-test-with-pig.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
pom.xml 704a2eeee9f4e4805e3e08c2a547b2a375f6b1b2
yaml-unit/doc/OozieYamlUnit.twiki e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/doc/hdfs.yaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/doc/skip.yaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/doc/yaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/pom.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/oozieyaml/test-bundle.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/oozieyaml/test-workflow-action-from-bundle.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/bundle-multiple-coord-1.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/bundle-multiple-coord.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/bundle-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/coordinator-test-1.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/coordinator-test-2.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/coordinator-test-3.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/echo.pig e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/wf.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/workflow-test-with-pig.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/coordinator-test/pom.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/coordinator-test/src/test/oozieyaml/test-coordinator-current-time.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/coordinator-test/src/test/oozieyaml/test-coordinator-hcat.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/coordinator-test/src/test/oozieyaml/test-coordinator.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/coordinator-test/src/test/resources/coordinator-hcat-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/coordinator-test/src/test/resources/coordinator-latest-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/coordinator-test/src/test/resources/coordinator-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/pom.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/pom.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-all-action.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-init.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-job-definition.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-skip-status.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-withsetup.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/resources/echo.pig e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/resources/job-1.properties e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/resources/properties.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/resources/workflow-test-with-all-action.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/resources/workflow-test-with-output.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/resources/workflow-test-with-pig.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/pom.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/GenerateBatchJsonMojo.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/GenerateMojo.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/TestBuilder.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/YamlTagConstructor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/CopyDependenciesFilesHdfs.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/Import.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/InitStatement.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/JobDefinition.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/SetupStatement.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/Statement.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/Tags.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/TearDownStatement.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/TestClassBuilder.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/UnitTestGenerator.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/UnitTestGeneratorUtil.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/YamlConfig.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/YamlString.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/AddPartitions.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/CreateDatabase.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/CreateHCatPartition.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/CreateTable.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/DeleteDatabase.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/DeleteHCatPartition.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/DeletePartitions.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/DeleteTable.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/HCatOperation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/CopyDependenciesFilesHdfs.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/CopyFromLocal.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsChmod.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsDelete.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsMkdir.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsMove.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsOperation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsTouchz.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/Mkdir.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/Touchz.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/Skip.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/SkipCoordJob.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/SkipJobSubmission.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/SkipWorkflow.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/SkipWorkflowAction.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/SkipWorkflowDeserializer.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/YamlStringDeserializer.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/JobCheck.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/JobConfValidation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/NotStatusValidation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/Sleep.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/StatusValidation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateBundle.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateCoodJob.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateCoordAction.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateJob.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateWorkflow.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateWorkflowAction.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidationOutPut.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/Validations.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/WaitForJobStatus.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/resources/META-INF/m2e/lifecycle-mapping-metadata.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/A.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/Job.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/Properties.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestHdfs.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestJobValidation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestOozieYaml.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestSetup.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestSkip.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestTouchWithTag.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/YamlTestUtil.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
Diff: https://reviews.apache.org/r/52373/diff/
Testing
-------
Thanks,
Purshotam Shah
Re: Review Request 52373: OOZIE-2688 YAML representation of Oozie unit
testcase
Posted by Purshotam Shah via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52373/
-----------------------------------------------------------
(Updated Aug. 5, 2017, 1:03 a.m.)
Review request for oozie.
Bugs: OOZIE-2688
https://issues.apache.org/jira/browse/OOZIE-2688
Repository: oozie-git
Description
-------
Oozie YAML Unit test
Maven configuration
YAML configuration
Declaration
Import
setup
Initialization
Job Definition
HDFS operation
HCat operation
Job stubbing
Usage Guideline
validation
output validation
job validation
Usage Guideline
TearUp
Tag
Examples
Bundle Test
Coordinator test
Coordinator test with HCat
Workflow Test
Workflow Test with skip Job submission
API
Maven configuration
To enable oozie unit yaml, one need add oozie-yaml plugin to maven.
<plugin>
<groupId>org.apache.oozie</groupId>
<artifactId>oozie-yaml</artifactId>
<version>0.1-SNAPSHOT</version>
<executions>
<execution>
<goals>
<goal>oozieyaml</goal>
</goals>
</execution>
</executions>
</plugin>
Oozie-YAML plugin to Maven will search for any oozie YAML file and if it's there, it will convert it into respective JUnit test case and will execute them. There will be a surefire report.
YAML configuration
Whole YAML configuration is divided into following sections
Declaration
Import
Setup
Initialization
Job Definition
Stubbing
HDFS operation
HCat operation
Validation
TearUp
Tag
Declaration
This section can be used to defined test case properties like test case name, package name, etc.
name: String, required
Name of the test cases. Java file will be created with the same name.
package: String, Optional
The Java package of the generated test class.
skip_test : boolean, Optional
A boolean that indicates whether or not to execute this test. This parameter defaults to false.
copy_dependencies_files_hdfs: List of Strings, Optional
List of HDFS of paths where all Maven dependent jar will be copied. This is useful if for pig/hive jobs.
Examples:
package: org.apache.oozie.yamltest
name: TestWorkflowActionFromBundle
copy_dependencies_files_hdfs:
- /tmp/workflow-test/lib
Import
imports: List of Strings, Optional
A list of Java classes to import in the generated Oozie unit Java file. * Examples: *
import:
- org.apache.oozie.util.DateUtils
- java.util.Date
setup
setup: List of Strings, Optional
List of java statement that can be used for JUnit setup method.
setup can also include HDFS and HCat operation.
Initialization
initialization: List of Strings, Optional
List of java statement that can be used for initialization. This statement will be added first in test case.
Examples:
initialization:
- "Date date = new Date()"
- "String currentTime = DateUtils.formatDateOozieTZ(date)"
Job Definition
This section can be used to define job properties
job_property: String, Optional
job_property can be a Java Properties file (.properties) or a Hadoop XML Configuration file (.xml).
properties: String, Optional
Overridden key/value configuration use in job submission. This value overrides the value of job_property.
Examples:
job:
properties:
raw_logs_path: "/tmp/test"
aggregated_logs_path: "/user/test"
workflowPath: "/testDirPath"
start: "2010-02-01T10:00Z"
job:
job_property : src/test/resources/job-1.properties
properties:
oozie.wf.application.path: "/tmp/workflow-test/"
jobTracker: !func getJobTrackerUri()
nameNode: !func getNameNodeUri()
HDFS operation
Set of HDFS operation. Can be used to create dummy input dataset for coordinator or input file for workflow. Supported operation are
mkdir: List, Optional
The mkdir command creates the specified directory. If the directory already exists, it does a no-op.
touchz: List, Optional
Touchz creates a zero length file in the specified path if none exists. If one already exists, then touchz will perform a touch operation.
delete: List, Optional
List of file/dir to be deleted.
delete: List, Optional
Deletes the specified path.
chmod: List, Optional
The chmod command changes the permissions for the specified path.
permission: String, Optional
Permission of path. Permissions can be specified using the Unix Symbolic representation (e.g. -rwxrw-rw-) or an octal representation (755).
path: String, Optional
HDFS path of file/dir whose permission need to changed.
copyFromLocal: List, Optional
List of file to be copied to HDFS
source: String, mandatory
destination: String, mandatory
move: List, Optional
source: String, mandatory
destination: String, mandatory
Examples:
hdfs:
touchz:
- /tmp/test/2010/02/01/09/_SUCCESS
copyFromLocal:
- source: coordinator-test-3.xml
destination: /tmp/workflow-test/coordinator-test-3.xml
HCat operation
Set of HCat operation. Can be used to create HCat partition for coordinator/workflow. HCat operation can be configured in nested form. At parent level, at has two operation delete and create
create: Can be used to create HCat partition.
delete: Can be used to delete HCat partition.
Both delete and create have following structure.
databases:
- database: db_test
tables:
- table: table_test
columns: "dt,country"
partitions:
- dt=20100201;country=usa
database: List, Optional
List of databases to be created/deleted.
table: List, Optional
List of tables to create/delete for the database.
columns: String, Optional
Table columns definition.
partitions: List, Optional
partition to be created.
Examples:
hcat:
create:
databases:
- database: db_test
tables:
- table: table_test
columns: "dt,country"
partitions:
- dt=20100201;country=usa
- database: db_test_1
Job stubbing
List of coordinator job, workflow, workflow action, pig script, hive script, etc. to be skipped while running test It's a nested configuration.
skip_all_job_submission: boolean, Optional
All job submission will be skipped.
skip_pig_job_submission: boolean, Optional
All pig job submission will be skipped.
skip_hive_job_submission: boolean, Opt.ional
All hive job submission will be skipped.
skip_spark_job_submission: boolean, Optional
All spark job submission will be skipped.
skip_java_job_submission: boolean, Optional
All java job submission will be skipped.
skip_shell_job_submission: boolean, Optional
All shell job submission will be skipped.
skip_distcp_job_submission: boolean, Optional
All shell job submission will be skipped.
coordinator: List, Optional
List of coordinator to be skipped.
workflow_action:List, Optional
List of workflow action to be skipped.
skip_job_submission:boolean, Optional
Indicate that action should be executed, but only Hadoop job submission is ignored.
coordinators: List, Optional
List of coordinators to be skipped.
coordinator: String, Optional
Name of the coordinator which has to be skipped.
workflow:
just a marked that workflow of a coordinator will be skipped.
workflow_actions:
List of workflow action that will be skipped.
workflow_action: String, Optional
Name of the workflow action.
skip_job_submission:boolean, Optional
If true, job submission will be skipped.
status: String, Optional
Status of skipped action. It will be useful if somebody wants to test different execution path of workflow. The default value is OK. Supported status are OK and ERROR.
output: List, Optional
Copy output to for skipped action
source: String, Optional
destination: String, Optional
Usage Guideline
Skip is a nested configuration, if submitted job is bundle, to skip workflow action one need to do a nested configuration.
skip:
coordinators:
- coordinator: coord-1
- coordinator: coord-2
workflow:
- coordinator: coord-3
workflow:
workflow_actions:
- workflow_action : pig
skip_job_submission: true
status: OK
output :
- path:
source: /tmp/resources/action-1-output
destination: /tmp/test/output/
- path:
source: src/test/resources/action-2-output
destination: /tmp/test/output/
Above example will skip coord-1 submission. Coord-2 will be submitted, but there won't be any workflow submission. For coord-3, job submission of action "pig" will be skipped. File /tmp/resources/action-1-output and /tmp/resources/action-2-output will copied to path /tmp/test/output/.
If submitted job is coordinator below configuration can be used for skipping jobs.
Examples:
skip:
workflow:
workflow_actions:
- status: OK
output:
- source: src/test/resources/action-1-output
destination: "/tmp/test/output/"
- source: src/test/resources/action-2-output
destination: "/tmp/test/output/"
workflow_action: pig
skip_job_submission: true
- status: FAILED
workflow_action: hive-action
skip:
skip_all_job: true
validation
Validation is dived into two section.
output validation
job validation
output validation
TODO
job validation
This section can be used to validate job status and attributes. Job validation syntax is very similar to skip syntax. One need to use nested configuration to validate job.
sleep : integer, Optional
Thread sleep.
wait_for_status: integer, Optional
Testcase will wait for status.
status: String, Optional
Assert job status
not_status: String, Optional
Assert job status as false
nominal_time: String, Optional
Assert coord action nominal time
coordinators: List, Optional
List of coordinators for validation
coordinator: String, Optional
Name of the coordinator which will be used for validation
workflow_action: String, Optional
Name of the workflow action which will be used for validation
job_conf: List, Optional
Validate job conf.
key: String, Optional
expected_value: String, Optional
Usage Guideline
Like stubbing, validate_job can have nested configuration. If the submitted job is bundle then to validate workflow action, one need to use nested configuration.
Validating a Bundle Job
validate_job:
sleep: 60000
status: RUNNING
coordinators:
- coordinator: coord-3
not_status: PREP
coordinator_actions:
- coordinator_action : "@2"
sleep: 6000
wait_for_status: RUNNING
nominal_time: 2010-02-01T11:00Z
workflow:
workflow_actions:
- workflow_action : pig
status: OK
Validating a Coord Job
validations:
validate_job:
sleep: 6000
job_conf:
- key: queueName
expected_value: defult
coordinator_actions:
- coordinator_action : "@2"
not_status: WAITING
nominal_time: 2010-02-01T11:00Z
Validating a workflow Job
validations:
validate_job:
sleep: 60000
status: SUCCEEDED
workflow_actions:
- workflow_action : pig
status: OK
TearUp
tearUp: List of Strings, Optional
List of java statement that can be used for JUnit tearUp method.
tearUp can also include HDFS and HCat operation.
Tag
Oozie yaml also support tagging. Supported tags are
!func : This indicate that value is function, not string.
jobTracker: !func getJobTrackerUri(), this will be interpreted as jobTracker = getJobTrackerUri() . If !func tag is missing then it will be interpreted as jobTracker = "getJobTrackerUri()" .
!var : This indicate that value is variable, not string.
Examples:
initialization:
- "Date date = new Date()"
- "String currentTime = DateUtils.formatDateOozieTZ(date)"
job:
properties:
start: !var currentTime
!exp : This indicate that value is java statement, not string.
Examples:
touchz:
- !exp \"/tmp/test/\" + formatDate(currentTime, \"yyyy/MM/dd/HH\") + \"/_SUCCESS\"
Examples
Bundle Test
name: TestBundleDisableCoord
output_path: ./target/oozieyaml-sourcesjob:
properties:
raw_logs_path: "/tmp/test"
aggregated_logs_path: "/user/test"
workflowPath: "/testDirPath"
start: "2010-02-01T10:00Z"
end: "2012-02-03T23:59Z"
oozie.bundle.application.path: "/tmp/workflow-test/bundle-multiple-coord.xml"
queueName: "defult"
coord_path_1: "/tmp/workflow-test/coordinator-test-1.xml"
coord_path_2: "/tmp/workflow-test/coordinator-test-2.xml"
jobTracker: !func getJobTrackerUri()
nameNode: !func getNameNodeUri()
hdfs:
touchz:
- /tmp/test/2010/02/01/09/_SUCCESS
- /tmp/test/2010/02/01/10/_SUCCESS
- /tmp/test/2010/02/01/11/_SUCCESS
- /tmp/test/2010/02/01/12/_SUCCESS
copyFromLocal:
- source: bundle-multiple-coord.xml
destination: "/tmp/workflow-test/bundle-multiple-coord.xml"
- source: coordinator-test-1.xml
destination: /tmp/workflow-test/coordinator-test-1.xml
- source: coordinator-test-2.xml
skip:
coordinators:
- coordinator: coord_1
validations:
validate_job:
sleep: 6000
status: RUNNING
coordinators:
- coordinator: coord_2
not_status: PREP
coordinator_actions:
- coordinator_action : "@2"
not_status: WAITING
nominal_time: 2010-02-01T11:00Z
Coordinator test
name: TestCoordinator
output_path: ./target/oozieyaml-sourcesjob:
properties:
raw_logs_path: "/tmp/test"
aggregated_logs_path: "/user/test"
workflowPath: "/testDirPath"
start: "2010-02-01T10:00Z"
end: "2012-02-03T23:59Z"
oozie.coord.application.path: "/tmp/workflow-test/coordinator.xml"
queueName: "defult"
jobTracker: !func getJobTrackerUri()
nameNode: !func getNameNodeUri()
hdfs:
touchz:
- /tmp/test/2010/02/01/09/_SUCCESS
- /tmp/test/2010/02/01/10/_SUCCESS
- /tmp/test/2010/02/01/11/_SUCCESS
- /tmp/test/2010/02/01/12/_SUCCESS
copyFromLocal:
- source: coordinator-test.xml
destination: "/tmp/workflow-test/coordinator.xml"
validations:
validate_job:
sleep: 6000
job_conf:
- key: queueName
expected_value: defult
coordinator_actions:
- coordinator_action : "@2"
not_status: WAITING
nominal_time: 2010-02-01T11:00Z
Coordinator test with HCat
name: TestCoordinatorWithHcat
output_path: ./target/oozieyaml-sourcesjob:
properties:
raw_logs_path: "/tmp/test"
aggregated_logs_path: "/user/test"
workflowAppUri: "/testDirPath"
start: "2010-02-01T10:00Z"
end: "2012-02-03T23:59Z"
oozie.coord.application.path: "/tmp/workflow-test/coordinator.xml"
queueName: "defult"
db: db_test
table: table_test
dataOut: dataOut
jobTracker: !func getJobTrackerUri()
nameNode: !func getNameNodeUri()
hcatNode: !func \"hcat://\" + getMetastoreAuthority()
hdfs:
mkdir:
- "/tmp/workflow-test/"
copyFromLocal:
- source: coordinator-hcat-test.xml
destination: "/tmp/workflow-test/coordinator.xml"
hcat:
create:
databases:
- database: db_test
tables:
- table: table_test
columns: "dt,country"
partitions:
- dt=20100201;country=usa
validations:
validate_job:
sleep: 6000
coordinator_actions:
- coordinator_action : "@1"
not_status: WAITING
nominal_time: 2010-02-01T10:00Z
Workflow Test
name: TestWorkflowWithPig
output_path: ./target/oozieyaml-sources
copy_dependencies_files_hdfs:
- /tmp/workflow-test/libjob:
properties:
INPUT: "/tmp/input/dummp.txt"
oozie.wf.application.path: "/tmp/workflow-test/"
queueName: "defult"
OUTPUT: "/tmp/output"
jobTracker: !func getJobTrackerUri()
nameNode: !func getNameNodeUri()
hdfs:
mkdir:
- "/tmp/workflow-test/"
touchz:
- /tmp/input/dummp.txt
copyFromLocal:
- source: echo.pig
destination: "/tmp/workflow-test/echo.pig"
- source: workflow-test-with-pig.xml
destination: "/tmp/workflow-test/workflow.xml"
validations:
validate_job:
sleep: 60000
status: SUCCEEDED
workflow_actions:
- workflow_action : pig
status: OK
Workflow Test with skip Job submission
name: TestWorkflowJobStatus
output_path: ./target/oozieyaml-sources
copy_dependencies_files_hdfs:
- /tmp/workflow-test/libjob:
properties:
INPUT: "/tmp/input/dummp.txt"
oozie.wf.application.path: "/tmp/workflow-test/"
queueName: "defult"
OUTPUT: "/tmp/output"
jobTracker: !func getJobTrackerUri()
nameNode: !func getNameNodeUri()
hdfs:
mkdir:
- "/tmp/workflow-test/"
touchz:
- /tmp/input/dummp.txt
copyFromLocal:
- source: echo.pig
destination: "/tmp/workflow-test/echo.pig"
- source: workflow-test-with-output.xml
destination: "/tmp/workflow-test/workflow.xml"
skip:
workflow_actions:
- workflow_action : pig
skip_job_submission: true
status: OK
output:
- source: /tmp/workflow-test/echo.pig
destination: "/tmp/workflow-test/dummy.pig"
- workflow_action : pig_2
skip_job_submission: true
status: ERROR
validations:
validate_job:
wait_for_status: KILLED
workflow_actions:
- workflow_action : pig
status: OK
- workflow_action : pig_1
status: OK
- workflow_action : pig_2
status: ERROR
API
String getJobTrackerUri():
Returns job tracker URI.
String getNameNodeUri():
Returns name node URI.
String getMetastoreAuthority()
Returns hcat URI ( without hcat:// prefix).
FileSystem getFileSystem():
Returns filesystem.
Date addDays(Date date, int amount)
Return new date after adding days.
Date addHours(Date date, int amount)
Return new date after adding hours.
Date addMonths(Date date, int amount)
Return new date after adding months.
Date addMinutes(Date date, int amount)
Return new date after adding minutes.
String addDays(String date, int amount)
Return new date after adding days. Input date is in TZ format.
String addHours(String date, int amount)
Return new date after adding hours. Input date is in TZ format.
String addMonths(String date, int amount)
Return new date after adding months. Input date is in TZ format.
String addMinutes(String date, int amount)
Return new date after adding minutes. Input date is in TZ format.
String formatDate(String date, String format)
Format date. Input date is in TZ format.
String formatDate(Date date, String format)
Diffs (updated)
-----
client/src/main/java/org/apache/oozie/client/WorkflowJob.java 131d9b2336d7b643927af1cb65aceac95c6dc171
core/src/main/java/org/apache/oozie/WorkflowJobBean.java 028164dc61252f3d5e3113385233e295c63a35f8
core/src/main/java/org/apache/oozie/action/ActionExecutor.java 919509d35aefae8eed60849ca9c1da4e7ea291de
core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java c2a6192c44fe8c647253c58274569dd03994344d
core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 23e1f69e7facd6b0a240ea9719fd2f93f766feaa
core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 338e508571d31790df43f69a2eb94293b946de5e
core/src/main/java/org/apache/oozie/command/bundle/BundleStartXCommand.java cc98a6db803cebbefc68d7f8dd264e24d933c9eb
core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java 16d70362a1bc49612138dcd3e0120d77cd8decd0
core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java e95a60ad49f503695cc5fce24a67924dc6ec60e5
core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java f50c1a49a81beb0d2d95805880b78dd7635c0d88
core/src/main/java/org/apache/oozie/mini/MiniOozie.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieActionExecutorUtil.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieDistcpActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieEmailActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieExecutorUtil.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieFsActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieHive2ActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieHiveActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieJavaActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieLocalClient.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieMapReduceActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOoziePigActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieShellActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieSparkActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/mini/MiniOozieSubWorkflowActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
core/src/main/java/org/apache/oozie/store/WorkflowStore.java 821abc53f46f069e5040a82fc1ef2d6b8f69f72b
core/src/main/java/org/apache/oozie/util/JobUtils.java 681b58a850f8b9888e31324e2a5b1e78711222ef
core/src/main/resources/localoozie-log4j.properties 906397fe0f284c187a8f05ece7a1a0677572e845
core/src/test/java/org/apache/oozie/test/MiniOozieTestCase.java 12ede028655f2a30613a9f802e48cd3a27af6ed9
core/src/test/java/org/apache/oozie/test/XFsTestCase.java 2b33d9379fc7b250ef6cf9b8860c0369b51623f6
core/src/test/java/org/apache/oozie/test/XTestCase.java 0ef4a4be6998744914e2ab56697cc2ae517070b9
core/src/test/resources/test-oozie-log4j.properties 3040d4f621b785d98cdccae2fa039e19f7069484
docs/src/site/twiki/YamlUnit.twiki e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
docs/src/site/twiki/index.twiki b7c962dbcf02dddb14a93ee11f012458b015c3fe
minitest/pom.xml 10a89b06ab1d9baad801818a77c8cb84ed941e57
minitest/src/main/java/org/apache/oozie/test/MiniOozieTestCase.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/java/org/apache/oozie/test/TestBundle.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/java/org/apache/oozie/test/TestCoordinator.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/java/org/apache/oozie/test/TestCoordinatorWithHCat.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/java/org/apache/oozie/test/TestWorkflow.java 4257b60b132d40a9bbaf24ec75ae1a125ecb6855
minitest/src/test/java/org/apache/oozie/test/TestWorkflowWithCert.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/java/org/apache/oozie/test/TestWorkflowWithHive.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/java/org/apache/oozie/test/TestWorkflowWithPig.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/bundle-test.properties e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/bundle-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/coordinator-test-withhcat.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/coordinator-test.properties e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/coordinator-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/echo.pig e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/hive.script e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/oozie-log4j.properties 2503851ad143c4e4384e8760219dd590f1cfa6d4
minitest/src/test/resources/wf-hive-job.properties e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/wf-pig-job.properties e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/wf-test-with-cert.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/wf-test-with-hive.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
minitest/src/test/resources/wf-test-with-pig.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
pom.xml 0d1105b2bae0f310406b7dc4d5905284ca3ebf98
yaml-unit/examples/bundle-test/pom.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/oozieyaml/test-bundle-same-name-coords.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/oozieyaml/test-bundle.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/oozieyaml/test-workflow-action-from-bundle.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/bundle-multiple-coord-1.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/bundle-multiple-coord.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/bundle-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/coordinator-test-1.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/coordinator-test-2.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/coordinator-test-3.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/echo.pig e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/wf.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/bundle-test/src/test/resources/workflow-test-with-pig.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/coordinator-test/pom.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/coordinator-test/src/test/oozieyaml/test-coordinator-current-time.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/coordinator-test/src/test/oozieyaml/test-coordinator-hcat.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/coordinator-test/src/test/oozieyaml/test-coordinator.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/coordinator-test/src/test/resources/coordinator-hcat-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/coordinator-test/src/test/resources/coordinator-latest-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/coordinator-test/src/test/resources/coordinator-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/pom.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/pom.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-all-action.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-counters.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-init.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-job-definition.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-skip-all-job.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-skip-status.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-withsetup.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/resources/echo.pig e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/resources/job-1.properties e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/resources/properties.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/resources/workflow-test-with-all-action.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/resources/workflow-test-with-counters.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/resources/workflow-test-with-output.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/examples/workflow-test/src/test/resources/workflow-test-with-pig.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/pom.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/GenerateBatchJsonMojo.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/GenerateMojo.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/TestBuilder.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/YamlTagConstructor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/Import.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/InitStatement.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/JobDefinition.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/SetupStatement.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/Statement.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/Tags.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/TearDownStatement.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/TestClassBuilder.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/UnitTestGenerator.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/UnitTestGeneratorUtil.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/YamlConfig.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/YamlString.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/AddPartitions.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/CreateDatabase.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/CreateHCatPartition.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/CreateTable.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/DeleteDatabase.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/DeleteHCatPartition.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/DeletePartitions.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/DeleteTable.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/HCatOperation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/CopyDependenciesFilesHdfs.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/CopyFromLocal.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsChmod.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsDelete.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsMkdir.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsMove.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsOperation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsTouchz.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/Skip.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/SkipCoordJob.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/SkipJobSubmission.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/SkipWorkflow.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/SkipWorkflowAction.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/SkipWorkflowDeserializer.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/YamlStringDeserializer.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/JobCheck.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/JobConfValidation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/NotStatusValidation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/Sleep.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/StatusValidation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateBundle.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateCoodJob.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateCoordAction.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateJob.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateWorkflow.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateWorkflowAction.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidationOutPut.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/Validations.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/WaitForJobStatus.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/main/resources/META-INF/m2e/lifecycle-mapping-metadata.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestHdfs.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestJobValidation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestSetup.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestSkip.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/YamlTestUtil.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
Diff: https://reviews.apache.org/r/52373/diff/2/
Changes: https://reviews.apache.org/r/52373/diff/1-2/
Testing
-------
Thanks,
Purshotam Shah
Re: Review Request 52373: OOZIE-2688 YAML representation of Oozie unit
testcase
Posted by Satish Saley <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52373/#review151099
-----------------------------------------------------------
yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/GenerateBatchJsonMojo.java (line 19)
<https://reviews.apache.org/r/52373/#comment219228>
Can we rename it to org.apache.oozie.yamlunit?
We are already under "oozie", so prefix "oozie" in "oozieyamlunit" not needed.
- Satish Saley
On Sept. 28, 2016, 5:53 p.m., Purshotam Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52373/
> -----------------------------------------------------------
>
> (Updated Sept. 28, 2016, 5:53 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-2688
> https://issues.apache.org/jira/browse/OOZIE-2688
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> Oozie YAML Unit test
>
> Maven configuration
> YAML configuration
> Declaration
> Import
> setup
> Initialization
> Job Definition
> HDFS operation
> HCat operation
> Job stubbing
> Usage Guideline
> validation
> output validation
> job validation
> Usage Guideline
> TearUp
> Tag
> Examples
> Bundle Test
> Coordinator test
> Coordinator test with HCat
> Workflow Test
> Workflow Test with skip Job submission
> API
> Maven configuration
>
> To enable oozie unit yaml, one need add oozie-yaml plugin to maven.
>
> <plugin>
> <groupId>org.apache.oozie</groupId>
> <artifactId>oozie-yaml</artifactId>
> <version>0.1-SNAPSHOT</version>
> <executions>
> <execution>
> <goals>
> <goal>oozieyaml</goal>
> </goals>
> </execution>
> </executions>
> </plugin>
> Oozie-YAML plugin to Maven will search for any oozie YAML file and if it's there, it will convert it into respective JUnit test case and will execute them. There will be a surefire report.
> YAML configuration
>
> Whole YAML configuration is divided into following sections
>
> Declaration
> Import
> Setup
> Initialization
> Job Definition
> Stubbing
> HDFS operation
> HCat operation
> Validation
> TearUp
> Tag
> Declaration
>
> This section can be used to defined test case properties like test case name, package name, etc.
>
> name: String, required
> Name of the test cases. Java file will be created with the same name.
> package: String, Optional
> The Java package of the generated test class.
> skip_test : boolean, Optional
> A boolean that indicates whether or not to execute this test. This parameter defaults to false.
> copy_dependencies_files_hdfs: List of Strings, Optional
> List of HDFS of paths where all Maven dependent jar will be copied. This is useful if for pig/hive jobs.
> Examples:
>
> package: org.apache.oozie.yamltest
> name: TestWorkflowActionFromBundle
> copy_dependencies_files_hdfs:
> - /tmp/workflow-test/lib
> Import
>
> imports: List of Strings, Optional
> A list of Java classes to import in the generated Oozie unit Java file. * Examples: *
> import:
> - org.apache.oozie.util.DateUtils
> - java.util.Date
> setup
>
> setup: List of Strings, Optional
> List of java statement that can be used for JUnit setup method.
> setup can also include HDFS and HCat operation.
> Initialization
>
> initialization: List of Strings, Optional
> List of java statement that can be used for initialization. This statement will be added first in test case.
> Examples:
>
> initialization:
> - "Date date = new Date()"
> - "String currentTime = DateUtils.formatDateOozieTZ(date)"
> Job Definition
>
> This section can be used to define job properties
>
> job_property: String, Optional
> job_property can be a Java Properties file (.properties) or a Hadoop XML Configuration file (.xml).
> properties: String, Optional
> Overridden key/value configuration use in job submission. This value overrides the value of job_property.
> Examples:
>
> job:
> properties:
> raw_logs_path: "/tmp/test"
> aggregated_logs_path: "/user/test"
> workflowPath: "/testDirPath"
> start: "2010-02-01T10:00Z"
> job:
> job_property : src/test/resources/job-1.properties
> properties:
> oozie.wf.application.path: "/tmp/workflow-test/"
> jobTracker: !func getJobTrackerUri()
> nameNode: !func getNameNodeUri()
> HDFS operation
>
> Set of HDFS operation. Can be used to create dummy input dataset for coordinator or input file for workflow. Supported operation are
>
> mkdir: List, Optional
> The mkdir command creates the specified directory. If the directory already exists, it does a no-op.
> touchz: List, Optional
> Touchz creates a zero length file in the specified path if none exists. If one already exists, then touchz will perform a touch operation.
> delete: List, Optional
> List of file/dir to be deleted.
> delete: List, Optional
> Deletes the specified path.
> chmod: List, Optional
> The chmod command changes the permissions for the specified path.
> permission: String, Optional
> Permission of path. Permissions can be specified using the Unix Symbolic representation (e.g. -rwxrw-rw-) or an octal representation (755).
> path: String, Optional
> HDFS path of file/dir whose permission need to changed.
> copyFromLocal: List, Optional
> List of file to be copied to HDFS
> source: String, mandatory
> destination: String, mandatory
> move: List, Optional
> source: String, mandatory
> destination: String, mandatory
> Examples:
>
> hdfs:
> touchz:
> - /tmp/test/2010/02/01/09/_SUCCESS
> copyFromLocal:
> - source: coordinator-test-3.xml
> destination: /tmp/workflow-test/coordinator-test-3.xml
> HCat operation
>
> Set of HCat operation. Can be used to create HCat partition for coordinator/workflow. HCat operation can be configured in nested form. At parent level, at has two operation delete and create
>
> create: Can be used to create HCat partition.
> delete: Can be used to delete HCat partition.
> Both delete and create have following structure.
> databases:
> - database: db_test
> tables:
> - table: table_test
> columns: "dt,country"
> partitions:
> - dt=20100201;country=usa
> database: List, Optional
> List of databases to be created/deleted.
> table: List, Optional
> List of tables to create/delete for the database.
> columns: String, Optional
> Table columns definition.
> partitions: List, Optional
> partition to be created.
> Examples:
>
> hcat:
> create:
> databases:
> - database: db_test
> tables:
> - table: table_test
> columns: "dt,country"
> partitions:
> - dt=20100201;country=usa
> - database: db_test_1
> Job stubbing
>
> List of coordinator job, workflow, workflow action, pig script, hive script, etc. to be skipped while running test It's a nested configuration.
>
> skip_all_job_submission: boolean, Optional
> All job submission will be skipped.
> skip_pig_job_submission: boolean, Optional
> All pig job submission will be skipped.
> skip_hive_job_submission: boolean, Opt.ional
> All hive job submission will be skipped.
> skip_spark_job_submission: boolean, Optional
> All spark job submission will be skipped.
> skip_java_job_submission: boolean, Optional
> All java job submission will be skipped.
> skip_shell_job_submission: boolean, Optional
> All shell job submission will be skipped.
> skip_distcp_job_submission: boolean, Optional
> All shell job submission will be skipped.
> coordinator: List, Optional
> List of coordinator to be skipped.
> workflow_action:List, Optional
> List of workflow action to be skipped.
> skip_job_submission:boolean, Optional
> Indicate that action should be executed, but only Hadoop job submission is ignored.
> coordinators: List, Optional
> List of coordinators to be skipped.
> coordinator: String, Optional
> Name of the coordinator which has to be skipped.
> workflow:
> just a marked that workflow of a coordinator will be skipped.
> workflow_actions:
> List of workflow action that will be skipped.
> workflow_action: String, Optional
> Name of the workflow action.
> skip_job_submission:boolean, Optional
> If true, job submission will be skipped.
> status: String, Optional
> Status of skipped action. It will be useful if somebody wants to test different execution path of workflow. The default value is OK. Supported status are OK and ERROR.
> output: List, Optional
> Copy output to for skipped action
> source: String, Optional
> destination: String, Optional
> Usage Guideline
>
> Skip is a nested configuration, if submitted job is bundle, to skip workflow action one need to do a nested configuration.
>
> skip:
> coordinators:
> - coordinator: coord-1
> - coordinator: coord-2
> workflow:
> - coordinator: coord-3
> workflow:
> workflow_actions:
> - workflow_action : pig
> skip_job_submission: true
> status: OK
> output :
> - path:
> source: /tmp/resources/action-1-output
> destination: /tmp/test/output/
> - path:
> source: src/test/resources/action-2-output
> destination: /tmp/test/output/
> Above example will skip coord-1 submission. Coord-2 will be submitted, but there won't be any workflow submission. For coord-3, job submission of action "pig" will be skipped. File /tmp/resources/action-1-output and /tmp/resources/action-2-output will copied to path /tmp/test/output/.
> If submitted job is coordinator below configuration can be used for skipping jobs.
> Examples:
> skip:
> workflow:
> workflow_actions:
> - status: OK
> output:
> - source: src/test/resources/action-1-output
> destination: "/tmp/test/output/"
> - source: src/test/resources/action-2-output
> destination: "/tmp/test/output/"
> workflow_action: pig
> skip_job_submission: true
> - status: FAILED
> workflow_action: hive-action
> skip:
> skip_all_job: true
> validation
>
> Validation is dived into two section.
>
> output validation
> job validation
> output validation
>
> TODO
>
> job validation
>
> This section can be used to validate job status and attributes. Job validation syntax is very similar to skip syntax. One need to use nested configuration to validate job.
>
> sleep : integer, Optional
> Thread sleep.
> wait_for_status: integer, Optional
> Testcase will wait for status.
> status: String, Optional
> Assert job status
> not_status: String, Optional
> Assert job status as false
> nominal_time: String, Optional
> Assert coord action nominal time
> coordinators: List, Optional
> List of coordinators for validation
> coordinator: String, Optional
> Name of the coordinator which will be used for validation
> workflow_action: String, Optional
> Name of the workflow action which will be used for validation
> job_conf: List, Optional
> Validate job conf.
> key: String, Optional
> expected_value: String, Optional
> Usage Guideline
>
> Like stubbing, validate_job can have nested configuration. If the submitted job is bundle then to validate workflow action, one need to use nested configuration.
>
> Validating a Bundle Job
> validate_job:
> sleep: 60000
> status: RUNNING
> coordinators:
> - coordinator: coord-3
> not_status: PREP
> coordinator_actions:
> - coordinator_action : "@2"
> sleep: 6000
> wait_for_status: RUNNING
> nominal_time: 2010-02-01T11:00Z
> workflow:
> workflow_actions:
> - workflow_action : pig
> status: OK
> Validating a Coord Job
> validations:
> validate_job:
> sleep: 6000
> job_conf:
> - key: queueName
> expected_value: defult
> coordinator_actions:
> - coordinator_action : "@2"
> not_status: WAITING
> nominal_time: 2010-02-01T11:00Z
> Validating a workflow Job
> validations:
> validate_job:
> sleep: 60000
> status: SUCCEEDED
> workflow_actions:
> - workflow_action : pig
> status: OK
> TearUp
>
> tearUp: List of Strings, Optional
> List of java statement that can be used for JUnit tearUp method.
> tearUp can also include HDFS and HCat operation.
> Tag
>
> Oozie yaml also support tagging. Supported tags are
>
> !func : This indicate that value is function, not string.
> jobTracker: !func getJobTrackerUri(), this will be interpreted as jobTracker = getJobTrackerUri() . If !func tag is missing then it will be interpreted as jobTracker = "getJobTrackerUri()" .
> !var : This indicate that value is variable, not string.
> Examples:
> initialization:
> - "Date date = new Date()"
> - "String currentTime = DateUtils.formatDateOozieTZ(date)"
> job:
> properties:
> start: !var currentTime
> !exp : This indicate that value is java statement, not string.
> Examples:
> touchz:
> - !exp \"/tmp/test/\" + formatDate(currentTime, \"yyyy/MM/dd/HH\") + \"/_SUCCESS\"
>
> Examples
>
> Bundle Test
>
> name: TestBundleDisableCoord
> output_path: ./target/oozieyaml-sourcesjob:
> properties:
> raw_logs_path: "/tmp/test"
> aggregated_logs_path: "/user/test"
> workflowPath: "/testDirPath"
> start: "2010-02-01T10:00Z"
> end: "2012-02-03T23:59Z"
> oozie.bundle.application.path: "/tmp/workflow-test/bundle-multiple-coord.xml"
> queueName: "defult"
> coord_path_1: "/tmp/workflow-test/coordinator-test-1.xml"
> coord_path_2: "/tmp/workflow-test/coordinator-test-2.xml"
> jobTracker: !func getJobTrackerUri()
> nameNode: !func getNameNodeUri()
> hdfs:
> touchz:
> - /tmp/test/2010/02/01/09/_SUCCESS
> - /tmp/test/2010/02/01/10/_SUCCESS
> - /tmp/test/2010/02/01/11/_SUCCESS
> - /tmp/test/2010/02/01/12/_SUCCESS
> copyFromLocal:
> - source: bundle-multiple-coord.xml
> destination: "/tmp/workflow-test/bundle-multiple-coord.xml"
> - source: coordinator-test-1.xml
> destination: /tmp/workflow-test/coordinator-test-1.xml
> - source: coordinator-test-2.xml
> skip:
> coordinators:
> - coordinator: coord_1
> validations:
> validate_job:
> sleep: 6000
> status: RUNNING
> coordinators:
> - coordinator: coord_2
> not_status: PREP
> coordinator_actions:
> - coordinator_action : "@2"
> not_status: WAITING
> nominal_time: 2010-02-01T11:00Z
> Coordinator test
>
> name: TestCoordinator
> output_path: ./target/oozieyaml-sourcesjob:
> properties:
> raw_logs_path: "/tmp/test"
> aggregated_logs_path: "/user/test"
> workflowPath: "/testDirPath"
> start: "2010-02-01T10:00Z"
> end: "2012-02-03T23:59Z"
> oozie.coord.application.path: "/tmp/workflow-test/coordinator.xml"
> queueName: "defult"
> jobTracker: !func getJobTrackerUri()
> nameNode: !func getNameNodeUri()
> hdfs:
> touchz:
> - /tmp/test/2010/02/01/09/_SUCCESS
> - /tmp/test/2010/02/01/10/_SUCCESS
> - /tmp/test/2010/02/01/11/_SUCCESS
> - /tmp/test/2010/02/01/12/_SUCCESS
> copyFromLocal:
> - source: coordinator-test.xml
> destination: "/tmp/workflow-test/coordinator.xml"
> validations:
> validate_job:
> sleep: 6000
> job_conf:
> - key: queueName
> expected_value: defult
> coordinator_actions:
> - coordinator_action : "@2"
> not_status: WAITING
> nominal_time: 2010-02-01T11:00Z
> Coordinator test with HCat
>
> name: TestCoordinatorWithHcat
> output_path: ./target/oozieyaml-sourcesjob:
> properties:
> raw_logs_path: "/tmp/test"
> aggregated_logs_path: "/user/test"
> workflowAppUri: "/testDirPath"
> start: "2010-02-01T10:00Z"
> end: "2012-02-03T23:59Z"
> oozie.coord.application.path: "/tmp/workflow-test/coordinator.xml"
> queueName: "defult"
> db: db_test
> table: table_test
> dataOut: dataOut
> jobTracker: !func getJobTrackerUri()
> nameNode: !func getNameNodeUri()
> hcatNode: !func \"hcat://\" + getMetastoreAuthority()
> hdfs:
> mkdir:
> - "/tmp/workflow-test/"
> copyFromLocal:
> - source: coordinator-hcat-test.xml
> destination: "/tmp/workflow-test/coordinator.xml"
> hcat:
> create:
> databases:
> - database: db_test
> tables:
> - table: table_test
> columns: "dt,country"
> partitions:
> - dt=20100201;country=usa
> validations:
> validate_job:
> sleep: 6000
> coordinator_actions:
> - coordinator_action : "@1"
> not_status: WAITING
> nominal_time: 2010-02-01T10:00Z
> Workflow Test
>
> name: TestWorkflowWithPig
> output_path: ./target/oozieyaml-sources
> copy_dependencies_files_hdfs:
> - /tmp/workflow-test/libjob:
> properties:
> INPUT: "/tmp/input/dummp.txt"
> oozie.wf.application.path: "/tmp/workflow-test/"
> queueName: "defult"
> OUTPUT: "/tmp/output"
> jobTracker: !func getJobTrackerUri()
> nameNode: !func getNameNodeUri()
> hdfs:
> mkdir:
> - "/tmp/workflow-test/"
> touchz:
> - /tmp/input/dummp.txt
> copyFromLocal:
> - source: echo.pig
> destination: "/tmp/workflow-test/echo.pig"
> - source: workflow-test-with-pig.xml
> destination: "/tmp/workflow-test/workflow.xml"
> validations:
> validate_job:
> sleep: 60000
> status: SUCCEEDED
> workflow_actions:
> - workflow_action : pig
> status: OK
> Workflow Test with skip Job submission
>
> name: TestWorkflowJobStatus
> output_path: ./target/oozieyaml-sources
> copy_dependencies_files_hdfs:
> - /tmp/workflow-test/libjob:
> properties:
> INPUT: "/tmp/input/dummp.txt"
> oozie.wf.application.path: "/tmp/workflow-test/"
> queueName: "defult"
> OUTPUT: "/tmp/output"
> jobTracker: !func getJobTrackerUri()
> nameNode: !func getNameNodeUri()
> hdfs:
> mkdir:
> - "/tmp/workflow-test/"
> touchz:
> - /tmp/input/dummp.txt
> copyFromLocal:
> - source: echo.pig
> destination: "/tmp/workflow-test/echo.pig"
> - source: workflow-test-with-output.xml
> destination: "/tmp/workflow-test/workflow.xml"
> skip:
> workflow_actions:
> - workflow_action : pig
> skip_job_submission: true
> status: OK
> output:
> - source: /tmp/workflow-test/echo.pig
> destination: "/tmp/workflow-test/dummy.pig"
> - workflow_action : pig_2
> skip_job_submission: true
> status: ERROR
> validations:
> validate_job:
> wait_for_status: KILLED
> workflow_actions:
> - workflow_action : pig
> status: OK
> - workflow_action : pig_1
> status: OK
> - workflow_action : pig_2
> status: ERROR
> API
>
> String getJobTrackerUri():
> Returns job tracker URI.
> String getNameNodeUri():
> Returns name node URI.
> String getMetastoreAuthority()
> Returns hcat URI ( without hcat:// prefix).
> FileSystem getFileSystem():
> Returns filesystem.
> Date addDays(Date date, int amount)
> Return new date after adding days.
> Date addHours(Date date, int amount)
> Return new date after adding hours.
> Date addMonths(Date date, int amount)
> Return new date after adding months.
> Date addMinutes(Date date, int amount)
> Return new date after adding minutes.
> String addDays(String date, int amount)
> Return new date after adding days. Input date is in TZ format.
> String addHours(String date, int amount)
> Return new date after adding hours. Input date is in TZ format.
> String addMonths(String date, int amount)
> Return new date after adding months. Input date is in TZ format.
> String addMinutes(String date, int amount)
> Return new date after adding minutes. Input date is in TZ format.
> String formatDate(String date, String format)
> Format date. Input date is in TZ format.
> String formatDate(Date date, String format)
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/oozie/client/WorkflowJob.java 131d9b2336d7b643927af1cb65aceac95c6dc171
> core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc
> core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797
> core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d
> core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 65996d97145e85b1e4d1de87fe9891921df8f0db
> core/src/main/java/org/apache/oozie/command/bundle/BundleStartXCommand.java cc98a6db803cebbefc68d7f8dd264e24d933c9eb
> core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java 45abe5a5e88f482c6b0e8c6385d7ea054d242c7d
> core/src/main/java/org/apache/oozie/command/wf/ActionEndXCommand.java 740b8d3f101352e951001af689e541ffbf606b52
> core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java e95a60ad49f503695cc5fce24a67924dc6ec60e5
> core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java 3c032678d06641ae7bb2f2a9c0631948190ea662
> core/src/main/java/org/apache/oozie/mini/MiniOozie.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> core/src/main/java/org/apache/oozie/mini/MiniOozieActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> core/src/main/java/org/apache/oozie/mini/MiniOozieDistcpActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> core/src/main/java/org/apache/oozie/mini/MiniOozieEmailActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> core/src/main/java/org/apache/oozie/mini/MiniOozieExecutorUtil.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> core/src/main/java/org/apache/oozie/mini/MiniOozieFsActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> core/src/main/java/org/apache/oozie/mini/MiniOozieHive2ActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> core/src/main/java/org/apache/oozie/mini/MiniOozieHiveActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> core/src/main/java/org/apache/oozie/mini/MiniOozieJavaActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> core/src/main/java/org/apache/oozie/mini/MiniOozieLocalClient.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> core/src/main/java/org/apache/oozie/mini/MiniOozieMapReduceActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> core/src/main/java/org/apache/oozie/mini/MiniOoziePigActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> core/src/main/java/org/apache/oozie/mini/MiniOozieShellActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> core/src/main/java/org/apache/oozie/mini/MiniOozieSparkActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> core/src/main/java/org/apache/oozie/mini/MiniOozieSubWorkflowActionExecutor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> core/src/main/java/org/apache/oozie/service/LiteWorkflowStoreService.java ffc29af2f834da6d0890ea7215c5a62fd7cd693e
> core/src/main/java/org/apache/oozie/store/WorkflowStore.java c565e74893b863caef6c93015cfe38fe520d04ec
> core/src/main/java/org/apache/oozie/util/JobUtils.java a7a53b355d194b55959ede20c85d38e02294853d
> core/src/main/resources/localoozie-log4j.properties 906397fe0f284c187a8f05ece7a1a0677572e845
> core/src/main/resources/oozie-log4j.properties c86b30143f5f58135493d9e6bdba29f402a0b1fd
> core/src/test/java/org/apache/oozie/test/MiniHCatServer.java 8699ff881b2f27d8369b1a66083fcf785a56ef0f
> core/src/test/java/org/apache/oozie/test/MiniOozieTestCase.java 12ede028655f2a30613a9f802e48cd3a27af6ed9
> core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7
> core/src/test/java/org/apache/oozie/test/XTestCase.java 70f77be191bd844759ab1506a8d4c7aa4cb74f4c
> core/src/test/resources/test-oozie-log4j.properties 3040d4f621b785d98cdccae2fa039e19f7069484
> docs/src/site/twiki/YamlUnit.twiki e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> docs/src/site/twiki/index.twiki 8591530a2418b9d2bd4cdf5f25597bdd18ee4a75
> minitest/pom.xml 21aa57534c3d89cd1c349ec81b788b9694b59bb9
> minitest/src/main/java/org/apache/oozie/test/MiniOozieTestCase.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/java/org/apache/oozie/test/TestBundle.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/java/org/apache/oozie/test/TestCoordinator.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/java/org/apache/oozie/test/TestCoordinatorWithHCat.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/java/org/apache/oozie/test/TestWorkflow.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/java/org/apache/oozie/test/TestWorkflowWithCert.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/java/org/apache/oozie/test/TestWorkflowWithHive.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/java/org/apache/oozie/test/TestWorkflowWithPig.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/resources/bundle-test.properties e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/resources/bundle-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/resources/coordinator-test-withhcat.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/resources/coordinator-test.properties e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/resources/coordinator-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/resources/echo.pig e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/resources/hive.script e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/resources/oozie-log4j.properties c142d725140930bfa89cd2b163d0768a4c3a750a
> minitest/src/test/resources/wf-hive-job.properties e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/resources/wf-pig-job.properties e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/resources/wf-test-with-cert.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/resources/wf-test-with-hive.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> minitest/src/test/resources/wf-test-with-pig.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> pom.xml 704a2eeee9f4e4805e3e08c2a547b2a375f6b1b2
> yaml-unit/doc/OozieYamlUnit.twiki e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/doc/hdfs.yaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/doc/skip.yaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/doc/yaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/bundle-test/pom.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/bundle-test/src/test/oozieyaml/test-bundle.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/bundle-test/src/test/oozieyaml/test-workflow-action-from-bundle.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/bundle-test/src/test/resources/bundle-multiple-coord-1.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/bundle-test/src/test/resources/bundle-multiple-coord.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/bundle-test/src/test/resources/bundle-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/bundle-test/src/test/resources/coordinator-test-1.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/bundle-test/src/test/resources/coordinator-test-2.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/bundle-test/src/test/resources/coordinator-test-3.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/bundle-test/src/test/resources/echo.pig e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/bundle-test/src/test/resources/wf.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/bundle-test/src/test/resources/workflow-test-with-pig.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/coordinator-test/pom.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/coordinator-test/src/test/oozieyaml/test-coordinator-current-time.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/coordinator-test/src/test/oozieyaml/test-coordinator-hcat.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/coordinator-test/src/test/oozieyaml/test-coordinator.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/coordinator-test/src/test/resources/coordinator-hcat-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/coordinator-test/src/test/resources/coordinator-latest-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/coordinator-test/src/test/resources/coordinator-test.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/pom.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/workflow-test/pom.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-all-action.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-init.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-job-definition.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-skip-status.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow-withsetup.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/workflow-test/src/test/oozieyaml/test-workflow.oozieyaml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/workflow-test/src/test/resources/echo.pig e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/workflow-test/src/test/resources/job-1.properties e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/workflow-test/src/test/resources/properties.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/workflow-test/src/test/resources/workflow-test-with-all-action.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/workflow-test/src/test/resources/workflow-test-with-output.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/examples/workflow-test/src/test/resources/workflow-test-with-pig.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/pom.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/GenerateBatchJsonMojo.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/GenerateMojo.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/TestBuilder.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/YamlTagConstructor.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/CopyDependenciesFilesHdfs.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/Import.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/InitStatement.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/JobDefinition.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/SetupStatement.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/Statement.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/Tags.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/TearDownStatement.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/TestClassBuilder.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/UnitTestGenerator.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/UnitTestGeneratorUtil.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/YamlConfig.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/YamlString.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/AddPartitions.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/CreateDatabase.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/CreateHCatPartition.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/CreateTable.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/DeleteDatabase.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/DeleteHCatPartition.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/DeletePartitions.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/DeleteTable.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hcat/HCatOperation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/CopyDependenciesFilesHdfs.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/CopyFromLocal.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsChmod.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsDelete.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsMkdir.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsMove.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsOperation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/HdfsTouchz.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/Mkdir.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/hdfs/Touchz.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/Skip.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/SkipCoordJob.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/SkipJobSubmission.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/SkipWorkflow.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/SkipWorkflowAction.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/SkipWorkflowDeserializer.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/skip/YamlStringDeserializer.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/JobCheck.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/JobConfValidation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/NotStatusValidation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/Sleep.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/StatusValidation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateBundle.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateCoodJob.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateCoordAction.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateJob.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateWorkflow.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidateWorkflowAction.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/ValidationOutPut.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/Validations.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/java/org/apache/oozie/oozieyamlunit/config/validate/WaitForJobStatus.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/main/resources/META-INF/m2e/lifecycle-mapping-metadata.xml e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/A.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/Job.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/Properties.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestHdfs.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestJobValidation.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestOozieYaml.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestSetup.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestSkip.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/TestTouchWithTag.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> yaml-unit/src/test/java/org/apache/oozie/oozieyamlunit/YamlTestUtil.java e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
>
> Diff: https://reviews.apache.org/r/52373/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Purshotam Shah
>
>