You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@falcon.apache.org by sa...@apache.org on 2015/03/16 06:33:07 UTC
[2/2] falcon git commit: FALCON-1010 Add documentation in
falcon-regression's README.md,
making it easy for contributors to add tests to falcon-regression. contributed
by Karishma g
FALCON-1010 Add documentation in falcon-regression's README.md, making it easy for contributors to add tests to falcon-regression. contributed by Karishma g
Project: http://git-wip-us.apache.org/repos/asf/falcon/repo
Commit: http://git-wip-us.apache.org/repos/asf/falcon/commit/4f17787b
Tree: http://git-wip-us.apache.org/repos/asf/falcon/tree/4f17787b
Diff: http://git-wip-us.apache.org/repos/asf/falcon/diff/4f17787b
Branch: refs/heads/master
Commit: 4f17787b3797158177936330eec76e9e6d6123ad
Parents: 25486e3
Author: samarthg <sa...@apacge.org>
Authored: Mon Mar 16 05:32:40 2015 +0000
Committer: samarthg <sa...@apacge.org>
Committed: Mon Mar 16 05:32:40 2015 +0000
----------------------------------------------------------------------
falcon-regression/CHANGES.txt | 3 +
falcon-regression/README.md | 137 +++++++++++++++++++++++++++++++++++++
2 files changed, 140 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/falcon/blob/4f17787b/falcon-regression/CHANGES.txt
----------------------------------------------------------------------
diff --git a/falcon-regression/CHANGES.txt b/falcon-regression/CHANGES.txt
index 018bb5e..3a8ab9d 100644
--- a/falcon-regression/CHANGES.txt
+++ b/falcon-regression/CHANGES.txt
@@ -64,6 +64,9 @@ Trunk (Unreleased)
IMPROVEMENTS
+ FALCON-1010 Add documentation in falcon-regression's README.md, making it easy for
+ contributors to add tests to falcon-regression(Karishma G via Samarth )
+
FALCON-1011 Fix PrismProcessScheduleTest in falcon regression ( Pragya M via
Samarth G)
http://git-wip-us.apache.org/repos/asf/falcon/blob/4f17787b/falcon-regression/README.md
----------------------------------------------------------------------
diff --git a/falcon-regression/README.md b/falcon-regression/README.md
index 5e1743c..acb7175 100644
--- a/falcon-regression/README.md
+++ b/falcon-regression/README.md
@@ -130,6 +130,142 @@ For testing with kerberos set keytabs properties for different users:
falcon.super2.user.keytab=/home/qa/hadoopqa/keytabs/falcon2.headless.keytab
other.user.keytab=/home/qa/hadoopqa/keytabs/root.headless.keytab
+Adding tests to falcon regression:
+----------------------------------
+If you wish to contribute to falcon regression, it's as easy as it gets.
+All test classes must be added to the directory:
+
+ falcon/falcon-regression/merlin/src/test/java
+
+This directory contains sub directories such as prism, ui, security, etc
+which contain tests specific to these aspects of falcon. Any general test
+can be added directly to the parent directory above. If you wish to write
+a series of tests for a new feature, feel free to create a new sub directory.
+Your test can use the various process/feed/cluster/workflow templates present in:
+
+ falcon/falcon-regression/merlin/src/test/resources
+
+or you can add your own bundle of XMLs in this directory. Please avoid redundancy of any resource.
+
+Each test class can contain multiple related tests. Let us look at a sample test class.
+*Refer to comments in code for aid* :
+
+```java
+ //The License note must be added to each test
+
+ /**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+ package org.apache.falcon.regression;
+
+
+ import org.apache.falcon.regression.core.bundle.Bundle;
+ import org.apache.falcon.regression.core.helpers.ColoHelper;
+ import org.apache.falcon.regression.core.response.ServiceResponse;
+ import org.apache.falcon.regression.core.util.AssertUtil;
+ import org.apache.falcon.regression.core.util.BundleUtil;
+ import org.apache.falcon.regression.testHelper.BaseTestClass;
+ import org.testng.annotations.AfterMethod;
+ import org.testng.annotations.BeforeMethod;
+ import org.testng.annotations.Test;
+
+ @Test(groups = "embedded")
+
+ //Every test class must inherit the BaseTestClass. This class
+ //helps using properties mentioned in Merlin.properties, in the test.
+
+ public class FeedSubmitTest extends BaseTestClass {
+
+ private ColoHelper cluster = servers.get(0);
+ private String feed;
+
+ @BeforeMethod(alwaysRun = true)
+ public void setUp() throws Exception {
+
+ //Several Util classes are available, such as BundleUtil, which for example
+ //has been used here to read the ELBundle present in falcon/falcon-regression/src/test/resources
+
+ bundles[0] = BundleUtil.readELBundle();
+ bundles[0].generateUniqueBundle();
+ bundles[0] = new Bundle(bundles[0], cluster);
+
+ //submit the cluster
+ ServiceResponse response =
+ prism.getClusterHelper().submitEntity(bundles[0].getClusters().get(0));
+ AssertUtil.assertSucceeded(response);
+ feed = bundles[0].getInputFeedFromBundle();
+ }
+
+ @AfterMethod(alwaysRun = true)
+ public void tearDown() {
+ removeBundles();
+ }
+
+ //Java docs must be added for each test function, explaining what the function does
+
+ /**
+ * Submit correctly adjusted feed. Response should reflect success.
+ *
+ * @throws Exception
+ */
+ @Test(groups = {"singleCluster"})
+ public void submitValidFeed() throws Exception {
+ ServiceResponse response = prism.getFeedHelper().submitEntity(feed);
+ AssertUtil.assertSucceeded(response);
+ }
+
+ /**
+ * Submit and remove feed. Try to submit it again. Response should reflect success.
+ *
+ * @throws Exception
+ */
+ @Test(groups = {"singleCluster"})
+ public void submitValidFeedPostDeletion() throws Exception {
+ ServiceResponse response = prism.getFeedHelper().submitEntity(feed);
+ AssertUtil.assertSucceeded(response);
+
+ response = prism.getFeedHelper().delete(feed);
+ AssertUtil.assertSucceeded(response);
+ response = prism.getFeedHelper().submitEntity(feed);
+ AssertUtil.assertSucceeded(response);
+ }
+ }
+```
+
+* This class, as the name suggests was to test the Feed Submition aspect of Falcon.
+It contains multiple test functions, all of which however are various test cases for the same
+feature. This organisation in code must be maintained.
+
+* In order to be able to manipulate feeds, processes and clusters for the various tests,
+objects of classes FeedMerlin, ProcessMerlin, ClusterMerlin can be used. There are already existing
+functions which use these objects, such as setProcessInput, setFeedValidity, setProcessConcurrency,
+setInputFeedPeriodicity etc. in Bundle.java which should serve your purpose well enough.
+
+* To add more on the utils, you can use functions in HadoopUtil to create HDFS dirs, delete them,
+and add data on HDFS, OozieUtil to hit Oozie for checking coordinator/workflow status, TimeUtil to
+get lists of dates and directories to aid in data creation, HCatUtil for Hcatalog related utilities,
+and many others to make writing tests very easy.
+
+* Coding conventions are strictly followed. Use the checkstyle xml present in
+ falcon/checkstyle/src/main/resources/falcon
+
+ in your project to not get checkstyle errors.
+
Testing on Windows
------------------
Some tests switch user to run commands as a different user. Location of binary to switch user is
@@ -148,6 +284,7 @@ info and logs at the end of the test. This can be done by configuring Merlin.pro
log.capture.location = ../
Dumping entities generated by falcon
+
------------------------------------
Add -Dmerlin.dump.staging to the maven command. For example: