samza git commit: Disable flaky tests.

[ja...@apache.org]

Repository: samza
Updated Branches:
  refs/heads/master c50dd080b -> df5aa6c4e


Disable flaky tests.

Author: Shanthoosh Venkataraman <sv...@linkedin.com>

Reviewers: Navina Ramesh<na...@apache.org>

Closes #158 from shanthoosh/master


Project: http://git-wip-us.apache.org/repos/asf/samza/repo
Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/df5aa6c4
Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/df5aa6c4
Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/df5aa6c4

Branch: refs/heads/master
Commit: df5aa6c4eeb9b89a40780b19e08bb6d96b97730b
Parents: c50dd08
Author: Shanthoosh Venkataraman <sv...@linkedin.com>
Authored: Wed May 10 09:27:57 2017 -0700
Committer: vjagadish1989 <jv...@linkedin.com>
Committed: Wed May 10 09:27:57 2017 -0700

----------------------------------------------------------------------
 .../versioned/samza-async-user-guide.md         | 25 ++++++++++++++++++++
 .../samza/monitor/TestLocalStoreMonitor.java    |  9 +++++--
 2 files changed, 32 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/samza/blob/df5aa6c4/docs/learn/tutorials/versioned/samza-async-user-guide.md
----------------------------------------------------------------------
diff --git a/docs/learn/tutorials/versioned/samza-async-user-guide.md b/docs/learn/tutorials/versioned/samza-async-user-guide.md
index 30865a8..4396e61 100644
--- a/docs/learn/tutorials/versioned/samza-async-user-guide.md
+++ b/docs/learn/tutorials/versioned/samza-async-user-guide.md
@@ -118,6 +118,31 @@ task.max.concurrency=4
 
 **NOTE:** In case of AsyncStreamTask, processAsync() is still invoked in the order of the message arrivals, but the completion can go out of order. In case of StreamTask with multithreading, process() can run out-of-order since they are dispatched to a thread pool. This option should **NOT** be used when strict ordering of the output is required.
 
+### Non blocking commit
+
+By default in samza, concurrent execution of process and commit is not supported. This was designed with a motive to prevent commit flushing to disk and process updating the local store at the same time.
+
+In some scenarios, process calls will take a lot of time to complete. For instance, having high latency remote service calls in process will affect the execution frequency of commit(since only one can happen at a time). This incurs a additional delay in publishing the processed messages from a samza job to downstream samza jobs.
+
+Non blocking commit supports simultaneous processes and commits and is essential to prevent the starvation of commit from slow process completions.
+
+####Benefits.
+
+A) Reduces delay in publishing the processed messages by enabling the task to commit aggressively(a commit is no longer blocked by ongoing process calls).
+
+B) Increases the throughput of process calls, since process is not blocked by a ongoing commit.
+
+C) Decreases the time between fetching and processing the message.
+
+###NOTE:
+
+This should be used only when all the local store operations from a samza task are idempotent. Otherwise, during commit writes from messages in flight will be flushed to disk. 
+
+{% highlight jproperties %}
+# Turned off by default. Users should explicitly set following config to turn it on.
+task.async.commit = true
+{% endhighlight %}
+
 ### Guaranteed Semantics
 
 In any of the scenarios, Samza guarantees the following semantics:

http://git-wip-us.apache.org/repos/asf/samza/blob/df5aa6c4/samza-rest/src/test/java/org/apache/samza/monitor/TestLocalStoreMonitor.java
----------------------------------------------------------------------
diff --git a/samza-rest/src/test/java/org/apache/samza/monitor/TestLocalStoreMonitor.java b/samza-rest/src/test/java/org/apache/samza/monitor/TestLocalStoreMonitor.java
index 2ad6935..971eaa4 100644
--- a/samza-rest/src/test/java/org/apache/samza/monitor/TestLocalStoreMonitor.java
+++ b/samza-rest/src/test/java/org/apache/samza/monitor/TestLocalStoreMonitor.java
@@ -97,8 +97,8 @@ public class TestLocalStoreMonitor {
     }
   }
 
-  // TODO fix in SAMZA-1243
-  // @Test
+// TODO: Fix in SAMZA-1243
+//  @Test
   public void shouldDeleteLocalTaskStoreWhenItHasNoOffsetFile() throws Exception {
     localStoreMonitor.monitor();
     assertTrue("Task store directory should not exist.", !taskStoreDir.exists());
@@ -148,8 +148,13 @@ public class TestLocalStoreMonitor {
     assertEquals(0, localStoreMonitorMetrics.diskSpaceFreedInBytes.getCount());
   }
 
+<<<<<<< HEAD
   // TODO fix in SAMZA-1243
   // @Test
+=======
+// TODO: Fix in SAMZA-1243
+//  @Test
+>>>>>>> Disable flaky tests.
   public void shouldDeleteTaskStoreWhenTaskPreferredStoreIsNotLocalHost() throws Exception {
     Task task = new Task("notLocalHost", "test-task", "0",
                          new ArrayList<>(), ImmutableList.of("test-store"));