You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by pm...@apache.org on 2018/10/16 01:36:05 UTC

samza git commit: Removing regex-input static-input exclusivity check

Repository: samza
Updated Branches:
  refs/heads/master 62857f601 -> 96658ee7e


Removing regex-input static-input exclusivity check

This PR converts a hard-check that fails in case of regex-defined input if there is an overlap
between regex-input and task-input, to a warning.

Why?
Because with Samza 1.0, rewrite is called multiple times to expand system descriptors, input descriptors, etc. This hard-check fails in this case causing deployment to stall. Therefore we convert it to a warning.

NOTE: This now allows users to define stream config for their inputs which may overlap with regex-input. This was earlier explicitly disallowed by the hard-check.

Author: rmatharu@linkedin.com <rm...@linkedin.com>

Reviewers: Prateek Maheshwari <pm...@apache.org>

Closes #731 from rmatharu/regexcheck


Project: http://git-wip-us.apache.org/repos/asf/samza/repo
Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/96658ee7
Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/96658ee7
Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/96658ee7

Branch: refs/heads/master
Commit: 96658ee7e1445340ef1799ca93eb6bd335c33690
Parents: 62857f6
Author: rmatharu@linkedin.com <rm...@linkedin.com>
Authored: Mon Oct 15 18:36:00 2018 -0700
Committer: Prateek Maheshwari <pm...@apache.org>
Committed: Mon Oct 15 18:36:00 2018 -0700

----------------------------------------------------------------------
 .../main/scala/org/apache/samza/config/RegExTopicGenerator.scala  | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/samza/blob/96658ee7/samza-kafka/src/main/scala/org/apache/samza/config/RegExTopicGenerator.scala
----------------------------------------------------------------------
diff --git a/samza-kafka/src/main/scala/org/apache/samza/config/RegExTopicGenerator.scala b/samza-kafka/src/main/scala/org/apache/samza/config/RegExTopicGenerator.scala
index ce4544b..a81ff13 100644
--- a/samza-kafka/src/main/scala/org/apache/samza/config/RegExTopicGenerator.scala
+++ b/samza-kafka/src/main/scala/org/apache/samza/config/RegExTopicGenerator.scala
@@ -71,7 +71,8 @@ class RegExTopicGenerator extends ConfigRewriter with Logging {
       info("Generating new configs for matching stream %s." format m)
 
       if (existingInputStreams.contains(m)) {
-        throw new SamzaException("Regex '%s' matches existing, statically defined input %s." format (regex, m))
+        warn("Regex '%s' matches existing, statically defined input %s. " +
+          "Please ensure regex-defined and statically-defined inputs are exclusive." format (regex, m))
       }
 
       newInputStreams.add(m)