You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Chamikara Jayalath (JIRA)" <ji...@apache.org> on 2016/12/15 23:54:58 UTC

[jira] [Created] (BEAM-1167) Jobs that contain FileBasedSources with globs that expand into zero files should fail

Chamikara Jayalath created BEAM-1167:
----------------------------------------

             Summary: Jobs that contain FileBasedSources with globs that expand into zero files should fail
                 Key: BEAM-1167
                 URL: https://issues.apache.org/jira/browse/BEAM-1167
             Project: Beam
          Issue Type: Bug
          Components: sdk-java-core, sdk-py
            Reporter: Chamikara Jayalath


Currently we do not fail if a glob that is used to create a FileBasedSource expand into zero files. This can be error prone since a typo when specifying a glob pattern can result in a job that silently pass without producing any output. I believe this should be fixed as follows.

If a glob pattern specified as input to a FileBasedSource expand into zero files.
(1) If validation of FileBasedSource is enabled, fail the job at pipeline construction.
(2) If validation is disabled, fail the job at runtime by raising an exception at initial splitting.

Both Java and Python SDKs have to be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)