You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/10/28 17:27:58 UTC
[jira] [Commented] (BEAM-852) Validate sources when they are
created
[ https://issues.apache.org/jira/browse/BEAM-852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616018#comment-15616018 ]
ASF GitHub Bot commented on BEAM-852:
-------------------------------------
GitHub user sb2nov opened a pull request:
https://github.com/apache/incubator-beam/pull/1220
[BEAM-852] Add validation to file based sources during create time
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [x] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [x] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
---
Adding validation to the fileSourceIO so that we can inform the user if the files don't exist at the create time. There is a flag to bypass verification incase your local system doesn't have the same access as the runner etc.
PTAL @chamikaramj
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sb2nov/incubator-beam BEAM-852-validate_file_source_at_create
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-beam/pull/1220.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1220
----
commit 5e866687c3ef8f3e67838e868a3deaeb9555e460
Author: Sourabh Bajaj <so...@google.com>
Date: 2016-10-28T17:18:51Z
[BEAM-852] Add validation to file based sources during create time
----
> Validate sources when they are created
> --------------------------------------
>
> Key: BEAM-852
> URL: https://issues.apache.org/jira/browse/BEAM-852
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py
> Reporter: Chamikara Jayalath
> Assignee: Sourabh Bajaj
>
> We currently do not validate some sources at creation time. For example text, Avro. Validating sources early will improve user experience since it will help catch issues early. For example, we can fail before submitting a job to a runner.
> It should also be possible to disable validation to support environments where users do not have access to the input at job submission. Java SDK already follows a similar model.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)