You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/05/05 19:08:13 UTC
[jira] [Commented] (BEAM-258) Execute selected RunnableOnService
tests with Flink runner
[ https://issues.apache.org/jira/browse/BEAM-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272882#comment-15272882 ]
ASF GitHub Bot commented on BEAM-258:
-------------------------------------
GitHub user kennknowles opened a pull request:
https://github.com/apache/incubator-beam/pull/291
[BEAM-258] Configure RunnableOnService tests for Flink runner
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [x] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [x] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [x] If this contribution is large, please file an Apache
[Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
---
This is a sample configuration for now.
There are these kind of failures in the tests right now:
1. Since the batch runner only supports global windows, I've filtered those tests out. I added the `UnsupportedOperationException` to the windowing translator so I could distinguish them.
2. Those tests that have simple & supported pipelines succeed at building the pipeline, but somehow the graph is empty - I have checked and it seemed like translators are not even being invoked. This is beyond my current scope of digging in.
3. Some other tests fail with other misc errors. Perhaps they use state, which results in `NullPointerException`.
4. Pretty much all of the tests require side inputs since that is how `PAssert` works, so they cannot work in streaming mode.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kennknowles/incubator-beam flink-integration
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-beam/pull/291.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #291
----
commit 52198eb7e9c2df627871e7f96404d188dc4a4ee7
Author: Kenneth Knowles <kl...@google.com>
Date: 2016-05-02T21:29:30Z
Add Window.Bound translator to Flink batch
This adds a Window.Bound translator that allows only
GlobalWindows. It is a temporary measure, but one that
brings the Flink batch translator in line with the
Beam model - instead of "ignoring" windows, the GBK
is a perfectly valid GBK for GlobalWindows.
Previously, the SDK's runner test suite would fail
due to the lack of a translator - now some of them
will fail due to windowing support, but others have
a chance.
commit 095f9840dd0d8d78f041e494613375664f7d3eaa
Author: Kenneth Knowles <kl...@google.com>
Date: 2016-05-02T20:11:12Z
Add TestFlinkPipelineRunner to FlinkRunnerRegistrar
This makes the runner available for selection by integration tests.
commit 750a49d286f4d11d6ad63460d8b244a5ebde975e
Author: Kenneth Knowles <kl...@google.com>
Date: 2016-05-02T21:04:20Z
Configure RunnableOnService tests for Flink in batch mode
Today Flink batch supports only global windows. This is a situation we
intend our build to allow, eventually via JUnit category filtering.
For now all the test classes that use non-global windows are excluded
entirely via maven configuration. In the future, it should be on a
per-test-method basis.
----
> Execute selected RunnableOnService tests with Flink runner
> ----------------------------------------------------------
>
> Key: BEAM-258
> URL: https://issues.apache.org/jira/browse/BEAM-258
> Project: Beam
> Issue Type: Test
> Components: runner-flink
> Reporter: Kenneth Knowles
> Assignee: Kenneth Knowles
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)