You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by ppastuszkaocado <gi...@git.apache.org> on 2016/07/19 11:24:13 UTC
[GitHub] incubator-beam pull request #687: [BEAM-461] Create custom Kinesis source
GitHub user ppastuszkaocado opened a pull request:
https://github.com/apache/incubator-beam/pull/687
[BEAM-461] Create custom Kinesis source
This adds custom Kinesis source as new beam-sdks-java-io-kinesis module.
Build on Travis has passed.
This has been tested with unit tests (included in the pull request), but also using DirectRunner (I've confirmed, that source is able to successfully read data produced by Kinesis Producer Library). I've also tried to run this on real dataflow, but it doesn't seem to work (the pipeline starts, but it doesn't read anything at all) - I'm going to investigate the issue, but I'd like you to do the review in the meantime.
Speaking about tests... I've got bunch of integration / end-to-end level tests (using DirectRunner and DataflowRunner), which are using real kinesis to read data from. I'm not sure where should I put them and how would they be executed in apache CI environment?
BTW. Is this "large" enough, so that I should sign ICLA?
@dhalperi, @jbonofre - can you please review this?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ocadotechnology/incubator-beam beam_461_kinesis_io
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-beam/pull/687.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #687
----
commit 76b302e1d011e3c1acc9bd2e4e348520f1d7e1d2
Author: Pastuszka Przemys\u0142aw <pa...@gmail.com>
Date: 2016-07-18T18:03:37Z
Moving to apache beam
commit fd6195d32936febe1acf20ea4f8b4a7c4f93f053
Author: Przemys\u0142aw Pastuszka <p....@ocado.com>
Date: 2016-07-19T08:04:09Z
Fix build
commit fb70c6699c95a69a448b3d8d26da0ba381a9e188
Author: Przemys\u0142aw Pastuszka <p....@ocado.com>
Date: 2016-07-19T08:56:02Z
Get rid of dependency on Java 8
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-beam pull request #687: [BEAM-461] Create custom Kinesis source
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-beam/pull/687
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---