You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by GitBox <gi...@apache.org> on 2020/03/11 19:38:20 UTC
[GitHub] [beam] KevinGG opened a new pull request #11100: [BEAM-7923]
General Update
KevinGG opened a new pull request #11100: [BEAM-7923] General Update
URL: https://github.com/apache/beam/pull/11100
1. Updated pipeline_fragment to not prune newly added TestStream
transforms.
2. Updated pipeline_instrument to support streaming pipeline and prune
transforms that don't contribute to the desired outputs.
3. Updated ipython version upperbound to <8. We are targeting ipython >
7.3 for the introduction of cell magic `%pip`. Note `%pip` has different
behavior than `!pip` because the previous one guaranteees the
installation in the kernel not arbitrary environment the notebook is
running in.
4. Updated the README for Interactive Beam. The main change is
recommending using Jupyterlab over old Jupyter notebook and setup
instructions such as setting virtual env through Python3 venv module, installing
new labextensions over nbextensions and install extra dependencies.
**Please** add a meaningful description for your change here
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay merged pull request #11100: [BEAM-7923] Streaming
support and pipeline pruning when instrumenting a pipeline with interactivity
Posted by GitBox <gi...@apache.org>.
aaltay merged pull request #11100: [BEAM-7923] Streaming support and pipeline pruning when instrumenting a pipeline with interactivity
URL: https://github.com/apache/beam/pull/11100
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] KevinGG commented on issue #11100: [BEAM-7923] General Update
Posted by GitBox <gi...@apache.org>.
KevinGG commented on issue #11100: [BEAM-7923] General Update
URL: https://github.com/apache/beam/pull/11100#issuecomment-597829767
@rohdesamuel
@aaltay
PTAL, thanks!
This PR depends on some other pending PRs, there is no need to run any test for it temporarily.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] KevinGG commented on issue #11100: [BEAM-7923] Streaming
support and pipeline pruning when instrumenting a pipeline with interactivity
Posted by GitBox <gi...@apache.org>.
KevinGG commented on issue #11100: [BEAM-7923] Streaming support and pipeline pruning when instrumenting a pipeline with interactivity
URL: https://github.com/apache/beam/pull/11100#issuecomment-598449533
Resolved some merge conflicts, will need another rebase once PR https://github.com/apache/beam/pull/11050 is merged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] KevinGG commented on issue #11100: [BEAM-7923] Streaming
support and pipeline pruning when instrumenting a pipeline with interactivity
Posted by GitBox <gi...@apache.org>.
KevinGG commented on issue #11100: [BEAM-7923] Streaming support and pipeline pruning when instrumenting a pipeline with interactivity
URL: https://github.com/apache/beam/pull/11100#issuecomment-597895621
> I think this change is fine. But it was really difficult to review.
> For this PR - Please add a more descriptive PR title.
> For future PRs - Please separate unrelated changes from each other have logical commit sets.
>
> LGTM once the PR title and description is updated to be more specific.
Thanks, Ahmet!
Updated the title to `Streaming support and pipeline pruning when instrumenting a pipeline with interactivity`.
I'll do several rebases when dependency PRs are merged and IM you for tests.
For future PRs, we'll separate changes into more fine grained commit sets, thanks for the review!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11100: [BEAM-7923] Streaming
support and pipeline pruning when instrumenting a pipeline with interactivity
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11100: [BEAM-7923] Streaming support and pipeline pruning when instrumenting a pipeline with interactivity
URL: https://github.com/apache/beam/pull/11100#issuecomment-598490115
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11100: [BEAM-7923] Streaming
support and pipeline pruning when instrumenting a pipeline with interactivity
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11100: [BEAM-7923] Streaming support and pipeline pruning when instrumenting a pipeline with interactivity
URL: https://github.com/apache/beam/pull/11100#issuecomment-598825261
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] aaltay commented on issue #11100: [BEAM-7923] Streaming
support and pipeline pruning when instrumenting a pipeline with interactivity
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11100: [BEAM-7923] Streaming support and pipeline pruning when instrumenting a pipeline with interactivity
URL: https://github.com/apache/beam/pull/11100#issuecomment-598858285
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services