You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/03/05 19:24:40 UTC

[GitHub] [beam] xinyuiscool opened a new pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

xinyuiscool opened a new pull request #14155:
URL: https://github.com/apache/beam/pull/14155


   Passing PipelineOptions to DoFn @Setup makes it easier for the users to access their pipeline arguments. It also allows dynamically creating PipelineOptions while constructing the pipeline, and can benefit integration with other config management framework.
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
    - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
   
   Post-Commit Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   
   Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2
   --- | --- | --- | --- | --- | --- | ---
   Go | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | ---
   Java | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon)](https://ci-beam
 .apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [![Build Status](https://ci-beam.a
 pache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/)
   Python | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[![Build Status](https://ci-beam
 .apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/) | ---
   XLang | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/) | ---
   
   Pre-Commit Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   
   --- |Java | Python | Go | Website | Whitespace | Typescript
   --- | --- | --- | --- | --- | --- | ---
   Non-portable | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/)<br>[![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/) <br>[![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/be
 am_PreCommit_Go_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/) | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/)
   Portable | --- | [![Build Status](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/badge/icon)](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | --- | --- | ---
   
   See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
   
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14155:
URL: https://github.com/apache/beam/pull/14155#issuecomment-802434393


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=h1) Report
   > Merging [#14155](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=desc) (a5a00a4) into [master](https://codecov.io/gh/apache/beam/commit/195bbf35a92bf4c4e53e0896c1e35ec5494cbc98?el=desc) (195bbf3) will **increase** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   > :exclamation: Current head a5a00a4 differs from pull request most recent head 7e755a3. Consider uploading reports for the commit 7e755a3 to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14155/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master   #14155   +/-   ##
   =======================================
     Coverage   83.39%   83.39%           
   =======================================
     Files         469      469           
     Lines       58708    58708           
   =======================================
   + Hits        48957    48959    +2     
   + Misses       9751     9749    -2     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...y38/build/srcs/sdks/python/apache\_beam/io/utils.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdXRpbHMucHk=) | | |
   | [.../sdks/python/apache\_beam/transforms/deduplicate.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9kZWR1cGxpY2F0ZS5weQ==) | | |
   | [...on/apache\_beam/io/gcp/bigquery\_io\_read\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X2lvX3JlYWRfcGlwZWxpbmUucHk=) | | |
   | [...n/apache\_beam/examples/complete/game/user\_score.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvZ2FtZS91c2VyX3Njb3JlLnB5) | | |
   | [...ild/srcs/sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | | |
   | [...srcs/sdks/python/apache\_beam/coders/slow\_stream.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL3Nsb3dfc3RyZWFtLnB5) | | |
   | [...ython/apache\_beam/portability/api/endpoints\_pb2.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2VuZHBvaW50c19wYjIucHk=) | | |
   | [...pache\_beam/examples/cookbook/bigquery\_tornadoes.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29va2Jvb2svYmlncXVlcnlfdG9ybmFkb2VzLnB5) | | |
   | [.../runners/interactive/testing/test\_cache\_manager.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS90ZXN0aW5nL3Rlc3RfY2FjaGVfbWFuYWdlci5weQ==) | | |
   | [...ache\_beam/runners/dataflow/test\_dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy90ZXN0X2RhdGFmbG93X3J1bm5lci5weQ==) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=footer). Last update [195bbf3...7e755a3](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kennknowles commented on a change in pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
kennknowles commented on a change in pull request #14155:
URL: https://github.com/apache/beam/pull/14155#discussion_r596241162



##########
File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFn.java
##########
@@ -91,6 +91,16 @@
  */
 public abstract class DoFn<InputT extends @Nullable Object, OutputT extends @Nullable Object>
     implements Serializable, HasDisplayData {
+  /** Information accessible while within the {@link Setup} method. */
+  @SuppressWarnings("ClassCanBeStatic") // Converting class to static is an API change.
+  public abstract class SetupContext {

Review comment:
       Yes, this feature of "context" objects is double edged: it is very hard to know what is actually in the context, or used.
   
   Very concretely: we want to be able to analyze the `DoFn` to figure out the signature. Like if it does not actually ask for `PipelineOptions` we may not have to plumb them. Or for runners that do not keep them at runtime we could reject the pipeline, etc. For state and timers, we could know if it was a stateful DoFn and also validate the tags. It also made side inputs have less boilerplate and be parameters instead of fixed values at DoFn construction, etc.
   
   We still have `ProcessContext` mostly for just legacy backwards compatibility but if people really want to use it they can. But in this case for `SetupContext` it has just one thing in it so there isn't really a need. It just adds indirection where you don't need it.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] xinyuiscool edited a comment on pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
xinyuiscool edited a comment on pull request #14155:
URL: https://github.com/apache/beam/pull/14155#issuecomment-791631528


   @kennknowles : could you or someone else please take a look at this change? Thanks! I tested it by building the core java and runners and run validating runners.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] xinyuiscool commented on a change in pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
xinyuiscool commented on a change in pull request #14155:
URL: https://github.com/apache/beam/pull/14155#discussion_r596186441



##########
File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFn.java
##########
@@ -91,6 +91,16 @@
  */
 public abstract class DoFn<InputT extends @Nullable Object, OutputT extends @Nullable Object>
     implements Serializable, HasDisplayData {
+  /** Information accessible while within the {@link Setup} method. */
+  @SuppressWarnings("ClassCanBeStatic") // Converting class to static is an API change.
+  public abstract class SetupContext {

Review comment:
       You mean we call invokeSetup() directly with PipelineOptions? I thought we always define a context for invoking the methods in DoFn, so we can add more parameters if needed in the future (like StartBundleContext, which only has pipelineOptions inside). If this not needed, I will go ahead and remove this context.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] xinyuiscool commented on pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
xinyuiscool commented on pull request #14155:
URL: https://github.com/apache/beam/pull/14155#issuecomment-807552333


   @kennknowles : I fixed a few things found in the tests and now they are passing again. Could you please take another look? As suggested, I removed the SetupContext and use PipelineOptions in @setup directly.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] xinyuiscool commented on pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
xinyuiscool commented on pull request #14155:
URL: https://github.com/apache/beam/pull/14155#issuecomment-807463107


   Run Portable_Python PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] xinyuiscool commented on a change in pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
xinyuiscool commented on a change in pull request #14155:
URL: https://github.com/apache/beam/pull/14155#discussion_r597338571



##########
File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFn.java
##########
@@ -91,6 +91,16 @@
  */
 public abstract class DoFn<InputT extends @Nullable Object, OutputT extends @Nullable Object>
     implements Serializable, HasDisplayData {
+  /** Information accessible while within the {@link Setup} method. */
+  @SuppressWarnings("ClassCanBeStatic") // Converting class to static is an API change.
+  public abstract class SetupContext {

Review comment:
       Makes sense. The explicit parameters in the args does make static analysis and validation possible. I agree there is no need to have a SetupContext. Updated the patch to remove it.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] commented on pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #14155:
URL: https://github.com/apache/beam/pull/14155#issuecomment-802434393


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=h1) Report
   > Merging [#14155](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=desc) (a5a00a4) into [master](https://codecov.io/gh/apache/beam/commit/195bbf35a92bf4c4e53e0896c1e35ec5494cbc98?el=desc) (195bbf3) will **increase** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   > :exclamation: Current head a5a00a4 differs from pull request most recent head 7206a0a. Consider uploading reports for the commit 7206a0a to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14155/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master   #14155   +/-   ##
   =======================================
     Coverage   83.39%   83.39%           
   =======================================
     Files         469      469           
     Lines       58708    58708           
   =======================================
   + Hits        48957    48959    +2     
   + Misses       9751     9749    -2     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [.../srcs/sdks/python/apache\_beam/transforms/window.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy93aW5kb3cucHk=) | | |
   | [...he\_beam/io/flink/flink\_streaming\_impulse\_source.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmxpbmsvZmxpbmtfc3RyZWFtaW5nX2ltcHVsc2Vfc291cmNlLnB5) | | |
   | [...d/srcs/sdks/python/apache\_beam/transforms/stats.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9zdGF0cy5weQ==) | | |
   | [...srcs/sdks/python/apache\_beam/dataframe/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL19faW5pdF9fLnB5) | | |
   | [...ld/srcs/sdks/python/apache\_beam/io/filesystemio.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZXN5c3RlbWlvLnB5) | | |
   | [...rcs/sdks/python/apache\_beam/typehints/typecheck.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3R5cGVjaGVjay5weQ==) | | |
   | [...on/apache\_beam/runners/worker/statesampler\_slow.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc3RhdGVzYW1wbGVyX3Nsb3cucHk=) | | |
   | [...sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | | |
   | [...on/apache\_beam/io/gcp/internal/clients/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL2NsaWVudHMvX19pbml0X18ucHk=) | | |
   | [...d/srcs/sdks/python/apache\_beam/runners/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9fX2luaXRfXy5weQ==) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=footer). Last update [195bbf3...7206a0a](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14155:
URL: https://github.com/apache/beam/pull/14155#issuecomment-802434393


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=h1) Report
   > Merging [#14155](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=desc) (a5a00a4) into [master](https://codecov.io/gh/apache/beam/commit/195bbf35a92bf4c4e53e0896c1e35ec5494cbc98?el=desc) (195bbf3) will **increase** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   > :exclamation: Current head a5a00a4 differs from pull request most recent head 7206a0a. Consider uploading reports for the commit 7206a0a to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14155/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master   #14155   +/-   ##
   =======================================
     Coverage   83.39%   83.39%           
   =======================================
     Files         469      469           
     Lines       58708    58708           
   =======================================
   + Hits        48957    48959    +2     
   + Misses       9751     9749    -2     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [.../sdks/python/apache\_beam/options/value\_provider.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vb3B0aW9ucy92YWx1ZV9wcm92aWRlci5weQ==) | | |
   | [...d/srcs/sdks/python/apache\_beam/testing/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9fX2luaXRfXy5weQ==) | | |
   | [...on/apache\_beam/io/gcp/bigquery\_io\_read\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X2lvX3JlYWRfcGlwZWxpbmUucHk=) | | |
   | [...eam/runners/interactive/options/capture\_control.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9vcHRpb25zL2NhcHR1cmVfY29udHJvbC5weQ==) | | |
   | [...ild/srcs/sdks/python/apache\_beam/tools/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdG9vbHMvX19pbml0X18ucHk=) | | |
   | [.../build/srcs/sdks/python/apache\_beam/tools/utils.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdG9vbHMvdXRpbHMucHk=) | | |
   | [...srcs/sdks/python/apache\_beam/typehints/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL19faW5pdF9fLnB5) | | |
   | [...am/examples/snippets/transforms/elementwise/map.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9lbGVtZW50d2lzZS9tYXAucHk=) | | |
   | [...sdks/python/apache\_beam/io/aws/clients/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vYXdzL2NsaWVudHMvX19pbml0X18ucHk=) | | |
   | [...y38/build/srcs/sdks/python/apache\_beam/io/kafka.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8va2Fma2EucHk=) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=footer). Last update [195bbf3...7206a0a](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] xinyuiscool commented on pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
xinyuiscool commented on pull request #14155:
URL: https://github.com/apache/beam/pull/14155#issuecomment-791631528


   @kennknowles : could you or someone else please take a look at this change? Thanks!


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] xinyuiscool commented on pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
xinyuiscool commented on pull request #14155:
URL: https://github.com/apache/beam/pull/14155#issuecomment-807463467


   Run Portable_Python PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] xinyuiscool commented on pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
xinyuiscool commented on pull request #14155:
URL: https://github.com/apache/beam/pull/14155#issuecomment-808580677


   Run Java PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kennknowles merged pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
kennknowles merged pull request #14155:
URL: https://github.com/apache/beam/pull/14155


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14155:
URL: https://github.com/apache/beam/pull/14155#issuecomment-802434393


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=h1) Report
   > Merging [#14155](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=desc) (a5a00a4) into [master](https://codecov.io/gh/apache/beam/commit/195bbf35a92bf4c4e53e0896c1e35ec5494cbc98?el=desc) (195bbf3) will **increase** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   > :exclamation: Current head a5a00a4 differs from pull request most recent head ed15589. Consider uploading reports for the commit ed15589 to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14155/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master   #14155   +/-   ##
   =======================================
     Coverage   83.39%   83.39%           
   =======================================
     Files         469      469           
     Lines       58708    58708           
   =======================================
   + Hits        48957    48959    +2     
   + Misses       9751     9749    -2     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...cs/sdks/python/apache\_beam/typehints/decorators.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL2RlY29yYXRvcnMucHk=) | | |
   | [...s/sdks/python/apache\_beam/runners/worker/logger.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvbG9nZ2VyLnB5) | | |
   | [...cs/sdks/python/apache\_beam/transforms/combiners.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9jb21iaW5lcnMucHk=) | | |
   | [...s/python/apache\_beam/testing/synthetic\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9zeW50aGV0aWNfcGlwZWxpbmUucHk=) | | |
   | [...uild/srcs/sdks/python/apache\_beam/io/tfrecordio.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdGZyZWNvcmRpby5weQ==) | | |
   | [.../srcs/sdks/python/apache\_beam/io/flink/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmxpbmsvX19pbml0X18ucHk=) | | |
   | [...s/python/apache\_beam/portability/api/schema\_pb2.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL3NjaGVtYV9wYjIucHk=) | | |
   | [...on/apache\_beam/runners/direct/helper\_transforms.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvaGVscGVyX3RyYW5zZm9ybXMucHk=) | | |
   | [...sdks/python/apache\_beam/io/restriction\_trackers.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vcmVzdHJpY3Rpb25fdHJhY2tlcnMucHk=) | | |
   | [...am/examples/snippets/transforms/elementwise/map.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9lbGVtZW50d2lzZS9tYXAucHk=) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=footer). Last update [195bbf3...ed15589](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] xinyuiscool commented on pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
xinyuiscool commented on pull request #14155:
URL: https://github.com/apache/beam/pull/14155#issuecomment-807464142


   Run Java_Examples_Dataflow PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kennknowles commented on a change in pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
kennknowles commented on a change in pull request #14155:
URL: https://github.com/apache/beam/pull/14155#discussion_r595688093



##########
File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFn.java
##########
@@ -91,6 +91,16 @@
  */
 public abstract class DoFn<InputT extends @Nullable Object, OutputT extends @Nullable Object>
     implements Serializable, HasDisplayData {
+  /** Information accessible while within the {@link Setup} method. */
+  @SuppressWarnings("ClassCanBeStatic") // Converting class to static is an API change.
+  public abstract class SetupContext {

Review comment:
       You shouldn't need this. This is the "old" DoFn way of doing it.

##########
File path: runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/SplittableParDoNaiveBounded.java
##########
@@ -134,9 +134,30 @@ public PCollectionTuple expand(PCollection<KV<byte[], KV<InputT, RestrictionT>>>
     }
 
     @Setup
-    public void setup() {
+    public void setup(SetupContext c) {

Review comment:
       Other `DoFn` methods support a `PipelineOptions` type argument and the `DoFnSignature` detects this.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14155:
URL: https://github.com/apache/beam/pull/14155#issuecomment-802434393


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=h1) Report
   > Merging [#14155](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=desc) (a5a00a4) into [master](https://codecov.io/gh/apache/beam/commit/195bbf35a92bf4c4e53e0896c1e35ec5494cbc98?el=desc) (195bbf3) will **increase** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   > :exclamation: Current head a5a00a4 differs from pull request most recent head 7206a0a. Consider uploading reports for the commit 7206a0a to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14155/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master   #14155   +/-   ##
   =======================================
     Coverage   83.39%   83.39%           
   =======================================
     Files         469      469           
     Lines       58708    58708           
   =======================================
   + Hits        48957    48959    +2     
   + Misses       9751     9749    -2     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...am/examples/snippets/transforms/aggregation/min.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9taW4ucHk=) | | |
   | [...rcs/sdks/python/apache\_beam/runners/job/manager.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9qb2IvbWFuYWdlci5weQ==) | | |
   | [...ers/dataflow/dataflow\_exercise\_metrics\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19leGVyY2lzZV9tZXRyaWNzX3BpcGVsaW5lLnB5) | | |
   | [...sdks/python/apache\_beam/runners/worker/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvX19pbml0X18ucHk=) | | |
   | [.../python/apache\_beam/io/gcp/resource\_identifiers.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL3Jlc291cmNlX2lkZW50aWZpZXJzLnB5) | | |
   | [.../gcp/internal/clients/storage/storage\_v1\_client.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL2NsaWVudHMvc3RvcmFnZS9zdG9yYWdlX3YxX2NsaWVudC5weQ==) | | |
   | [...s/sdks/python/apache\_beam/transforms/sideinputs.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9zaWRlaW5wdXRzLnB5) | | |
   | [...ache\_beam/examples/snippets/transforms/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9fX2luaXRfXy5weQ==) | | |
   | [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | | |
   | [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=footer). Last update [195bbf3...7206a0a](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kennknowles commented on a change in pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
kennknowles commented on a change in pull request #14155:
URL: https://github.com/apache/beam/pull/14155#discussion_r596201351



##########
File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFn.java
##########
@@ -91,6 +91,16 @@
  */
 public abstract class DoFn<InputT extends @Nullable Object, OutputT extends @Nullable Object>
     implements Serializable, HasDisplayData {
+  /** Information accessible while within the {@link Setup} method. */
+  @SuppressWarnings("ClassCanBeStatic") // Converting class to static is an API change.
+  public abstract class SetupContext {

Review comment:
       I mean you would call `invokeSetup` with an `ArgumentProvider` that provides the options (this already exists). Then the job of the invoker (generated bytecode) is to pass the right arguments to the DoFn.
   
   Basically: no more contexts passed to annotation-style `DoFn`s. In fact, we should deprecate the existing ones. Instead, `ArgumentProvider` is basically the same thing, but only passed to `DoFnInvoker` and the invoker passes just the arguments needed to the `DoFn`.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] xinyuiscool commented on a change in pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
xinyuiscool commented on a change in pull request #14155:
URL: https://github.com/apache/beam/pull/14155#discussion_r597338732



##########
File path: runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/SplittableParDoNaiveBounded.java
##########
@@ -134,9 +134,30 @@ public PCollectionTuple expand(PCollection<KV<byte[], KV<InputT, RestrictionT>>>
     }
 
     @Setup
-    public void setup() {
+    public void setup(SetupContext c) {

Review comment:
       Removed SetupConext and use PipelineOptions directly.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] xinyuiscool commented on pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
xinyuiscool commented on pull request #14155:
URL: https://github.com/apache/beam/pull/14155#issuecomment-808369943


   Make sense. I updated the DoFnInvokersTest to verify the setup() will be invoked with pipelineOptions by DoFnInvoker, but it doesn't verify from the runner side. Let me add a ValidatesRunner test there. Thanks for the suggestion.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kennknowles commented on pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
kennknowles commented on pull request #14155:
URL: https://github.com/apache/beam/pull/14155#issuecomment-819823851


   Run ULR Loopback ValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] edited a comment on pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14155:
URL: https://github.com/apache/beam/pull/14155#issuecomment-802434393


   # [Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=h1) Report
   > Merging [#14155](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=desc) (a5a00a4) into [master](https://codecov.io/gh/apache/beam/commit/195bbf35a92bf4c4e53e0896c1e35ec5494cbc98?el=desc) (195bbf3) will **increase** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   > :exclamation: Current head a5a00a4 differs from pull request most recent head 7206a0a. Consider uploading reports for the commit 7206a0a to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/beam/pull/14155/graphs/tree.svg?width=650&height=150&src=pr&token=qcbbAh8Fj1)](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master   #14155   +/-   ##
   =======================================
     Coverage   83.39%   83.39%           
   =======================================
     Files         469      469           
     Lines       58708    58708           
   =======================================
   + Hits        48957    48959    +2     
   + Misses       9751     9749    -2     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...python/apache\_beam/runners/portability/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9fX2luaXRfXy5weQ==) | | |
   | [...on/apache\_beam/runners/direct/sdf\_direct\_runner.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3Qvc2RmX2RpcmVjdF9ydW5uZXIucHk=) | | |
   | [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | | |
   | [...he\_beam/testing/benchmarks/nexmark/nexmark\_perf.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19wZXJmLnB5) | | |
   | [.../runners/interactive/testing/pipeline\_assertion.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS90ZXN0aW5nL3BpcGVsaW5lX2Fzc2VydGlvbi5weQ==) | | |
   | [...ld/srcs/sdks/python/apache\_beam/utils/histogram.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaGlzdG9ncmFtLnB5) | | |
   | [...d/srcs/sdks/python/apache\_beam/transforms/stats.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9zdGF0cy5weQ==) | | |
   | [...ild/srcs/sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | | |
   | [...\_beam/io/gcp/internal/clients/bigquery/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL2NsaWVudHMvYmlncXVlcnkvX19pbml0X18ucHk=) | | |
   | [...ive/messaging/interactive\_environment\_inspector.py](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9tZXNzYWdpbmcvaW50ZXJhY3RpdmVfZW52aXJvbm1lbnRfaW5zcGVjdG9yLnB5) | | |
   | ... and [928 more](https://codecov.io/gh/apache/beam/pull/14155/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=footer). Last update [195bbf3...7206a0a](https://codecov.io/gh/apache/beam/pull/14155?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] xinyuiscool commented on a change in pull request #14155: [BEAM-11914] Add PipelineOptions as DoFn @Setup parameters

Posted by GitBox <gi...@apache.org>.
xinyuiscool commented on a change in pull request #14155:
URL: https://github.com/apache/beam/pull/14155#discussion_r596233734



##########
File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFn.java
##########
@@ -91,6 +91,16 @@
  */
 public abstract class DoFn<InputT extends @Nullable Object, OutputT extends @Nullable Object>
     implements Serializable, HasDisplayData {
+  /** Information accessible while within the {@link Setup} method. */
+  @SuppressWarnings("ClassCanBeStatic") // Converting class to static is an API change.
+  public abstract class SetupContext {

Review comment:
       Right, currently my impl is to pass this SetupContext to ArgumentProvider to get the arguments. I can remove this context and pass in the PipelineOptions directly.
   
   Regarding to removing all contexts: I feel at least ProcessContext is kind of useful from the user point of view. Since @ProcessElement can take in quite a few parameters, it's a bit hard for the users to remember each kind of them (sometimes even I have to look it up to see what's the name of the types). ProcessContext does provide a good entry point to access element, output receiver, and side input. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org