You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "lostluck (via GitHub)" <gi...@apache.org> on 2023/03/29 21:50:20 UTC

[GitHub] [beam] lostluck opened a new pull request, #26035: (#25314) Have boot loaders log to logging service where possible.

lostluck opened a new pull request, #26035:
URL: https://github.com/apache/beam/pull/26035

   Have the boot loaders log to the logging service once we have an endpoint. This allows their severity to be recorded properly for better surfacing in runners, such as for fatal errors.
   
   Sets the default log level to Debug for boot loader logging instead of Info, since boot loader configuration is unlikely to be of interest to most Beam users. Previously fatal logs use.
   
   Also moves the "provision" code to a package path to indicate it's for Beam's container bootloader use, and not for Beam Go SDK users. It was never intended for end users, so this reduction in surface clarifies the intention among Go SDK packages.
   
   Fixes #25314 and #24470 which were both around the incorrect surfacing of these errors as "info", and how they hampered debugging.
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier).
   
   To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Go tests](https://github.com/apache/beam/workflows/Go%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1490757148

   R: @robertwb  for typescript and general experience with the bootloaders commentary.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] codecov[bot] commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1489396994

   ## [Codecov](https://codecov.io/gh/apache/beam/pull/26035?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#26035](https://codecov.io/gh/apache/beam/pull/26035?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a3d8642) into [master](https://codecov.io/gh/apache/beam/commit/5bdc4527a4702a2de3c423be172b92df0e0c13a0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5bdc452) will **increase** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   ```diff
   @@           Coverage Diff           @@
   ##           master   #26035   +/-   ##
   =======================================
     Coverage   71.34%   71.35%           
   =======================================
     Files         779      779           
     Lines      102767   102750   -17     
   =======================================
   - Hits        73319    73316    -3     
   + Misses      27976    27962   -14     
     Partials     1472     1472           
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/26035?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [sdks/go/pkg/beam/core/runtime/harness/datamgr.go](https://codecov.io/gh/apache/beam/pull/26035?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9kYXRhbWdyLmdv) | `56.91% <ø> (-0.17%)` | :arrow_down: |
   | [sdks/go/pkg/beam/core/runtime/harness/harness.go](https://codecov.io/gh/apache/beam/pull/26035?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9oYXJuZXNzLmdv) | `11.26% <ø> (+0.24%)` | :arrow_up: |
   | [sdks/go/pkg/beam/core/runtime/harness/logging.go](https://codecov.io/gh/apache/beam/pull/26035?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvaGFybmVzcy9sb2dnaW5nLmdv) | `26.47% <ø> (+0.50%)` | :arrow_up: |
   
   :mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1492133935

   Run Python_Transforms PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1492133264

   Run Python_Dataframes PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1491064215

   Looks like github API quota was hit, and that killed a few of the jenkins jobs. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] jrmccluskey commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1490447531

   Are we okay to delete the provision package in pkg/beam with this change?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1489376150

   Since this is changing the boot loaders, the test set is rather large.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] github-actions[bot] commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1489452340

   Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`:
   
   R: @pabloem for label python.
   R: @kennknowles for label java.
   R: @jrmccluskey for label go.
   
   Available commands:
   - `stop reviewer notifications` - opt out of the automated review tooling
   - `remind me after tests pass` - tag the comment author after tests pass
   - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
   
   The PR bot will only process comments in the main thread (not review comments).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] kennknowles commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "kennknowles (via GitHub)" <gi...@apache.org>.
kennknowles commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1492590119

   Nice. Thank you!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1492266444

   Failures and pending are the CodeCov (can't help it right now, due to the boot loaders being untested), and the non-blocking typescript tests. Merging!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] github-actions[bot] commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1490758508

   Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1492132988

   Run PythonDocker PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1491064512

   Run GoPortable PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck merged pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck merged PR #26035:
URL: https://github.com/apache/beam/pull/26035


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1490823719

   The go test action failure was https://github.com/dominikh/go-tools/issues/1034 which appears to be fixed in more recent staticheck versions (currently on 2022.1). I'll move us to a more recent version.
   
   This seems to have resolved it too. Actions also now use Go 1.20+


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1491064653

   Run Java PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1490542423

   Hmmm, there are some uses internal to Dataflow. I'll add things back so those can be more gracefully migrated.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1492133613

   Run Python_PVR_Flink PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1490538690

   > Are we okay to delete the provision package in pkg/beam with this change?
   
   I'm of the opinion it shouldn't be there, as it promotes arbitrary use of implementation details, and makes an already difficult to learn system, harder.
   
   I would concede to leave in vestigial functions that call the new location for the functions. 
   
   But it seems unlikely that there are more than [the 0 public users](https://pkg.go.dev/github.com/apache/beam/sdks/v2/go/pkg/beam/provision?tab=importedby) of the package who would benefit from the migration.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1491064918

   Run Java_Examples_Dataflow PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1490784762

   @robertwb has confirmed offline that the typescript tests can be ignored for now (they don't push their own build/dev containers WRT testing PRs.) Ah, the trials of an experimental SDK.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1492132721

   Run Portable_Python PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [beam] lostluck commented on pull request #26035: (#25314) Have boot loaders log to logging service where possible.

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #26035:
URL: https://github.com/apache/beam/pull/26035#issuecomment-1492134224

   Run Spotless PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org