You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "Abacn (via GitHub)" <gi...@apache.org> on 2024/11/06 17:16:25 UTC

[PR] Remove antlr from dependency of jaca-core pom [beam]

Abacn opened a new pull request, #33030:
URL: https://github.com/apache/beam/pull/33030

   We already vendored and repackaged antlr runtime into the jar. It was intended to avoid dependency version conflict if user using different antlr versions. However, beam java core still declares it as a dependency.
   
   Before:
   
   java core pom has antlr4-runtime dependency, e.g. in https://repo1.maven.org/maven2/org/apache/beam/beam-sdks-java-core/2.60.0/beam-sdks-java-core-2.60.0.pom
   
   ```
   <dependency>
   <groupId>org.antlr</groupId>
   <artifactId>antlr4-runtime</artifactId>
   <version>4.7</version>
   <scope>compile</scope>
   ```
   
   after:
   
   ran `./gradlew :sdks:java:core:publishToMavenLocal -Ppublishing` and checked that antlr-runtime is no longer a dependency of beam java core in generated pom.
   
   Also checked the jar still repackaged antlr runtime in `org/apache/beam/repackaged/core/org/antlr/v4/runtime`
   
   
   **Please** add a meaningful description for your change here
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
   
   To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Go tests](https://github.com/apache/beam/workflows/Go%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove antlr from dependency from java-core pom [beam]

Posted by "pinfrederic (via GitHub)" <gi...@apache.org>.
pinfrederic commented on PR #33030:
URL: https://github.com/apache/beam/pull/33030#issuecomment-2460458739

   #Yes. I think with current version user can still use a different version, but they need to pin the dependency like https://github.com/apache/beam/issues/32696#issuecomment-2457430036
   
   Can you set the dependency as "compile only" in the gradle file? This would avoid us to do this 
   `exclude group: 'org.antlr', module: '*'`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove antlr from dependency from java-core pom [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #33030:
URL: https://github.com/apache/beam/pull/33030#issuecomment-2460497541

   > #Yes. I think with current version user can still use a different version, but they need to pin the dependency like [#32696 (comment)](https://github.com/apache/beam/issues/32696#issuecomment-2457430036)
   > 
   > Can you set the dependency as "compile only" in the gradle file? This would avoid us to do this `exclude group: 'org.antlr', module: '*'`
   
   After this change antlr will be a "compile only" dependency (more precisely its a custom gradle config, used for generating java code from antlr source), won't leak into the pom. 
   
   >> do we still have a risk of having conflicts with beam at runtime?
   >
   > Yes that is what I understood.
   
   I mean there could be other Beam's dependency, not Beam itself, having transient antlr dependency, that could be conflict with user dependency. This is based on the observation of #33016 #14787 - bumping antlr version then some tests breaks elsewhere. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove antlr from dependency from java-core pom [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #33030:
URL: https://github.com/apache/beam/pull/33030#issuecomment-2460619166

   many tests failing in ':runners:direct-java:needsRunnerTests' target due to
   
   ```
   java.lang.NoClassDefFoundError: org/antlr/v4/runtime/CharStream
   	at org.apache.beam.sdk.schemas.FieldAccessDescriptor.withFieldNames(FieldAccessDescriptor.java:187)
   	at org.apache.beam.sdk.schemas.FieldAccessDescriptor.withFieldNames(FieldAccessDescriptor.java:174)
   ```
   
   checked that the same test, if running under ':runners:direct-java:test', will pass.
   
   The reason it failed is because the tests running on unshaded compiled class (haven't redirect antlr yet)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove antlr from dependency of jaca-core pom [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #33030:
URL: https://github.com/apache/beam/pull/33030#issuecomment-2460404833

   Thanks for the feedback. 
   
   > I guess this time we can use a different version in our pipelines if we need it... can you confirm? 
   
   Yes. I think with current version user can still use a different version, but they need to pin the dependency like https://github.com/apache/beam/issues/32696#issuecomment-2457430036
   
   The problem on the current HEAD is that beam core "leaks" the antlr-runtime dependency while it was supposed to be shaded
   
   > do we still have a risk of having conflicts with beam at runtime?
   
   Yes that is what I understood.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove antlr from dependency from java-core pom [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn merged PR #33030:
URL: https://github.com/apache/beam/pull/33030


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove antlr from dependency from java-core pom [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #33030:
URL: https://github.com/apache/beam/pull/33030#issuecomment-2460436047

   Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment `assign set of reviewers`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove antlr from dependency from java-core pom [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #33030:
URL: https://github.com/apache/beam/pull/33030#issuecomment-2461223284

   Java PreCommit failing tests are `FanOutStreamingEngineWorkerHarnessTest`, also flaky on master branch: https://github.com/apache/beam/pull/32775#issuecomment-2461221539 not related to this PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove antlr from dependency from java-core pom [beam]

Posted by "pinfrederic (via GitHub)" <gi...@apache.org>.
pinfrederic commented on PR #33030:
URL: https://github.com/apache/beam/pull/33030#issuecomment-2460521022

   Alright, understood. Thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove antlr from dependency from java-core pom [beam]

Posted by "pinfrederic (via GitHub)" <gi...@apache.org>.
pinfrederic commented on PR #33030:
URL: https://github.com/apache/beam/pull/33030#issuecomment-2463086053

   Question: Do you build a docker image from the master branch after each PR merged? This would help me to try this change before the v2.61 to be released


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove antlr from dependency from java-core pom [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #33030:
URL: https://github.com/apache/beam/pull/33030#issuecomment-2460798857

   R: @shunping 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove antlr from dependency of jaca-core pom [beam]

Posted by "pinfrederic (via GitHub)" <gi...@apache.org>.
pinfrederic commented on PR #33030:
URL: https://github.com/apache/beam/pull/33030#issuecomment-2460374429

   I think that solution is even better than the previous one. 
   I guess this time we can use a different version in our pipelines if we need it... can you confirm? 
   With this solution do we still have a risk of having conflicts with beam at runtime?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove antlr from dependency of jaca-core pom [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #33030:
URL: https://github.com/apache/beam/pull/33030#issuecomment-2460433613

   R: @shunping 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org