You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2019/11/20 17:27:00 UTC
[jira] [Work logged] (BEAM-8747) Remove Unused non-vendored Guava
compile dependencies
[ https://issues.apache.org/jira/browse/BEAM-8747?focusedWorklogId=346888&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-346888 ]
ASF GitHub Bot logged work on BEAM-8747:
----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Nov/19 17:26
Start Date: 20/Nov/19 17:26
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #10172: [BEAM-8747] Guava dependency cleanup
URL: https://github.com/apache/beam/pull/10172
As per email thread "Completeness of Beam Java Dependency Check Report" ([archive link](https://lists.apache.org/thread.html/c477d120a4c4626cbe675f8b03d84c6fe7938e36c8e2b55c492224cf@%3Cdev.beam.apache.org%3E)), @kennknowles found there are "declared but unused" and "used but undeclared" Guava dependencies in some of modules. This PR addresses the findings.
I used gradle-lint-plugin to confirm Ken's findings. See detail in the ticket https://issues.apache.org/jira/projects/BEAM/issues/BEAM-8747 .
- GCP connector turned out using guava in fully-qualified class name.
- sdks/java/extensions/sql/jdbc had "declared but unused" Guava dependency.
- runners/google-cloud-dataflow-java/worker depends on 2 classes without declaring. These classes should not depend on Guava class. So I updated them to use vendored Guava.
- sdks/java/io/kinesis has "used but undeclared" Guava dependency.
- sdks/java/extensions/sql/zetasql has "used but undeclared" Guava dependency.
- sdks/java/io/elasticsearch-tests/elasticsearch-tests-2 has "declared but unused" Guava dependency.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | ---
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 346888)
Remaining Estimate: 0h
Time Spent: 10m
> Remove Unused non-vendored Guava compile dependencies
> -----------------------------------------------------
>
> Key: BEAM-8747
> URL: https://issues.apache.org/jira/browse/BEAM-8747
> Project: Beam
> Issue Type: Bug
> Components: build-system
> Reporter: Tomo Suzuki
> Assignee: Tomo Suzuki
> Priority: Major
> Attachments: Guava used as fully-qualified class name.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> [~kenn] says:
> BeamModulePlugin just contains lists of versions to ease coordination across Beam modules, but mostly does not create dependencies. Most of Beam's modules only depend on a few things there. For example Guava is not a core dependency, but here is where it is actually depended upon:
> $ find . -name build.gradle | xargs grep library.java.guava
> ./sdks/java/core/build.gradle: shadowTest library.java.guava_testlib
> ./sdks/java/extensions/sql/jdbc/build.gradle: compile library.java.guava
> ./sdks/java/io/google-cloud-platform/build.gradle: compile library.java.guava
> ./sdks/java/io/kinesis/build.gradle: testCompile library.java.guava_testlib
> These results appear to be misleading. Grepping for 'import com.google.common', I see this as the actual state of things:
> - GCP connector does not appear to actually depend on Guava in compile scope
> - The Beam SQL JDBC driver does not appear to actually depend on Guava in compile scope
> - The Dataflow Java worker does depend on Guava at compile scope but has incorrect dependencies (and it probably shouldn't)
> - KinesisIO does depend on Guava at compile scope but has incorrect dependencies (Kinesis libs have Guava on API surface so it is OK here, but should be correctly declared)
> - ZetaSQL translator does depend on Guava at compile scope but has incorrect dependencies (ZetaSQL has it on API surface so it is OK here, but should be correctly declared)
> We used to have an analysis that prevented this class of error.
> Once the errors are fixed, the guava_version is simply a version that we have discovered that seems to work for both Kinesis and ZetaSQL, libraries we do not control. Kinesis producer is built against 18.0. Kinesis client against 26.0-jre. ZetaSQL against 26.0-android.
> (or maybe I messed up in my analysis)
> Kenn
--
This message was sent by Atlassian Jira
(v8.3.4#803005)