You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@heron.apache.org by GitBox <gi...@apache.org> on 2020/01/11 06:15:27 UTC
[GitHub] [incubator-heron] nicknezis opened a new pull request #3433: Use
new Bazel plugin to dynamically find transitive Maven dependencies
nicknezis opened a new pull request #3433: Use new Bazel plugin to dynamically find transitive Maven dependencies
URL: https://github.com/apache/incubator-heron/pull/3433
Update the Bazel build scripts to use this new capability: https://github.com/bazelbuild/rules_jvm_external
The goal would be to better manage the project's dependencies and not have to manually update the various transitive dependencies. This tool will use a dependency's pom.xml to dynamically generate the Bazel targets for each of the dependencies.
The approach I took was to remove all `maven_jar` dependencies and then to add them back one by one. Hopefully I have the right subset of dependencies.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-heron] jin commented on issue #3433: WIP: Use new Bazel
plugin to dynamically find transitive Maven dependencies
Posted by GitBox <gi...@apache.org>.
jin commented on issue #3433: WIP: Use new Bazel plugin to dynamically find transitive Maven dependencies
URL: https://github.com/apache/incubator-heron/pull/3433#issuecomment-573919661
Looks like you're missing a `@maven` dep somewhere:
```
(05:44:45) WARNING: /home/travis/build/apache/incubator-heron/heron/healthmgr/tests/java/BUILD:52:12: in srcs attribute of java_library rule //heron/healthmgr/tests/java:healthmgr-tests: please do not import '//heron/healthmgr/src/java:org/apache/heron/healthmgr/HealthManager.java' directly. You should either move the file to this package or depend on an appropriate rule there
(05:44:53) Analyzing: 709 targets (392 packages loaded, 5535 targets configured)
(05:44:58) ERROR: /home/travis/build/apache/incubator-heron/contrib/spouts/kafka/src/java/BUILD:24:1: no such target '@maven//:org_apache_kafka_kafka_clients': target 'org_apache_kafka_kafka_clients' not declared in package '' defined by /home/travis/.cache/bazel/_bazel_travis/be6dac4936703c7eedcb4f5cf38cdd65/external/maven/BUILD and referenced by '//contrib/spouts/kafka/src/java:heron-kafka-spout-java'
(05:44:58) INFO: Call stack for the definition of repository 'com_google_guava_guava_jdk5_17_0' which is a http_file (rule definition at /home/travis/.cache/bazel/_bazel_travis/be6dac4936703c7eedcb4f5cf38cdd65/external/bazel_tools/tools/build_defs/repo/http.bzl:340:13):
- /home/travis/.cache/bazel/_bazel_travis/be6dac4936703c7eedcb4f5cf38cdd65/external/maven/defs.bzl:165:5
- /home/travis/build/apache/incubator-heron/WORKSPACE:136:1
(05:44:58) ERROR: /home/travis/build/apache/incubator-heron/contrib/spouts/kafka/test/java/BUILD:11:1: no such target '@maven//:org_apache_kafka_kafka_clients': target 'org_apache_kafka_kafka_clients' not declared in package '' defined by /home/travis/.cache/bazel/_bazel_travis/be6dac4936703c7eedcb4f5c
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-heron] nicknezis commented on issue #3433: WIP: Use new
Bazel plugin to dynamically find transitive Maven dependencies
Posted by GitBox <gi...@apache.org>.
nicknezis commented on issue #3433: WIP: Use new Bazel plugin to dynamically find transitive Maven dependencies
URL: https://github.com/apache/incubator-heron/pull/3433#issuecomment-573884308
@jin Oh wow. This would have saved me a lot of find and replace effort. I'll try it out to see if I get different results.
I suspect that I'm through the Maven related issues, but am now running into issues related to the fact that I updated Bazel from 26.0 to 29.1. Had to add `autoreconf -f -i` to some of the C++ library builds. Now having an issue with Python `bdist_wheel` command not working.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-heron] nicknezis commented on issue #3433: WIP: Use new
Bazel plugin to dynamically find transitive Maven dependencies
Posted by GitBox <gi...@apache.org>.
nicknezis commented on issue #3433: WIP: Use new Bazel plugin to dynamically find transitive Maven dependencies
URL: https://github.com/apache/incubator-heron/pull/3433#issuecomment-574018974
I ran the migration steps on a fresh branch. It said it was successful, but then hangs. This is the result. I tried 0.26 and 0.29 of buildozer. Also tried hardcoding the maven_jar versions instead of being variables in case that was messing things up. Either way I got the same result which I pushed here: https://github.com/apache/incubator-heron/compare/master...nicknezis:nicknezis/buildozer-refactor
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-heron] jin commented on issue #3433: WIP: Use new Bazel
plugin to dynamically find transitive Maven dependencies
Posted by GitBox <gi...@apache.org>.
jin commented on issue #3433: WIP: Use new Bazel plugin to dynamically find transitive Maven dependencies
URL: https://github.com/apache/incubator-heron/pull/3433#issuecomment-573843637
Curious: did you give https://github.com/bazelbuild/rules_jvm_external/tree/master/migration a shot?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-heron] nwangtw commented on issue #3433: WIP: Use new
Bazel plugin to dynamically find transitive Maven dependencies
Posted by GitBox <gi...@apache.org>.
nwangtw commented on issue #3433: WIP: Use new Bazel plugin to dynamically find transitive Maven dependencies
URL: https://github.com/apache/incubator-heron/pull/3433#issuecomment-576176359
I am wondering if we should upgrade to Bazel 2 directly if you need to upgrade it anyways?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-heron] nicknezis closed pull request #3433: WIP: Use new
Bazel plugin to dynamically find transitive Maven dependencies
Posted by GitBox <gi...@apache.org>.
nicknezis closed pull request #3433: WIP: Use new Bazel plugin to dynamically find transitive Maven dependencies
URL: https://github.com/apache/incubator-heron/pull/3433
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-heron] nicknezis commented on issue #3433: WIP: Use new
Bazel plugin to dynamically find transitive Maven dependencies
Posted by GitBox <gi...@apache.org>.
nicknezis commented on issue #3433: WIP: Use new Bazel plugin to dynamically find transitive Maven dependencies
URL: https://github.com/apache/incubator-heron/pull/3433#issuecomment-577701437
Closing this PR in favor of #3438
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-heron] nicknezis commented on issue #3433: WIP: Use new
Bazel plugin to dynamically find transitive Maven dependencies
Posted by GitBox <gi...@apache.org>.
nicknezis commented on issue #3433: WIP: Use new Bazel plugin to dynamically find transitive Maven dependencies
URL: https://github.com/apache/incubator-heron/pull/3433#issuecomment-576920404
@nwangtw I agree. `maven_jar` is no longer valid in Bazel 2.0, so I think this work will be needed eventually. It might be worth combining this issue with #3364 under a single effort.
I've made some progress on the various pieces that need to be upgraded, but I'm running into issues with the Python dependencies. At this point I've been trying different combinations of 29.1, 1.X and 2.X. I'll try to update this branch with my latest. Perhaps someone with Python experience could help me get over the hump.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services