You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by mxm <gi...@git.apache.org> on 2016/01/20 17:06:10 UTC
[GitHub] flink pull request: [FLINK-2933] Flink scala libraries exposed wit...
GitHub user mxm opened a pull request:
https://github.com/apache/flink/pull/1529
[FLINK-2933] Flink scala libraries exposed with maven should carry scala version
This pull request adds Scala suffixes to all Maven modules which dependent on a Scala version. The default Scala version is 2.10. It also includes a small script to list Scala-dependent modules. The current situation looks like this:
```
The following modules DON'T have a dependency on Scala:
flink-parent
flink-annotations
flink-batch-connectors
flink-contrib-parent
flink-core
flink-hcatalog
flink-jdbc
flink-libraries
flink-quickstart
flink-quickstart-java
flink-quickstart-scala
flink-shaded-curator
flink-shaded-curator-recipes
flink-shaded-curator-test
flink-shaded-hadoop
flink-shaded-hadoop2
flink-shaded-include-yarn-tests
flink-streaming-connectors
The following modules have a dependency on Scala:
flink-avro
flink-clients
flink-connector-elasticsearch
flink-connector-filesystem
flink-connector-flume
flink-connector-kafka
flink-connector-nifi
flink-connector-rabbitmq
flink-connector-twitter
flink-connector-wikiedits
flink-dist
flink-examples
flink-examples-batch
flink-examples-streaming
flink-fs-tests
flink-gelly
flink-gelly-scala
flink-hadoop-compatibility
flink-hbase
flink-java
flink-java8
flink-ml
flink-operator-stats
flink-optimizer
flink-python
flink-runtime
flink-runtime-web
flink-scala
flink-scala-shell
flink-storm
flink-storm-examples
flink-streaming-contrib
flink-streaming-java
flink-streaming-scala
flink-table
flink-test-utils
flink-tests
flink-tweet-inputformat
flink-yarn
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mxm/flink FLINK-2940
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1529.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1529
----
commit bac800a3c23f675e983fb744bbe747e30c5ca7b0
Author: Maximilian Michels <mx...@apache.org>
Date: 2016-01-20T09:39:40Z
add Scala suffixes to Scala dependent modules
commit 6a958abfaf221f75706ec630cd6bea3097f03bfc
Author: Maximilian Michels <mx...@apache.org>
Date: 2016-01-20T14:22:55Z
remove scala suffix from scala-free modules
commit 452d76d2c4dd8028add9957941601849d8433082
Author: Maximilian Michels <mx...@apache.org>
Date: 2016-01-20T15:43:19Z
[tools] adapt change-scala-version script
commit 0eb69085ba1c95ff150d47b435f0072642345c8c
Author: Maximilian Michels <mx...@apache.org>
Date: 2016-01-20T15:50:18Z
[tools] add script to list Scala dependent modules
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2933] Flink scala libraries exposed wit...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1529#issuecomment-175143232
Rebased and squashed commits.
For the documentation, I will update all unsuffixed module references with a Scala suffix variable. For example, flink-streaming-java will be `flink-streaming-java{{ scala_suffix }}` and show up as `flink-streaming-java_2.10` by default.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2933] Flink scala libraries exposed wit...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1529#issuecomment-174176742
@fhueske `flink-storm` depends on `flink-runtime` only transitively through `flink-streaming-java`. However, I know from working with the code that it also depends on `flink-clients` (we would have to add flink-clients to the pom once we removed flink-clients and flink-runtime from flink-streaming-java). That would again make flink-storm Scala-dependent.
@aljoscha Yes, that would be odd although as far as I know there are not that many `flink-java8` users. The module depends on `flink-streaming-java`. So I guess it will stay Scala-dependent as long as flink-streaming-java is.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2933] Flink scala libraries exposed wit...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the pull request:
https://github.com/apache/flink/pull/1529#issuecomment-174176173
Yes, I know about `flink-java`, that's why I mentioned that it would seem strange to users that `flink-java8` has a scala version suffix.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2933] Flink scala libraries exposed wit...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1529#issuecomment-174956562
Yep, that is still to do alongside with the mailing list announcement.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2933] Flink scala libraries exposed wit...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the pull request:
https://github.com/apache/flink/pull/1529#issuecomment-174171530
Can it not be prevented that flink-java8 and flink-storm and flink-gelly have the scala dependency? I think that would be very strange for java programmers. flink-java does not seem to have the dependency.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2933] Flink scala libraries exposed wit...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/1529#issuecomment-174176062
Does `flink-storm` directly depend on `flink-runtime` or only transitively through `flink-streaming-java`?
Removing Scala from `flink-java8` and `flink-gelly` would be nice but I agree with @mxm that `flink-streaming-java` is the most critical artifact.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2933] Flink scala libraries exposed wit...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/1529
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2933] Flink scala libraries exposed wit...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/1529#issuecomment-174948935
I think we should also update the documentation along with the change!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2933] Flink scala libraries exposed wit...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/1529#issuecomment-173300875
Thanks for the PR! This fixes an issue that has been reported several times by users.
IIRC, the outcome of the discussion on the dev ML was to remove the Scala dependency from `flink-java` before adding the suffix.
So we have to resolve FLINK-2972 first (one way or the other). Otherwise, we would update the `flink-java` SNAPSHOT artifacts twice and cause quite a bit trouble for our bleeding-edge users.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2933] Flink scala libraries exposed wit...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1529#issuecomment-173533784
I agree it would be nice to fix the Scala dependency of `flink-java` before merging. Let's try to find a solution.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2933] Flink scala libraries exposed wit...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1529#issuecomment-173621826
I've rebased and would like to merge soon. I'll post an announcement on the user/dev mailinglist that some module names will change.
```
Checking Scala-free modules:
OK flink
OK flink-annotations
OK flink-batch-connectors
OK flink-contrib-parent
OK flink-core
OK flink-hcatalog
OK flink-java
OK flink-jdbc
OK flink-libraries
OK flink-quickstart
OK flink-quickstart-java
OK flink-quickstart-scala
OK flink-shaded-curator
OK flink-shaded-curator-recipes
OK flink-shaded-curator-test
OK flink-shaded-hadoop
OK flink-shaded-hadoop2
OK flink-shaded-include-yarn-tests
OK flink-streaming-connectors
Checking Scala-dependent modules:
OK flink-avro
OK flink-clients
OK flink-connector-elasticsearch
OK flink-connector-filesystem
OK flink-connector-flume
OK flink-connector-kafka-0.8
OK flink-connector-kafka-0.9
OK flink-connector-kafka-base
OK flink-connector-nifi
OK flink-connector-rabbitmq
OK flink-connector-twitter
OK flink-connector-wikiedits
OK flink-dist
OK flink-examples
OK flink-examples-batch
OK flink-examples-streaming
OK flink-fs-tests
OK flink-gelly
OK flink-gelly-scala
OK flink-hadoop-compatibility
OK flink-hbase
OK flink-java8
OK flink-ml
OK flink-operator-stats
OK flink-optimizer
OK flink-python
OK flink-runtime
OK flink-runtime-web
OK flink-scala
OK flink-scala-shell
OK flink-storm
OK flink-storm-examples
OK flink-streaming-contrib
OK flink-streaming-java
OK flink-streaming-scala
OK flink-table
OK flink-test-utils
OK flink-tests
OK flink-tweet-inputformat
OK flink-yarn
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2933] Flink scala libraries exposed wit...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1529#issuecomment-174172295
`flink-java` does not have a dependency on Scala anymore because it has been removed in the course of this pull request (as referenced in this pull request, see #1535). The latest list (above your comment) does not state flink-java has a dependency on Scala.
It can't be prevented that `flink-storm` has a dependency on Scala because it depends on flink-runtime. As for `flink-gelly`, I still have to look into it why it depends on Scala. `flink-java8` still has to be cleaned but so far hasn't been top priority. More problematic seems to be `flink-streaming-java`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---