You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by mbalassi <gi...@git.apache.org> on 2015/05/23 11:21:36 UTC
[GitHub] flink pull request: [FLINK-1874] [streaming] Connector breakup
GitHub user mbalassi opened a pull request:
https://github.com/apache/flink/pull/719
[FLINK-1874] [streaming] Connector breakup
Connectors separated into maven modules, named e.g. flink-kafka-connector.
Plus an additional commit to remove the flume connector from the docs for the next release as it is mostly commented out currently and I do not see it as a priority to be fixed before the release.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mbalassi/flink connector-breakup
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/719.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #719
----
commit 6d8ec5d6fe99a197d3cc8866c2782142d7063c49
Author: mbalassi <mb...@apache.org>
Date: 2015-05-23T08:50:58Z
[FLINK-1874] [streaming] Connectors separated into individual projects
commit 1baf32422cf16c98c824a3390f97a713cf2604c9
Author: mbalassi <mb...@apache.org>
Date: 2015-05-23T08:52:07Z
[docs] [streaming] Flume connector temporarily removed from docs
As the Flume source is not working currently.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1874] [streaming] Connector breakup
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/719#issuecomment-104872808
Great, thank you for fixing this.
I'll soon review it!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1874] [streaming] Connector breakup
Posted by mbalassi <gi...@git.apache.org>.
Github user mbalassi commented on the pull request:
https://github.com/apache/flink/pull/719#issuecomment-106203949
I added the `parent` suffix to the main pom, because that is how we tend to name modules not containing source code themselves. Then I have realized that the name is becoming too long and partially irrelevant for the reasons Stephan mentioned.
I'd go for `flink-connector-kafka` then and have the modules in streaming for now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1874] [streaming] Connector breakup
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/719
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1874] [streaming] Connector breakup
Posted by mbalassi <gi...@git.apache.org>.
Github user mbalassi commented on the pull request:
https://github.com/apache/flink/pull/719#issuecomment-106299970
Thanks, merging as soon as travis verifies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1874] [streaming] Connector breakup
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/719#issuecomment-106285952
Okay, I agree with you that we can name the modules the way marton suggested it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1874] [streaming] Connector breakup
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/719#issuecomment-105914769
I'm not so sure about the module names.
The parent is called.
`flink-streaming-connectors-parent`
The modules itself are called:
```diff
+ <modules>
+ <module>flink-flume-connector</module>
+ <module>flink-kafka-connector</module>
+ <module>flink-rabbitmq-connector</module>
+ <module>flink-twitter-connector</module>
+ </modules>
```
It would be more logical to call them:
`flink-streaming-connectors-flume`, `flink-streaming-connectors-kafka`.
Including `streaming` into the name would make it clear that these connectors are not usable with the batch system.
For `kafka` we might want to provide a batch source at some point (we do already through the kafka hadoop input format).
I think I'm slightly in favor of calling them `flink-streaming-connectors-flume`.
Any other opinions?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1874] [streaming] Connector breakup
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/719#issuecomment-105926196
Should we so clearly differentiate between the streaming and the batch connector projects?
Why not have a `flink-connector-kafka` that contains both a batch and a streaming source? They will probably share a lot of code anyways.
Long term, when the streaming API supports batch API data sources (as bounded streams), this will come as a natural fit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---