You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by 397090770 <gi...@git.apache.org> on 2017/05/11 02:24:20 UTC
[GitHub] flink pull request #3869: [FLINK-6502] Add support ElasticsearchSink for Dat...
GitHub user 397090770 opened a pull request:
https://github.com/apache/flink/pull/3869
[FLINK-6502] Add support ElasticsearchSink for DataSet
Currently, Flink only support writing data in `DataStream` to ElasticSearch through `ElasticsearchSink`, it will be very useful if Flink internal support writing data in `DataSet` to ElasticSearch. See [http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ElasticsearchSink-on-DataSet-td12980.html](http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ElasticsearchSink-on-DataSet-td12980.html)
- [ ] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the JIRA id)
- [ ] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [ ] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis build has passed
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/397090770/flink FLINK-6502
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3869.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3869
----
commit a58ce4bcd86aae000c8308a66ca5c74a1812fb46
Author: yangping.wu <ya...@qunar.com>
Date: 2017-05-11T02:14:03Z
[FLINK-6502] Add support ElasticsearchSink for DataSet
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3869: [FLINK-6502] Add support ElasticsearchSink for DataSet
Posted by tzulitai <gi...@git.apache.org>.
Github user tzulitai commented on the issue:
https://github.com/apache/flink/pull/3869
@397090770 I'm afraid I will need to reject this PR as of its current state.
The main issue is that we can't just move those public user-facing classes. We either have to deprecate them and have a more general packaging path for them so that they can be shared across batch / stream ES connectors, or that we simply duplicate them for batch.
As for the base class logic, this PR needs refactoring so that common logic between the batch and stream is shared, instead of almost the exact same code copied between the two.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3869: [FLINK-6502] Add support ElasticsearchSink for DataSet
Posted by 397090770 <gi...@git.apache.org>.
Github user 397090770 commented on the issue:
https://github.com/apache/flink/pull/3869
@tzulitai Thank you for your review very much. The current structuring of the ES output format as follow:
as you can see, both `ElasticsearchSinkBase` for batch and `ElasticsearchSinkBase` for stream need `ActionRequestFailureHandler`, `RequestIndexer` ect, so I move those class to a different package, and do you have any suggestions? Thank you.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---