You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by smarthi <gi...@git.apache.org> on 2015/12/30 10:36:38 UTC
[GitHub] flink pull request: FLINK-3115: Update ElasticSearch connector to ...
GitHub user smarthi opened a pull request:
https://github.com/apache/flink/pull/1479
FLINK-3115: Update ElasticSearch connector to 2.x
Initial version of ElasticSearch v2.x Connector
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/smarthi/flink ElasticSearch2
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1479.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1479
----
commit 46ef2eda5f08a98c752ce71a8447a62556672e4a
Author: smarthi <sm...@apache.org>
Date: 2015-12-29T09:55:57Z
FLINK-3115: Update ElasticSearch connector to 2.x
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: FLINK-3115: Update ElasticSearch connector to ...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/1479#issuecomment-168211988
Cool addition, thanks a lot!
Looking through the code, there is a lot of work done per element inserted, like creating a hash map, an index request, etc. Is there some way to make the work-per-element more lightweight and do more work in the initialization?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: FLINK-3115:[WIP] Update ElasticSearch connecto...
Posted by smarthi <gi...@git.apache.org>.
Github user smarthi commented on the pull request:
https://github.com/apache/flink/pull/1479#issuecomment-169450577
Closing this PR, will replace with a newer PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: FLINK-3115: Update ElasticSearch connector to ...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/1479#discussion_r48660361
--- Diff: flink-streaming-connectors/flink-connector-elasticsearch2/src/main/java/org/apache/flink/streaming/connectors/elasticsearch2/examples/ElasticsearchExample.java ---
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.connectors.elasticsearch2.examples;
+
+import org.apache.flink.api.common.functions.RuntimeContext;
+import org.apache.flink.streaming.api.datastream.DataStreamSource;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+import org.apache.flink.streaming.api.functions.source.SourceFunction;
+import org.apache.flink.streaming.connectors.elasticsearch2.ElasticsearchSink;
+import org.apache.flink.streaming.connectors.elasticsearch2.IndexRequestBuilder;
+import org.elasticsearch.action.index.IndexRequest;
+import org.elasticsearch.client.Requests;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * This example shows how to use the Elasticsearch Sink. Before running it you must ensure that
+ * you have a cluster names "elasticsearch" running or change the cluster name in the config map.
+ */
+public class ElasticsearchExample {
+
+ public static void main(String[] args) throws Exception {
+
+ StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
+
+ DataStreamSource<String> source = env.addSource(new SourceFunction<String>() {
--- End diff --
You can probably simplify this using `StreamExecutionEnvironment.generateSequence(from, to).map(...)`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: FLINK-3115:[WIP] Update ElasticSearch connecto...
Posted by smarthi <gi...@git.apache.org>.
Github user smarthi closed the pull request at:
https://github.com/apache/flink/pull/1479
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---