You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Rahul Raj <ra...@gmail.com> on 2017/12/01 06:22:36 UTC
ElasticSearch Connector for version 6.x and scala 2.11
Hi All,
Is there a Flink Elastic search connector for version 6.0 and scala 2.11? I
couldn't find it listed here
https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/elasticsearch.html
.
Regards,
Rahul Raj
Re: ElasticSearch Connector for version 6.x and scala 2.11
Posted by Nico Kruber <ni...@data-artisans.com>.
Actually, Flink's netty dependency (4.0.27) is shaded away into the
"org.apache.flink.shaded.netty4.io.netty" package now (since version
1.4) and should thus not clash anymore.
However, other netty versions may come into play from the job itself or
from the integration of Hadoop's classpath (if available).
Nico
On 01/12/17 14:42, Jens Oberender wrote:
> Hi
>
> A workmate of mine tried to migrate the existing flink connector to
> ElasticSearch 6 but we had problems with netty dependencies that clashed
> (Flink uses 4.0.27 and ES is on 4.1).
> You can change the flink-connector-elasticsearch5 connector to ES 5.6.4,
> but then you have to do some adaptions to get it working, as they
> changed the API within a major version!
> But with that version you can write to ES 6.
>
> He put his changes on Github:
> https://github.com/cognitix/flink/tree/connector-elasticsearch6
> It's called flink-connector-elasticsearch6, there.
>
> We want to write a new and clean version for ElasticSearch 6, but
> currently don't have the time.
>
> Best regards,
> Jens Oberender
>
>
> Am 01.12.2017 um 09:52 schrieb Fabian Hueske:
>> Hi Rahul,
>>
>> Flink does not provide a connector for ElasticSearch 6 yet.
>> There is this JIRA issue to track the development progress [1].
>>
>> Best, Fabian
>>
>> [1] https://issues.apache.org/jira/browse/FLINK-8101
>>
>> 2017-12-01 7:22 GMT+01:00 Rahul Raj <rahulrajmsrit@gmail.com
>> <ma...@gmail.com>>:
>>
>> Hi All,
>>
>> Is there a Flink Elastic search connector for version 6.0 and scala
>> 2.11? I couldn't find it listed
>> here https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/elasticsearch.html
>> <https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/elasticsearch.html>
>> .
>>
>> Regards,
>> Rahul Raj
>>
>>
>
Re: ElasticSearch Connector for version 6.x and scala 2.11
Posted by Jens Oberender <je...@cognitix.de>.
Hi
A workmate of mine tried to migrate the existing flink connector to
ElasticSearch 6 but we had problems with netty dependencies that clashed
(Flink uses 4.0.27 and ES is on 4.1).
You can change the flink-connector-elasticsearch5 connector to ES 5.6.4,
but then you have to do some adaptions to get it working, as they
changed the API within a major version!
But with that version you can write to ES 6.
He put his changes on Github:
https://github.com/cognitix/flink/tree/connector-elasticsearch6
It's called flink-connector-elasticsearch6, there.
We want to write a new and clean version for ElasticSearch 6, but
currently don't have the time.
Best regards,
Jens Oberender
Am 01.12.2017 um 09:52 schrieb Fabian Hueske:
> Hi Rahul,
>
> Flink does not provide a connector for ElasticSearch 6 yet.
> There is this JIRA issue to track the development progress [1].
>
> Best, Fabian
>
> [1] https://issues.apache.org/jira/browse/FLINK-8101
>
> 2017-12-01 7:22 GMT+01:00 Rahul Raj <rahulrajmsrit@gmail.com
> <ma...@gmail.com>>:
>
> Hi All,
>
> Is there a Flink Elastic search connector for version 6.0 and scala
> 2.11? I couldn't find it listed
> here https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/elasticsearch.html
> <https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/elasticsearch.html>
> .
>
> Regards,
> Rahul Raj
>
>
--
Jens Oberender
Lead Java Developer
cognitix GmbH
Neumarkt 29-33
DE-04109 Leipzig
Tel.: +49 341 331 870 00
Fax.: +49 341 331 870 99
E-Mail: jens.oberender@cognitix.de
Internet: www.cognitix.de
Gesellschaft mit beschränkter Haftung
Geschäftsführer: Tilo Dinger
Handelsregister: Amtsgericht Leipzig, HRB 33904
PGP key fingerprint
5DC9 0F2B F2FE 6D74 F95D 1DC3 B926 6BB4 E2C5 C3A5
Re: ElasticSearch Connector for version 6.x and scala 2.11
Posted by Fabian Hueske <fh...@gmail.com>.
Hi Rahul,
Flink does not provide a connector for ElasticSearch 6 yet.
There is this JIRA issue to track the development progress [1].
Best, Fabian
[1] https://issues.apache.org/jira/browse/FLINK-8101
2017-12-01 7:22 GMT+01:00 Rahul Raj <ra...@gmail.com>:
> Hi All,
>
> Is there a Flink Elastic search connector for version 6.0 and scala 2.11?
> I couldn't find it listed here https://ci.apache.org/
> projects/flink/flink-docs-release-1.3/dev/connectors/elasticsearch.html .
>
> Regards,
> Rahul Raj
>