You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by "Vieru, Mihail" <mi...@zalando.de> on 2016/02/16 14:53:41 UTC
Availability for the ElasticSearch 2 streaming connector
Hi,
in reference to this ticket https://issues.apache.org/jira/browse/FLINK-3115
when do you think that an ElasticSearch 2 streaming connector will become
available? Will it make it for the 1.0 release?
That would be great, as we are planning to use that particular version of
ElasticSearch in the very near future.
Best regards,
Mihail
Re: Availability for the ElasticSearch 2 streaming connector
Posted by Zach Cox <zc...@gmail.com>.
Awesome, thanks Suneel. :D
I made the changes to support our use case, which needed flatMap behavior
(index 2 docs, or zero docs, per incoming element) instead of map, and we
also need to make either IndexRequest or UpdateRequest depending on the
element.
-Zach
On Thu, Feb 18, 2016 at 2:06 AM Suneel Marthi <sm...@apache.org> wrote:
> Thanks Zach, I have a few minor changes too locally; I'll push a PR out
> tomorrow that has ur changes too.
>
> On Wed, Feb 17, 2016 at 5:13 PM, Zach Cox <zc...@gmail.com> wrote:
>
>> I recently did exactly what Robert described: I copied the code from this
>> (closed) PR https://github.com/apache/flink/pull/1479, modified it a
>> bit, and just included it in my own project that uses the Elasticsearch 2
>> java api. Seems to work well. Here are the files so you can do the same:
>>
>> https://gist.github.com/zcox/59e486be7aeeca381be0
>>
>> -Zach
>>
>>
>> On Wed, Feb 17, 2016 at 4:06 PM Suneel Marthi <su...@gmail.com>
>> wrote:
>>
>>> Hey I missed this thread, sorry about that.
>>>
>>> I have a basic connector working with ES 2.0 which I can push out. Its
>>> not optimized yet and I don't have the time to look at it, if someone would
>>> like to take it over go ahead I can send a PR.
>>>
>>> On Wed, Feb 17, 2016 at 4:57 PM, Robert Metzger <rm...@apache.org>
>>> wrote:
>>>
>>>> Hi Mihail,
>>>>
>>>> It seems that nobody is actively working on the elasticsearch2
>>>> connector right now. The 1.0.0 release is already feature frozen, only bug
>>>> fixes or (some) pending pull requests go in.
>>>>
>>>> What you can always do is copy the code from our current elasticsearch
>>>> connector, set the dependency to the version you would like to use and
>>>> adopt our code to their API changes. I think it might take not much time to
>>>> get it working.
>>>> (The reason why we usually need more time for stuff like this are
>>>> integration tests and documentation).
>>>>
>>>> Please let me know if that solution doesn't work for you.
>>>>
>>>> Regards,
>>>> Robert
>>>>
>>>>
>>>> On Tue, Feb 16, 2016 at 2:53 PM, Vieru, Mihail <mihail.vieru@zalando.de
>>>> > wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> in reference to this ticket
>>>>> https://issues.apache.org/jira/browse/FLINK-3115 when do you think
>>>>> that an ElasticSearch 2 streaming connector will become available? Will it
>>>>> make it for the 1.0 release?
>>>>>
>>>>> That would be great, as we are planning to use that particular version
>>>>> of ElasticSearch in the very near future.
>>>>>
>>>>> Best regards,
>>>>> Mihail
>>>>>
>>>>
>>>>
>>>
>
Re: Availability for the ElasticSearch 2 streaming connector
Posted by Suneel Marthi <sm...@apache.org>.
Thanks Zach, I have a few minor changes too locally; I'll push a PR out
tomorrow that has ur changes too.
On Wed, Feb 17, 2016 at 5:13 PM, Zach Cox <zc...@gmail.com> wrote:
> I recently did exactly what Robert described: I copied the code from this
> (closed) PR https://github.com/apache/flink/pull/1479, modified it a bit,
> and just included it in my own project that uses the Elasticsearch 2 java
> api. Seems to work well. Here are the files so you can do the same:
>
> https://gist.github.com/zcox/59e486be7aeeca381be0
>
> -Zach
>
>
> On Wed, Feb 17, 2016 at 4:06 PM Suneel Marthi <su...@gmail.com>
> wrote:
>
>> Hey I missed this thread, sorry about that.
>>
>> I have a basic connector working with ES 2.0 which I can push out. Its
>> not optimized yet and I don't have the time to look at it, if someone would
>> like to take it over go ahead I can send a PR.
>>
>> On Wed, Feb 17, 2016 at 4:57 PM, Robert Metzger <rm...@apache.org>
>> wrote:
>>
>>> Hi Mihail,
>>>
>>> It seems that nobody is actively working on the elasticsearch2 connector
>>> right now. The 1.0.0 release is already feature frozen, only bug fixes or
>>> (some) pending pull requests go in.
>>>
>>> What you can always do is copy the code from our current elasticsearch
>>> connector, set the dependency to the version you would like to use and
>>> adopt our code to their API changes. I think it might take not much time to
>>> get it working.
>>> (The reason why we usually need more time for stuff like this are
>>> integration tests and documentation).
>>>
>>> Please let me know if that solution doesn't work for you.
>>>
>>> Regards,
>>> Robert
>>>
>>>
>>> On Tue, Feb 16, 2016 at 2:53 PM, Vieru, Mihail <mi...@zalando.de>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> in reference to this ticket
>>>> https://issues.apache.org/jira/browse/FLINK-3115 when do you think
>>>> that an ElasticSearch 2 streaming connector will become available? Will it
>>>> make it for the 1.0 release?
>>>>
>>>> That would be great, as we are planning to use that particular version
>>>> of ElasticSearch in the very near future.
>>>>
>>>> Best regards,
>>>> Mihail
>>>>
>>>
>>>
>>
Re: Availability for the ElasticSearch 2 streaming connector
Posted by Zach Cox <zc...@gmail.com>.
I recently did exactly what Robert described: I copied the code from this
(closed) PR https://github.com/apache/flink/pull/1479, modified it a bit,
and just included it in my own project that uses the Elasticsearch 2 java
api. Seems to work well. Here are the files so you can do the same:
https://gist.github.com/zcox/59e486be7aeeca381be0
-Zach
On Wed, Feb 17, 2016 at 4:06 PM Suneel Marthi <su...@gmail.com>
wrote:
> Hey I missed this thread, sorry about that.
>
> I have a basic connector working with ES 2.0 which I can push out. Its
> not optimized yet and I don't have the time to look at it, if someone would
> like to take it over go ahead I can send a PR.
>
> On Wed, Feb 17, 2016 at 4:57 PM, Robert Metzger <rm...@apache.org>
> wrote:
>
>> Hi Mihail,
>>
>> It seems that nobody is actively working on the elasticsearch2 connector
>> right now. The 1.0.0 release is already feature frozen, only bug fixes or
>> (some) pending pull requests go in.
>>
>> What you can always do is copy the code from our current elasticsearch
>> connector, set the dependency to the version you would like to use and
>> adopt our code to their API changes. I think it might take not much time to
>> get it working.
>> (The reason why we usually need more time for stuff like this are
>> integration tests and documentation).
>>
>> Please let me know if that solution doesn't work for you.
>>
>> Regards,
>> Robert
>>
>>
>> On Tue, Feb 16, 2016 at 2:53 PM, Vieru, Mihail <mi...@zalando.de>
>> wrote:
>>
>>> Hi,
>>>
>>> in reference to this ticket
>>> https://issues.apache.org/jira/browse/FLINK-3115 when do you think that
>>> an ElasticSearch 2 streaming connector will become available? Will it make
>>> it for the 1.0 release?
>>>
>>> That would be great, as we are planning to use that particular version
>>> of ElasticSearch in the very near future.
>>>
>>> Best regards,
>>> Mihail
>>>
>>
>>
>
Re: Availability for the ElasticSearch 2 streaming connector
Posted by Suneel Marthi <su...@gmail.com>.
Hey I missed this thread, sorry about that.
I have a basic connector working with ES 2.0 which I can push out. Its not
optimized yet and I don't have the time to look at it, if someone would
like to take it over go ahead I can send a PR.
On Wed, Feb 17, 2016 at 4:57 PM, Robert Metzger <rm...@apache.org> wrote:
> Hi Mihail,
>
> It seems that nobody is actively working on the elasticsearch2 connector
> right now. The 1.0.0 release is already feature frozen, only bug fixes or
> (some) pending pull requests go in.
>
> What you can always do is copy the code from our current elasticsearch
> connector, set the dependency to the version you would like to use and
> adopt our code to their API changes. I think it might take not much time to
> get it working.
> (The reason why we usually need more time for stuff like this are
> integration tests and documentation).
>
> Please let me know if that solution doesn't work for you.
>
> Regards,
> Robert
>
>
> On Tue, Feb 16, 2016 at 2:53 PM, Vieru, Mihail <mi...@zalando.de>
> wrote:
>
>> Hi,
>>
>> in reference to this ticket
>> https://issues.apache.org/jira/browse/FLINK-3115 when do you think that
>> an ElasticSearch 2 streaming connector will become available? Will it make
>> it for the 1.0 release?
>>
>> That would be great, as we are planning to use that particular version of
>> ElasticSearch in the very near future.
>>
>> Best regards,
>> Mihail
>>
>
>
Re: Availability for the ElasticSearch 2 streaming connector
Posted by Robert Metzger <rm...@apache.org>.
Hi Mihail,
It seems that nobody is actively working on the elasticsearch2 connector
right now. The 1.0.0 release is already feature frozen, only bug fixes or
(some) pending pull requests go in.
What you can always do is copy the code from our current elasticsearch
connector, set the dependency to the version you would like to use and
adopt our code to their API changes. I think it might take not much time to
get it working.
(The reason why we usually need more time for stuff like this are
integration tests and documentation).
Please let me know if that solution doesn't work for you.
Regards,
Robert
On Tue, Feb 16, 2016 at 2:53 PM, Vieru, Mihail <mi...@zalando.de>
wrote:
> Hi,
>
> in reference to this ticket
> https://issues.apache.org/jira/browse/FLINK-3115 when do you think that
> an ElasticSearch 2 streaming connector will become available? Will it make
> it for the 1.0 release?
>
> That would be great, as we are planning to use that particular version of
> ElasticSearch in the very near future.
>
> Best regards,
> Mihail
>