You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Ralph Goers <ra...@dslextreme.com> on 2022/01/31 07:50:13 UTC
Elasticsearch sink
https://issues.apache.org/jira/browse/FLUME-3103 mentions issues with the ElasticsearchSink.
This component uses version 0.90.0 which was released in April 2013. It is so old that it doesn’t
appear on the EOL schedule at https://www.elastic.co/support/eol. I think it is safe to say this
version went EOL at least 6 years ago.
I’d upgrade it but I am fearful of a few things:
1. Can anyone possibly be using this component as is?
2. Elastic changed the license in version 7. That creates problems using the “normal”
elasticsearch project.
Elastic has created a brand new Java client - https://github.com/elastic/elasticsearch-java -
but essentially implementing this would be like a brand new sink anyway since nothing is
compatible with 0.90.0 anymore.
Given how old this is would it make more sense just to remove it from the release? I do not
want to spend weeks upgrading that and at this point I’m the only one committing.
With so many version jumps my expectation is that the existing code is not going to upgrade
very well. So before I spend any time on this I’d like some feedback.
Thoughts?
Ralph
Re: Elasticsearch sink
Posted by Ralph Goers <ra...@dslextreme.com>.
Thanks, 6.1.0 is better than 0.90.0. At least there is a chance it might be
more easily upgradeable from there.
I wouldn’t take some other project’s code without their permission.
This does beg the question though as to whether all these sources and sinks
wouldn’t be better off in separate source repos. Personally, I would prefer that
the base Flume release only consist of the components that virtually every
deployment will use and then have a number of separate repos that are
independently updated and released.This would definitely make releasing
Flume much easier.
That might be a good goal for a 2.0 release.
Ralph
> On Jan 31, 2022, at 9:59 AM, Tristan Stevens <tr...@apache.org> wrote:
>
> Ralph,
> Replying on this thread.
>
> There's also https://issues.apache.org/jira/browse/FLUME-3021 and/or
> https://github.com/apache/flume/pull/198 which was based on ES 6.1.0.
>
> Also I see https://github.com/cognitree/flume-elasticsearch-sink based
> on 6.5.4 which is ASF 2.0 licensed so we *could* fork it into ours...???
>
> Would certainly make sense to update it, although I guess it competes with
> Logstash, but actually for users who send to other places as well as ES,
> then this could be a good thing.
>
> Tristan
>
> On Mon, 31 Jan 2022 at 07:50, Ralph Goers <ra...@dslextreme.com>
> wrote:
>
>> https://issues.apache.org/jira/browse/FLUME-3103 mentions issues with the
>> ElasticsearchSink.
>> This component uses version 0.90.0 which was released in April 2013. It is
>> so old that it doesn’t
>> appear on the EOL schedule at https://www.elastic.co/support/eol. I think
>> it is safe to say this
>> version went EOL at least 6 years ago.
>>
>> I’d upgrade it but I am fearful of a few things:
>> 1. Can anyone possibly be using this component as is?
>> 2. Elastic changed the license in version 7. That creates problems using
>> the “normal”
>> elasticsearch project.
>>
>> Elastic has created a brand new Java client -
>> https://github.com/elastic/elasticsearch-java -
>> but essentially implementing this would be like a brand new sink anyway
>> since nothing is
>> compatible with 0.90.0 anymore.
>>
>> Given how old this is would it make more sense just to remove it from the
>> release? I do not
>> want to spend weeks upgrading that and at this point I’m the only one
>> committing.
>>
>> With so many version jumps my expectation is that the existing code is not
>> going to upgrade
>> very well. So before I spend any time on this I’d like some feedback.
>>
>> Thoughts?
>>
>> Ralph
Re: Elasticsearch sink
Posted by Tristan Stevens <tr...@apache.org>.
Ralph,
Replying on this thread.
There's also https://issues.apache.org/jira/browse/FLUME-3021 and/or
https://github.com/apache/flume/pull/198 which was based on ES 6.1.0.
Also I see https://github.com/cognitree/flume-elasticsearch-sink based
on 6.5.4 which is ASF 2.0 licensed so we *could* fork it into ours...???
Would certainly make sense to update it, although I guess it competes with
Logstash, but actually for users who send to other places as well as ES,
then this could be a good thing.
Tristan
On Mon, 31 Jan 2022 at 07:50, Ralph Goers <ra...@dslextreme.com>
wrote:
> https://issues.apache.org/jira/browse/FLUME-3103 mentions issues with the
> ElasticsearchSink.
> This component uses version 0.90.0 which was released in April 2013. It is
> so old that it doesn’t
> appear on the EOL schedule at https://www.elastic.co/support/eol. I think
> it is safe to say this
> version went EOL at least 6 years ago.
>
> I’d upgrade it but I am fearful of a few things:
> 1. Can anyone possibly be using this component as is?
> 2. Elastic changed the license in version 7. That creates problems using
> the “normal”
> elasticsearch project.
>
> Elastic has created a brand new Java client -
> https://github.com/elastic/elasticsearch-java -
> but essentially implementing this would be like a brand new sink anyway
> since nothing is
> compatible with 0.90.0 anymore.
>
> Given how old this is would it make more sense just to remove it from the
> release? I do not
> want to spend weeks upgrading that and at this point I’m the only one
> committing.
>
> With so many version jumps my expectation is that the existing code is not
> going to upgrade
> very well. So before I spend any time on this I’d like some feedback.
>
> Thoughts?
>
> Ralph