Re: Flume 1.7.0 Release Schedule

[Mike Percy <mp...@apache.org>]

There is discussion on the dev list about a Flume 1.7.0 release, but no
committed date yet. We're making some progress towards that goal, though.

Mike

On Fri, Jun 10, 2016 at 10:06 PM, Jason Williams <ja...@gmail.com>
wrote:

> Hey Joe,
>
> I may try 1.7 if you think it's pretty stable in master. Flume as a relay
> actually solves our problem pretty well, in that if the Kafka DC becomes
> unavailable for any reason, the Flume file channel will queue up the
> entries reliably and replay when the link comes back.
>
> -J
>
> Sent via iPhone
>
> On Jun 10, 2016, at 21:25, Joe Lawson <jl...@opensourceconnections.com>
> wrote:
>
> So 1.7 would support Kafka 0.9. The issues are pretty minimal around the
> sink in master. Have you tried it? As for other means flume feels the
> purest in not manipulating the data. If you are shipping logs, Logstash
> (probably a bad word here :) is an option as it supports Kafka 0.9.
>
> You could do something more hacky like piping into the Kafka console
> producer too.
>
> I know (elastic's) beats supports Kafka 0.9 in master but don't think it
> is released either. Remember Kafka 0.9 came out in Oct or Dec so it's
> pretty durn new compared to 0.8 which had over a year to get momentum.
> Let's not even talk about 0.10 ;)
>
> maybe this helps?
>
> -Joe
> On Jun 10, 2016 10:34 PM, "Jason J. W. Williams" <
> jasonjwwilliams@gmail.com> wrote:
>
> Yeah.
>
> Flume (relay)----WAN (Avro SSL)---->Flume (concentrator)--->Kafka.
>
> The remote Flume is talking to a "concentrator" flume in the Kafka
> datacenter, so that the data is transferred via SSL over the WAN, since the
> Kafka sink doesn't support SSL in 1.6.0. Otherwise, we could push from the
> remote Flume directly into Kafka over the WAN.
>
> -J
>
>
> On Fri, Jun 10, 2016 at 8:29 PM, Joe Lawson <
> jlawson@opensourceconnections.com> wrote:
>
>> Do you mean that you are replicating topics over WAN using Flume as a
>> shipping agent? Sorry if I misunderstood the scenario.
>> On Jun 10, 2016 9:58 PM, "Jason J. W. Williams" <
>> jasonjwwilliams@gmail.com> wrote:
>>
>>> Thanks Joe that helps. What do you see folks using instead of Flume?
>>> We're using largely a WAN relay for Kafka for remote sites that don't
>>> warrant a local cluster.
>>>
>>> -J
>>>
>>> On Fri, Jun 10, 2016 at 6:17 PM, Joe Lawson <
>>> jlawson@opensourceconnections.com> wrote:
>>>
>>>> Flume seems to have lost some stream. It's improving but it feels like
>>>> the priorities have become somewhat introspective so I'd say build from
>>>> master is you want the latest.
>>>>
>>>> Don't hold your breath for 1.7. just an opinion. No bearing on reality.
>>>>
>>>> -Joe
>>>> On Jun 10, 2016 12:45 PM, "Jason J. W. Williams" <
>>>> jasonjwwilliams@gmail.com> wrote:
>>>>
>>>>> Is there an estimated release date for Flume 1.7.0? Having TLS support
>>>>> in the Kafka sink would allow us to simplify our topology a lot.=20
>>>>>
>>>>> -J
>>>>
>>>>
>>>
>