You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Fabio Moreira <fa...@neemu.com> on 2017/10/03 02:39:42 UTC
Re: Official support for http sink?
Hi,
I see the PR was merged at beginning of this year. Many thanks!
I am already using it and is working perfectly.
My API that receives data as http sink requires basic authentication. Since
I could not find any authentication method I've implemented a very simple
basic authentication. I wonder if I messed something.
In case not, I'll be more than glad to try to help improve authentication
methods for http-sink.
Best regards,
2016-11-09 22:25 GMT-02:00 Bessenyei Balázs Donát <be...@apache.org>:
> Hi All,
>
> Ben Wheeler has opened a Pull Request to help Flume have an integrated
> HTTP Sink.
>
> Now, I'd like to ask everyone who has the time to help out with
> reviewing the changes as they have a potentially high impact.
>
> Please, visit https://github.com/apache/flume/pull/84 and let us know
> your comments and thoughts.
>
>
> Thank you,
>
> Donat
>
>
> 2016-10-20 23:19 GMT+02:00 Shang Wu (CS) <Sh...@symantec.com>:
> > Hi Donat and Ben,
> >
> > Thanks a lot for the consideration and I would love to provide helps if
> needed, either something small like submitting a pom.xml or something large
> like shaping the code to PR-ready stage.
> >
> > Thanks again.
> >
> > BR,
> > Shang Wu, Senior Architect, BlueCoat, a Symantec Company
> >
> > On 10/20/16, 6:45 AM, "Balazs Donat Bessenyei" <be...@cloudera.com>
> wrote:
> >
> > I have contacted Ben Wheeler, the main author of
> > https://github.com/hmrc/flume-http-sink .
> >
> > He said that they are open to contributing the feature to Apache
> Flume.
> >
> > I will do my best to assist with the migration.
> >
> >
> > Thank you,
> >
> > Donat
> >
> >
> > On Tue, Oct 18, 2016 at 5:59 PM, Shang Wu (CS) <
> Shang_Wu@symantec.com> wrote:
> > > Hi Donat,
> > >
> > > Thanks a lot for the consideration.
> > >
> > > BR,
> > > Shang Wu, Senior Architect, BlueCoat, a Symantec Company
> > >
> > > On 10/18/16, 1:56 AM, "Balazs Donat Bessenyei" <
> bessbd@cloudera.com> wrote:
> > >
> > > Hi Shang Wu,
> > >
> > > Thank you for your e-mail.
> > > Both implementations seem to be good. (Also, their licences
> fit the
> > > Apache Flume codebase.)
> > > https://github.com/hmrc/flume-http-sink is more recent.
> > >
> > > I'll ping hmrc soon to see if they are willing to create a pull
> > > request. I think it would be a great improvement to Flume.
> > >
> > > If anyone has different ideas or objections, please let me
> know.
> > >
> > >
> > > Thank you,
> > >
> > > Donat
> > >
> > > On Mon, Oct 17, 2016 at 10:14 PM, Shang Wu (CS) <
> Shang_Wu@symantec.com> wrote:
> > > > Hi,
> > > >
> > > >
> > > >
> > > > We are Symantec Cloud Security group and Apache Flume looks
> very promising
> > > > for our backend components, especially its wide supports for
> those popular
> > > > data infrastructures via sources and sinks.
> > > >
> > > > Having said that, we are looking for a very generic approach
> for restful
> > > > API-based data storage systems, such as http sink, now.
> > > >
> > > >
> > > >
> > > > We’ve found two http sink implementations in github, as
> follows, and would
> > > > like to know any plan to include either of it, into the next
> release of
> > > > Apache Flume?
> > > >
> > > >
> > > >
> > > > https://github.com/josealvarezmuguerza/flume-http-sink
> > > >
> > > > https://github.com/hmrc/flume-http-sink
> > > >
> > > >
> > > >
> > > > Thanks adv.
> > > >
> > > > BR,
> > > >
> > > > Shang Wu, Senior Architect, BlueCoat, a Symantec Company.
> > >
> > >
> >
> >
>
--
[image: Linx | Neemu + Chaordic]
FÁBIO MOREIRA
platform
+55 92 3347.0070 fabioalmeid
Av.Via Láctea, 1374 - Aleixo
Manaus, AM - Brasil - 69060-085
Re: Official support for http sink?
Posted by Ben Wheeler <be...@digital.hmrc.gov.uk>.
Hi Fabio,
That seems ok to me. A pluggable java.net.Authenticator implementation
might be a good way, so that it supports more than just basic auth. Though
it might be good to offer some optional keys in the configuration for
username and password, and pass them in (if the implementor want to use
them, they can).
Kind Regards,
Ben
On 3 October 2017 at 03:39, Fabio Moreira <fa...@neemu.com> wrote:
> Hi,
>
> I see the PR was merged at beginning of this year. Many thanks!
>
> I am already using it and is working perfectly.
>
> My API that receives data as http sink requires basic authentication.
> Since I could not find any authentication method I've implemented a very
> simple basic authentication. I wonder if I messed something.
>
> In case not, I'll be more than glad to try to help improve authentication
> methods for http-sink.
>
> Best regards,
>
>
> 2016-11-09 22:25 GMT-02:00 Bessenyei Balázs Donát <be...@apache.org>:
>
>> Hi All,
>>
>> Ben Wheeler has opened a Pull Request to help Flume have an integrated
>> HTTP Sink.
>>
>> Now, I'd like to ask everyone who has the time to help out with
>> reviewing the changes as they have a potentially high impact.
>>
>> Please, visit https://github.com/apache/flume/pull/84 and let us know
>> your comments and thoughts.
>>
>>
>> Thank you,
>>
>> Donat
>>
>>
>> 2016-10-20 23:19 GMT+02:00 Shang Wu (CS) <Sh...@symantec.com>:
>> > Hi Donat and Ben,
>> >
>> > Thanks a lot for the consideration and I would love to provide helps if
>> needed, either something small like submitting a pom.xml or something large
>> like shaping the code to PR-ready stage.
>> >
>> > Thanks again.
>> >
>> > BR,
>> > Shang Wu, Senior Architect, BlueCoat, a Symantec Company
>> >
>> > On 10/20/16, 6:45 AM, "Balazs Donat Bessenyei" <be...@cloudera.com>
>> wrote:
>> >
>> > I have contacted Ben Wheeler, the main author of
>> > https://github.com/hmrc/flume-http-sink .
>> >
>> > He said that they are open to contributing the feature to Apache
>> Flume.
>> >
>> > I will do my best to assist with the migration.
>> >
>> >
>> > Thank you,
>> >
>> > Donat
>> >
>> >
>> > On Tue, Oct 18, 2016 at 5:59 PM, Shang Wu (CS) <
>> Shang_Wu@symantec.com> wrote:
>> > > Hi Donat,
>> > >
>> > > Thanks a lot for the consideration.
>> > >
>> > > BR,
>> > > Shang Wu, Senior Architect, BlueCoat, a Symantec Company
>> > >
>> > > On 10/18/16, 1:56 AM, "Balazs Donat Bessenyei" <
>> bessbd@cloudera.com> wrote:
>> > >
>> > > Hi Shang Wu,
>> > >
>> > > Thank you for your e-mail.
>> > > Both implementations seem to be good. (Also, their licences
>> fit the
>> > > Apache Flume codebase.)
>> > > https://github.com/hmrc/flume-http-sink is more recent.
>> > >
>> > > I'll ping hmrc soon to see if they are willing to create a
>> pull
>> > > request. I think it would be a great improvement to Flume.
>> > >
>> > > If anyone has different ideas or objections, please let me
>> know.
>> > >
>> > >
>> > > Thank you,
>> > >
>> > > Donat
>> > >
>> > > On Mon, Oct 17, 2016 at 10:14 PM, Shang Wu (CS) <
>> Shang_Wu@symantec.com> wrote:
>> > > > Hi,
>> > > >
>> > > >
>> > > >
>> > > > We are Symantec Cloud Security group and Apache Flume looks
>> very promising
>> > > > for our backend components, especially its wide supports
>> for those popular
>> > > > data infrastructures via sources and sinks.
>> > > >
>> > > > Having said that, we are looking for a very generic
>> approach for restful
>> > > > API-based data storage systems, such as http sink, now.
>> > > >
>> > > >
>> > > >
>> > > > We’ve found two http sink implementations in github, as
>> follows, and would
>> > > > like to know any plan to include either of it, into the
>> next release of
>> > > > Apache Flume?
>> > > >
>> > > >
>> > > >
>> > > > https://github.com/josealvarezmuguerza/flume-http-sink
>> > > >
>> > > > https://github.com/hmrc/flume-http-sink
>> > > >
>> > > >
>> > > >
>> > > > Thanks adv.
>> > > >
>> > > > BR,
>> > > >
>> > > > Shang Wu, Senior Architect, BlueCoat, a Symantec Company.
>> > >
>> > >
>> >
>> >
>>
>
>
>
> --
> [image: Linx | Neemu + Chaordic]
>
> FÁBIO MOREIRA
>
> platform
>
> +55 92 3347.0070 fabioalmeid
> Av.Via Láctea, 1374 - Aleixo
> <https://maps.google.com/?q=Av.Via+L%C3%A1ctea,+1374+-+Aleixo%C2%A0+Manaus,+AM+-+Brasil+-+69060-085&entry=gmail&source=g>
>
> Manaus, AM - Brasil - 69060-085
> <https://maps.google.com/?q=Av.Via+L%C3%A1ctea,+1374+-+Aleixo%C2%A0+Manaus,+AM+-+Brasil+-+69060-085&entry=gmail&source=g>
>
>
Re: Official support for http sink?
Posted by Bessenyei Balázs Donát <be...@apache.org>.
Hi Fabio,
It is always very nice to receive feedback from users. Thank you for that!
I think adding extra functionality like authentication would be great. I
suggest that you open a pull request if you agree.
If you need assistance with anything, please let us know.
Thank you,
Donat
2017-10-03 4:39 GMT+02:00 Fabio Moreira <fa...@neemu.com>:
> Hi,
>
> I see the PR was merged at beginning of this year. Many thanks!
>
> I am already using it and is working perfectly.
>
> My API that receives data as http sink requires basic authentication.
> Since I could not find any authentication method I've implemented a very
> simple basic authentication. I wonder if I messed something.
>
> In case not, I'll be more than glad to try to help improve authentication
> methods for http-sink.
>
> Best regards,
>
>
> 2016-11-09 22:25 GMT-02:00 Bessenyei Balázs Donát <be...@apache.org>:
>
>> Hi All,
>>
>> Ben Wheeler has opened a Pull Request to help Flume have an integrated
>> HTTP Sink.
>>
>> Now, I'd like to ask everyone who has the time to help out with
>> reviewing the changes as they have a potentially high impact.
>>
>> Please, visit https://github.com/apache/flume/pull/84 and let us know
>> your comments and thoughts.
>>
>>
>> Thank you,
>>
>> Donat
>>
>>
>> 2016-10-20 23:19 GMT+02:00 Shang Wu (CS) <Sh...@symantec.com>:
>> > Hi Donat and Ben,
>> >
>> > Thanks a lot for the consideration and I would love to provide helps if
>> needed, either something small like submitting a pom.xml or something large
>> like shaping the code to PR-ready stage.
>> >
>> > Thanks again.
>> >
>> > BR,
>> > Shang Wu, Senior Architect, BlueCoat, a Symantec Company
>> >
>> > On 10/20/16, 6:45 AM, "Balazs Donat Bessenyei" <be...@cloudera.com>
>> wrote:
>> >
>> > I have contacted Ben Wheeler, the main author of
>> > https://github.com/hmrc/flume-http-sink .
>> >
>> > He said that they are open to contributing the feature to Apache
>> Flume.
>> >
>> > I will do my best to assist with the migration.
>> >
>> >
>> > Thank you,
>> >
>> > Donat
>> >
>> >
>> > On Tue, Oct 18, 2016 at 5:59 PM, Shang Wu (CS) <
>> Shang_Wu@symantec.com> wrote:
>> > > Hi Donat,
>> > >
>> > > Thanks a lot for the consideration.
>> > >
>> > > BR,
>> > > Shang Wu, Senior Architect, BlueCoat, a Symantec Company
>> > >
>> > > On 10/18/16, 1:56 AM, "Balazs Donat Bessenyei" <
>> bessbd@cloudera.com> wrote:
>> > >
>> > > Hi Shang Wu,
>> > >
>> > > Thank you for your e-mail.
>> > > Both implementations seem to be good. (Also, their licences
>> fit the
>> > > Apache Flume codebase.)
>> > > https://github.com/hmrc/flume-http-sink is more recent.
>> > >
>> > > I'll ping hmrc soon to see if they are willing to create a
>> pull
>> > > request. I think it would be a great improvement to Flume.
>> > >
>> > > If anyone has different ideas or objections, please let me
>> know.
>> > >
>> > >
>> > > Thank you,
>> > >
>> > > Donat
>> > >
>> > > On Mon, Oct 17, 2016 at 10:14 PM, Shang Wu (CS) <
>> Shang_Wu@symantec.com> wrote:
>> > > > Hi,
>> > > >
>> > > >
>> > > >
>> > > > We are Symantec Cloud Security group and Apache Flume looks
>> very promising
>> > > > for our backend components, especially its wide supports
>> for those popular
>> > > > data infrastructures via sources and sinks.
>> > > >
>> > > > Having said that, we are looking for a very generic
>> approach for restful
>> > > > API-based data storage systems, such as http sink, now.
>> > > >
>> > > >
>> > > >
>> > > > We’ve found two http sink implementations in github, as
>> follows, and would
>> > > > like to know any plan to include either of it, into the
>> next release of
>> > > > Apache Flume?
>> > > >
>> > > >
>> > > >
>> > > > https://github.com/josealvarezmuguerza/flume-http-sink
>> > > >
>> > > > https://github.com/hmrc/flume-http-sink
>> > > >
>> > > >
>> > > >
>> > > > Thanks adv.
>> > > >
>> > > > BR,
>> > > >
>> > > > Shang Wu, Senior Architect, BlueCoat, a Symantec Company.
>> > >
>> > >
>> >
>> >
>>
>
>
>
> --
> [image: Linx | Neemu + Chaordic]
>
> FÁBIO MOREIRA
>
> platform
>
> +55 92 3347.0070 fabioalmeid
> Av.Via Láctea, 1374 - Aleixo
> <https://maps.google.com/?q=Av.Via+L%C3%A1ctea,+1374+-+Aleixo%C2%A0+Manaus,+AM+-+Brasil+-+69060-085&entry=gmail&source=g>
>
> Manaus, AM - Brasil - 69060-085
> <https://maps.google.com/?q=Av.Via+L%C3%A1ctea,+1374+-+Aleixo%C2%A0+Manaus,+AM+-+Brasil+-+69060-085&entry=gmail&source=g>
>
>
Re: Official support for http sink?
Posted by Bessenyei Balázs Donát <be...@apache.org>.
Hi Fabio,
It is always very nice to receive feedback from users. Thank you for that!
I think adding extra functionality like authentication would be great. I
suggest that you open a pull request if you agree.
If you need assistance with anything, please let us know.
Thank you,
Donat
2017-10-03 4:39 GMT+02:00 Fabio Moreira <fa...@neemu.com>:
> Hi,
>
> I see the PR was merged at beginning of this year. Many thanks!
>
> I am already using it and is working perfectly.
>
> My API that receives data as http sink requires basic authentication.
> Since I could not find any authentication method I've implemented a very
> simple basic authentication. I wonder if I messed something.
>
> In case not, I'll be more than glad to try to help improve authentication
> methods for http-sink.
>
> Best regards,
>
>
> 2016-11-09 22:25 GMT-02:00 Bessenyei Balázs Donát <be...@apache.org>:
>
>> Hi All,
>>
>> Ben Wheeler has opened a Pull Request to help Flume have an integrated
>> HTTP Sink.
>>
>> Now, I'd like to ask everyone who has the time to help out with
>> reviewing the changes as they have a potentially high impact.
>>
>> Please, visit https://github.com/apache/flume/pull/84 and let us know
>> your comments and thoughts.
>>
>>
>> Thank you,
>>
>> Donat
>>
>>
>> 2016-10-20 23:19 GMT+02:00 Shang Wu (CS) <Sh...@symantec.com>:
>> > Hi Donat and Ben,
>> >
>> > Thanks a lot for the consideration and I would love to provide helps if
>> needed, either something small like submitting a pom.xml or something large
>> like shaping the code to PR-ready stage.
>> >
>> > Thanks again.
>> >
>> > BR,
>> > Shang Wu, Senior Architect, BlueCoat, a Symantec Company
>> >
>> > On 10/20/16, 6:45 AM, "Balazs Donat Bessenyei" <be...@cloudera.com>
>> wrote:
>> >
>> > I have contacted Ben Wheeler, the main author of
>> > https://github.com/hmrc/flume-http-sink .
>> >
>> > He said that they are open to contributing the feature to Apache
>> Flume.
>> >
>> > I will do my best to assist with the migration.
>> >
>> >
>> > Thank you,
>> >
>> > Donat
>> >
>> >
>> > On Tue, Oct 18, 2016 at 5:59 PM, Shang Wu (CS) <
>> Shang_Wu@symantec.com> wrote:
>> > > Hi Donat,
>> > >
>> > > Thanks a lot for the consideration.
>> > >
>> > > BR,
>> > > Shang Wu, Senior Architect, BlueCoat, a Symantec Company
>> > >
>> > > On 10/18/16, 1:56 AM, "Balazs Donat Bessenyei" <
>> bessbd@cloudera.com> wrote:
>> > >
>> > > Hi Shang Wu,
>> > >
>> > > Thank you for your e-mail.
>> > > Both implementations seem to be good. (Also, their licences
>> fit the
>> > > Apache Flume codebase.)
>> > > https://github.com/hmrc/flume-http-sink is more recent.
>> > >
>> > > I'll ping hmrc soon to see if they are willing to create a
>> pull
>> > > request. I think it would be a great improvement to Flume.
>> > >
>> > > If anyone has different ideas or objections, please let me
>> know.
>> > >
>> > >
>> > > Thank you,
>> > >
>> > > Donat
>> > >
>> > > On Mon, Oct 17, 2016 at 10:14 PM, Shang Wu (CS) <
>> Shang_Wu@symantec.com> wrote:
>> > > > Hi,
>> > > >
>> > > >
>> > > >
>> > > > We are Symantec Cloud Security group and Apache Flume looks
>> very promising
>> > > > for our backend components, especially its wide supports
>> for those popular
>> > > > data infrastructures via sources and sinks.
>> > > >
>> > > > Having said that, we are looking for a very generic
>> approach for restful
>> > > > API-based data storage systems, such as http sink, now.
>> > > >
>> > > >
>> > > >
>> > > > We’ve found two http sink implementations in github, as
>> follows, and would
>> > > > like to know any plan to include either of it, into the
>> next release of
>> > > > Apache Flume?
>> > > >
>> > > >
>> > > >
>> > > > https://github.com/josealvarezmuguerza/flume-http-sink
>> > > >
>> > > > https://github.com/hmrc/flume-http-sink
>> > > >
>> > > >
>> > > >
>> > > > Thanks adv.
>> > > >
>> > > > BR,
>> > > >
>> > > > Shang Wu, Senior Architect, BlueCoat, a Symantec Company.
>> > >
>> > >
>> >
>> >
>>
>
>
>
> --
> [image: Linx | Neemu + Chaordic]
>
> FÁBIO MOREIRA
>
> platform
>
> +55 92 3347.0070 fabioalmeid
> Av.Via Láctea, 1374 - Aleixo
> <https://maps.google.com/?q=Av.Via+L%C3%A1ctea,+1374+-+Aleixo%C2%A0+Manaus,+AM+-+Brasil+-+69060-085&entry=gmail&source=g>
>
> Manaus, AM - Brasil - 69060-085
> <https://maps.google.com/?q=Av.Via+L%C3%A1ctea,+1374+-+Aleixo%C2%A0+Manaus,+AM+-+Brasil+-+69060-085&entry=gmail&source=g>
>
>