You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Buntu Dev <bu...@gmail.com> on 2015/01/23 21:46:12 UTC
Flume to handle spam detection and rate limiting
Hi -- I'm ingesting data to HDFS via Flume and wanted to know if there any
built-in features to handle spam detection and rate limiting to avoid any
possible flooding of data. Please let me know.
Thanks!
Re: Flume to handle spam detection and rate limiting
Posted by Mohit Durgapal <du...@gmail.com>.
Hi,
I don't think that there is any spam detection features that are in-built
in flume. But don't you think spam-filtering and check against other
attacks should happen a layer before flume, maybe add a layer of LBs
between the web-server(or any other source generating data stream) and
flume?
Did you consider using a load balancer like HAProxy?
Have a look at
http://blog.haproxy.com/2012/02/27/use-a-load-balancer-as-a-first-row-of-defense-against-ddos/
.
Hope that helps.
Regards
Mohit
On Sat, Jan 24, 2015 at 2:16 AM, Buntu Dev <bu...@gmail.com> wrote:
> Hi -- I'm ingesting data to HDFS via Flume and wanted to know if there any
> built-in features to handle spam detection and rate limiting to avoid any
> possible flooding of data. Please let me know.
>
> Thanks!
>