You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by jbd <jb...@jbdenis.net> on 2013/03/27 19:06:13 UTC
Forward mode and blacklist management
Hello everybody,
I'm using squid in forward proxy mode with squidguard for managing
blacklists. Squid use the url_rewrite_program directive to let the
squidguard processes manage the access to the requested URL.
From what I can see, to mimic the same behaviour in ATS, I've got two
options :
- Using a simple helper script to translate the squidguard database into
a remap.conf compatible format
- use the blacklist plugin
Do you have any feedback about any of those solutions ? I'm talking of
thousands of blacklisted URLs and about 500 requests/s, and I've got no
idea if the first solution will not put ATS on its knee ?
The blacklist plugin seems to simple to me : hardcoded number of site in
the source, simple linear scanning to find if the provided URL should be
rejected or not...
If anyone could provide some feedback about their use of ATS as a
forward proxy and how they manage the management of blacklists, it would
be cool =)
Regards,
Jean-Baptiste
Re: Forward mode and blacklist management
Posted by James Peach <jp...@apache.org>.
On Mar 28, 2013, at 4:57 AM, jbd <jb...@jbdenis.net> wrote:
> Hi,
>
>> Well, if you don't change the blacklist often, then just generate a
>> remap.config and reload on the live system with traffic_line -x. The
>> lookup on the "host" in remap.config is a hash, so it should be good
>> performance for many thousands of rules.
>
> Thank you for the feedback. I've just add a look to the adult category of my blacklist, it contains 1 million items, not 1000. :D
>
>> Writing a plugin that uses some sort of database for doing the blacklist
>> would not be difficult.
>
> Yes, shouldn't be too difficult, but I was curious about user experience in managing blacklist in ATS. Does not seem that there is a lot of forward mode users =)
I wonder if you could use the authproxy plugin for this? authproxy is a generic way to submit a HTTP request to a helper service who can decide whether the request should be allowed. Possibly, the blacklist could be in the helper service and authproxy could do the Traffic Server integration ...
J
Re: Forward mode and blacklist management
Posted by jbd <jb...@jbdenis.net>.
Hi,
> Well, if you don't change the blacklist often, then just generate a
> remap.config and reload on the live system with traffic_line -x. The
> lookup on the "host" in remap.config is a hash, so it should be good
> performance for many thousands of rules.
Thank you for the feedback. I've just add a look to the adult category
of my blacklist, it contains 1 million items, not 1000. :D
> Writing a plugin that uses some sort of database for doing the blacklist
> would not be difficult.
Yes, shouldn't be too difficult, but I was curious about user experience
in managing blacklist in ATS. Does not seem that there is a lot of
forward mode users =)
Jean-Baptiste
Re: Forward mode and blacklist management
Posted by jbd <jb...@jbdenis.net>.
Hi,
> Well, if you don't change the blacklist often, then just generate a
> remap.config and reload on the live system with traffic_line -x. The
> lookup on the "host" in remap.config is a hash, so it should be good
> performance for many thousands of rules.
Thank you for the feedback. I've just add a look to the adult category
of my blacklist, it contains 1 million items, not 1000. :D
> Writing a plugin that uses some sort of database for doing the blacklist
> would not be difficult.
Yes, shouldn't be too difficult, but I was curious about user experience
in managing blacklist in ATS. Does not seem that there is a lot of
forward mode users =)
Jean-Baptiste
Re: Forward mode and blacklist management
Posted by Leif Hedstrom <zw...@apache.org>.
On 3/27/13 11:06 AM, jbd wrote:
> Hello everybody,
>
> I'm using squid in forward proxy mode with squidguard for managing
> blacklists. Squid use the url_rewrite_program directive to let the
> squidguard processes manage the access to the requested URL.
>
> From what I can see, to mimic the same behaviour in ATS, I've got two
> options :
>
> - Using a simple helper script to translate the squidguard database
> into a remap.conf compatible format
>
> - use the blacklist plugin
>
> Do you have any feedback about any of those solutions ? I'm talking of
> thousands of blacklisted URLs and about 500 requests/s, and I've got
> no idea if the first solution will not put ATS on its knee ?
Well, if you don't change the blacklist often, then just generate a
remap.config and reload on the live system with traffic_line -x. The
lookup on the "host" in remap.config is a hash, so it should be good
performance for many thousands of rules.
Writing a plugin that uses some sort of database for doing the blacklist
would not be difficult.
-- Leif