Any existing implemention on limiting calling frequencies by client or by IP

[Jason Wang <ja...@gmail.com>]

Hi all,

I would like to limit the frequencies our APIs can be called. Given that
they will be public APIs.
The limit will most likely be done on IP addresses.

Is there existing mechanism in CXF for this? Otherwise I will create my own
interceptor to do it.

Cheers,
Jason

Re: Any existing implemention on limiting calling frequencies by client or by IP

[Sergey Beryozkin <sb...@gmail.com>]

Hi Andrei
On 21/11/13 12:08, Andrei Shakirin wrote:
> Hi Sergei,
>
>> -----Original Message-----
>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>> Sent: Donnerstag, 21. November 2013 11:35
>> To: users@cxf.apache.org
>> Subject: Re: Any existing implemention on limiting calling frequencies by
>> client or by IP
>>
>> Hi Dan
>> On 20/11/13 15:54, Daniel Kulp wrote:
>>>
>>> On Nov 19, 2013, at 10:20 PM, Jason Wang <ja...@gmail.com>
>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I would like to limit the frequencies our APIs can be called. Given
>>>> that they will be public APIs.
>>>> The limit will most likely be done on IP addresses.
>>>>
>>>> Is there existing mechanism in CXF for this? Otherwise I will create
>>>> my own interceptor to do it.
>>>
>>> Currently, no.  I had some similar discussions about this with some folks last
>> week related more about throttling per endpoint instead of per IP.
>> However, many of the thoughts are the same.   Kind of came up with this list
>> of thoughts to think about:
>>>
>>> 1) What should happen if more than the needed requests come in?  Should
>> they be queued and processed later?   Should a fault be thrown?  Should
>> some number be queued and then a fault thrown beyond that?   Lots of
>> possible config options here.
>>>
>>
>> May be we can ship a couple of basic interceptors which would return 503 if
>> the rate exceeds. One pair of interceptors would go to the core and would
>> simply check how many concurrent requests are under way, another pair will
>> go to the http module and it will rate the individual client IP addresses, the
>> ideas you suggested below can further be explored to support more
>> advanced options
>
> Yep, I find that this option is a nice first step for CXF throttling. As Dan said, I see more sophisticated implementation (with queuing) is more mediation as middleware task.
> I would also provide the possibility to activate these interceptors through WS-Policy assertion with corresponded parameters.
>
I thought more about Dan's suggestion to consider suspending the threads 
should the rate exceed the limit, I have to admit it is tempting to try 
to do that :-), especially given that we can have a thread suspended and 
resumed in CXF literally with the few lines of code, we can def 
prototype something simple enough,

Thanks Sergey


> Regards,
> Andrei.
>
>> Thanks, Sergey
>>
>>> 2) If you want to do this at an endpoint level via an executor, the CXF
>> schemas do have an "executor" element for the jaxws:endpoint element
>> that can be used to set a specific executor.  There are a couple of "Executor"
>> things that can provide limits that may be able to plug right in here.  That said,
>> I'd discourage this.   When using an executor, when a request comes in on a
>> Jetty (or other transport thread), we have to place the request on the
>> executor and then block the transport thread until the request finishes.
>> Thus, it ties up two threads and jetty cannot process more while it's waiting.
>> That said, there is definitely a possible enhancement here.  If using a
>> transport that supports the CXF continuations, we COULD start a
>> continuation prior to flipping to the executor.   Something to think about a bit
>> more.
>>>
>>> 3) Possibly the more "correct" answer to this is that this is a
>> mediation/Camel feature, not a service feature.   CXF is about
>> creating/exposing services.   Placing quality of services requirements around
>> that service is a mediation thing.   That could be considered Camel's job.   This
>> could be a  from("jetty://....").throttle(...).to("cxf:...") type thing.  Not sure if
>> the Camel throttling has support for per-ip throttling or not.  Would need to
>> investigate more.
>>>
>>> 4) You likely could implement this as a set of CXF interceptors that could
>> use the Continuations to "pause" the request for a few milliseconds or similar
>> if the load is too high.   Would require some extra coding.   Contributions back
>> could be welcome.
>>>
>>> 5) Jetty (and likely Tomcat and others) do have some throttling control built
>> in at the servlet engine level.  You may want to investigate that.
>> Additionally, if you run your web service behind an Apache proxy, I believe
>> the mod_proxy stuff in Apache has some settings for this.
>>>
>>> Anyway, lots of thoughts, but haven't had time to really look into any of
>> them yet.
>>>

Re: Any existing implemention on limiting calling frequencies by client or by IP

[Sergey Beryozkin <sb...@gmail.com>]

On 22/11/13 07:37, Jason Wang wrote:
> All great ideas guys. Should I raise an enhancement request?
>
Please do

> I can easily do one for my own usage. But to make it generic enough for
> everyone, there needs to be something to store the ip, attempts record.
> I am using ehcache at the moment. If I expose a setter for cache provider
> the same way hibernate does, will it be a solution for general CXF users?
IMHO having an in memory map should be enough for an out-of the box CXF 
HTTP-level interceptor, it does not need to be recreated after restarts, 
and if we have a case where there are many thousands or millions of 
clients coming from diff IP addresses then managing it with the proper 
persistence manager plus tackling all other related statistics and 
requirements will indeed be out of scope for CXF, we will be talking 
about the dedicated handler sitting in front of CXF, etc.

Though you are right, we can have a basic interface representing a 
record storage, so people can get the records managed whichever way they 
like

Sergey

>
> Cheers,
>
>
> On Fri, Nov 22, 2013 at 1:08 AM, Andrei Shakirin <as...@talend.com>wrote:
>
>> Hi Sergei,
>>
>>> -----Original Message-----
>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>> Sent: Donnerstag, 21. November 2013 11:35
>>> To: users@cxf.apache.org
>>> Subject: Re: Any existing implemention on limiting calling frequencies by
>>> client or by IP
>>>
>>> Hi Dan
>>> On 20/11/13 15:54, Daniel Kulp wrote:
>>>>
>>>> On Nov 19, 2013, at 10:20 PM, Jason Wang <ja...@gmail.com>
>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I would like to limit the frequencies our APIs can be called. Given
>>>>> that they will be public APIs.
>>>>> The limit will most likely be done on IP addresses.
>>>>>
>>>>> Is there existing mechanism in CXF for this? Otherwise I will create
>>>>> my own interceptor to do it.
>>>>
>>>> Currently, no.  I had some similar discussions about this with some
>> folks last
>>> week related more about throttling per endpoint instead of per IP.
>>> However, many of the thoughts are the same.   Kind of came up with this
>> list
>>> of thoughts to think about:
>>>>
>>>> 1) What should happen if more than the needed requests come in?  Should
>>> they be queued and processed later?   Should a fault be thrown?  Should
>>> some number be queued and then a fault thrown beyond that?   Lots of
>>> possible config options here.
>>>>
>>>
>>> May be we can ship a couple of basic interceptors which would return 503
>> if
>>> the rate exceeds. One pair of interceptors would go to the core and would
>>> simply check how many concurrent requests are under way, another pair
>> will
>>> go to the http module and it will rate the individual client IP
>> addresses, the
>>> ideas you suggested below can further be explored to support more
>>> advanced options
>>
>> Yep, I find that this option is a nice first step for CXF throttling. As
>> Dan said, I see more sophisticated implementation (with queuing) is more
>> mediation as middleware task.
>> I would also provide the possibility to activate these interceptors
>> through WS-Policy assertion with corresponded parameters.
>>
>> Regards,
>> Andrei.
>>
>>> Thanks, Sergey
>>>
>>>> 2) If you want to do this at an endpoint level via an executor, the CXF
>>> schemas do have an "executor" element for the jaxws:endpoint element
>>> that can be used to set a specific executor.  There are a couple of
>> "Executor"
>>> things that can provide limits that may be able to plug right in here.
>>   That said,
>>> I'd discourage this.   When using an executor, when a request comes in
>> on a
>>> Jetty (or other transport thread), we have to place the request on the
>>> executor and then block the transport thread until the request finishes.
>>> Thus, it ties up two threads and jetty cannot process more while it's
>> waiting.
>>> That said, there is definitely a possible enhancement here.  If using a
>>> transport that supports the CXF continuations, we COULD start a
>>> continuation prior to flipping to the executor.   Something to think
>> about a bit
>>> more.
>>>>
>>>> 3) Possibly the more "correct" answer to this is that this is a
>>> mediation/Camel feature, not a service feature.   CXF is about
>>> creating/exposing services.   Placing quality of services requirements
>> around
>>> that service is a mediation thing.   That could be considered Camel's
>> job.   This
>>> could be a  from("jetty://....").throttle(...).to("cxf:...") type thing.
>>   Not sure if
>>> the Camel throttling has support for per-ip throttling or not.  Would
>> need to
>>> investigate more.
>>>>
>>>> 4) You likely could implement this as a set of CXF interceptors that
>> could
>>> use the Continuations to "pause" the request for a few milliseconds or
>> similar
>>> if the load is too high.   Would require some extra coding.
>> Contributions back
>>> could be welcome.
>>>>
>>>> 5) Jetty (and likely Tomcat and others) do have some throttling
>> control built
>>> in at the servlet engine level.  You may want to investigate that.
>>> Additionally, if you run your web service behind an Apache proxy, I
>> believe
>>> the mod_proxy stuff in Apache has some settings for this.
>>>>
>>>> Anyway, lots of thoughts, but haven't had time to really look into any
>> of
>>> them yet.
>>>>
>>
>


-- 
Sergey Beryozkin

Talend Community Coders
http://coders.talend.com/

Blog: http://sberyozkin.blogspot.com

Re: Any existing implemention on limiting calling frequencies by client or by IP

[Jason Wang <ja...@gmail.com>]

All great ideas guys. Should I raise an enhancement request?

I can easily do one for my own usage. But to make it generic enough for
everyone, there needs to be something to store the ip, attempts record.
I am using ehcache at the moment. If I expose a setter for cache provider
the same way hibernate does, will it be a solution for general CXF users?

Cheers,


On Fri, Nov 22, 2013 at 1:08 AM, Andrei Shakirin <as...@talend.com>wrote:

> Hi Sergei,
>
> > -----Original Message-----
> > From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
> > Sent: Donnerstag, 21. November 2013 11:35
> > To: users@cxf.apache.org
> > Subject: Re: Any existing implemention on limiting calling frequencies by
> > client or by IP
> >
> > Hi Dan
> > On 20/11/13 15:54, Daniel Kulp wrote:
> > >
> > > On Nov 19, 2013, at 10:20 PM, Jason Wang <ja...@gmail.com>
> > wrote:
> > >
> > >> Hi all,
> > >>
> > >> I would like to limit the frequencies our APIs can be called. Given
> > >> that they will be public APIs.
> > >> The limit will most likely be done on IP addresses.
> > >>
> > >> Is there existing mechanism in CXF for this? Otherwise I will create
> > >> my own interceptor to do it.
> > >
> > > Currently, no.  I had some similar discussions about this with some
> folks last
> > week related more about throttling per endpoint instead of per IP.
> > However, many of the thoughts are the same.   Kind of came up with this
> list
> > of thoughts to think about:
> > >
> > > 1) What should happen if more than the needed requests come in?  Should
> > they be queued and processed later?   Should a fault be thrown?  Should
> > some number be queued and then a fault thrown beyond that?   Lots of
> > possible config options here.
> > >
> >
> > May be we can ship a couple of basic interceptors which would return 503
> if
> > the rate exceeds. One pair of interceptors would go to the core and would
> > simply check how many concurrent requests are under way, another pair
> will
> > go to the http module and it will rate the individual client IP
> addresses, the
> > ideas you suggested below can further be explored to support more
> > advanced options
>
> Yep, I find that this option is a nice first step for CXF throttling. As
> Dan said, I see more sophisticated implementation (with queuing) is more
> mediation as middleware task.
> I would also provide the possibility to activate these interceptors
> through WS-Policy assertion with corresponded parameters.
>
> Regards,
> Andrei.
>
> > Thanks, Sergey
> >
> > > 2) If you want to do this at an endpoint level via an executor, the CXF
> > schemas do have an "executor" element for the jaxws:endpoint element
> > that can be used to set a specific executor.  There are a couple of
> "Executor"
> > things that can provide limits that may be able to plug right in here.
>  That said,
> > I'd discourage this.   When using an executor, when a request comes in
> on a
> > Jetty (or other transport thread), we have to place the request on the
> > executor and then block the transport thread until the request finishes.
> > Thus, it ties up two threads and jetty cannot process more while it's
> waiting.
> > That said, there is definitely a possible enhancement here.  If using a
> > transport that supports the CXF continuations, we COULD start a
> > continuation prior to flipping to the executor.   Something to think
> about a bit
> > more.
> > >
> > > 3) Possibly the more "correct" answer to this is that this is a
> > mediation/Camel feature, not a service feature.   CXF is about
> > creating/exposing services.   Placing quality of services requirements
> around
> > that service is a mediation thing.   That could be considered Camel's
> job.   This
> > could be a  from("jetty://....").throttle(...).to("cxf:...") type thing.
>  Not sure if
> > the Camel throttling has support for per-ip throttling or not.  Would
> need to
> > investigate more.
> > >
> > > 4) You likely could implement this as a set of CXF interceptors that
> could
> > use the Continuations to "pause" the request for a few milliseconds or
> similar
> > if the load is too high.   Would require some extra coding.
> Contributions back
> > could be welcome.
> > >
> > > 5) Jetty (and likely Tomcat and others) do have some throttling
> control built
> > in at the servlet engine level.  You may want to investigate that.
> > Additionally, if you run your web service behind an Apache proxy, I
> believe
> > the mod_proxy stuff in Apache has some settings for this.
> > >
> > > Anyway, lots of thoughts, but haven't had time to really look into any
> of
> > them yet.
> > >
>

RE: Any existing implemention on limiting calling frequencies by client or by IP

[Andrei Shakirin <as...@talend.com>]

Hi Sergei,

> -----Original Message-----
> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
> Sent: Donnerstag, 21. November 2013 11:35
> To: users@cxf.apache.org
> Subject: Re: Any existing implemention on limiting calling frequencies by
> client or by IP
> 
> Hi Dan
> On 20/11/13 15:54, Daniel Kulp wrote:
> >
> > On Nov 19, 2013, at 10:20 PM, Jason Wang <ja...@gmail.com>
> wrote:
> >
> >> Hi all,
> >>
> >> I would like to limit the frequencies our APIs can be called. Given
> >> that they will be public APIs.
> >> The limit will most likely be done on IP addresses.
> >>
> >> Is there existing mechanism in CXF for this? Otherwise I will create
> >> my own interceptor to do it.
> >
> > Currently, no.  I had some similar discussions about this with some folks last
> week related more about throttling per endpoint instead of per IP.
> However, many of the thoughts are the same.   Kind of came up with this list
> of thoughts to think about:
> >
> > 1) What should happen if more than the needed requests come in?  Should
> they be queued and processed later?   Should a fault be thrown?  Should
> some number be queued and then a fault thrown beyond that?   Lots of
> possible config options here.
> >
> 
> May be we can ship a couple of basic interceptors which would return 503 if
> the rate exceeds. One pair of interceptors would go to the core and would
> simply check how many concurrent requests are under way, another pair will
> go to the http module and it will rate the individual client IP addresses, the
> ideas you suggested below can further be explored to support more
> advanced options

Yep, I find that this option is a nice first step for CXF throttling. As Dan said, I see more sophisticated implementation (with queuing) is more mediation as middleware task.
I would also provide the possibility to activate these interceptors through WS-Policy assertion with corresponded parameters.

Regards,
Andrei.

> Thanks, Sergey
> 
> > 2) If you want to do this at an endpoint level via an executor, the CXF
> schemas do have an "executor" element for the jaxws:endpoint element
> that can be used to set a specific executor.  There are a couple of "Executor"
> things that can provide limits that may be able to plug right in here.  That said,
> I'd discourage this.   When using an executor, when a request comes in on a
> Jetty (or other transport thread), we have to place the request on the
> executor and then block the transport thread until the request finishes.
> Thus, it ties up two threads and jetty cannot process more while it's waiting.
> That said, there is definitely a possible enhancement here.  If using a
> transport that supports the CXF continuations, we COULD start a
> continuation prior to flipping to the executor.   Something to think about a bit
> more.
> >
> > 3) Possibly the more "correct" answer to this is that this is a
> mediation/Camel feature, not a service feature.   CXF is about
> creating/exposing services.   Placing quality of services requirements around
> that service is a mediation thing.   That could be considered Camel's job.   This
> could be a  from("jetty://....").throttle(...).to("cxf:...") type thing.  Not sure if
> the Camel throttling has support for per-ip throttling or not.  Would need to
> investigate more.
> >
> > 4) You likely could implement this as a set of CXF interceptors that could
> use the Continuations to "pause" the request for a few milliseconds or similar
> if the load is too high.   Would require some extra coding.   Contributions back
> could be welcome.
> >
> > 5) Jetty (and likely Tomcat and others) do have some throttling control built
> in at the servlet engine level.  You may want to investigate that.
> Additionally, if you run your web service behind an Apache proxy, I believe
> the mod_proxy stuff in Apache has some settings for this.
> >
> > Anyway, lots of thoughts, but haven't had time to really look into any of
> them yet.
> >

Re: Any existing implemention on limiting calling frequencies by client or by IP

[Sergey Beryozkin <sb...@gmail.com>]

Hi Dan
On 20/11/13 15:54, Daniel Kulp wrote:
>
> On Nov 19, 2013, at 10:20 PM, Jason Wang <ja...@gmail.com> wrote:
>
>> Hi all,
>>
>> I would like to limit the frequencies our APIs can be called. Given that
>> they will be public APIs.
>> The limit will most likely be done on IP addresses.
>>
>> Is there existing mechanism in CXF for this? Otherwise I will create my own
>> interceptor to do it.
>
> Currently, no.  I had some similar discussions about this with some folks last week related more about throttling per endpoint instead of per IP.  However, many of the thoughts are the same.   Kind of came up with this list of thoughts to think about:
>
> 1) What should happen if more than the needed requests come in?  Should they be queued and processed later?   Should a fault be thrown?  Should some number be queued and then a fault thrown beyond that?   Lots of possible config options here.
>

May be we can ship a couple of basic interceptors which would return 503 
if the rate exceeds. One pair of interceptors would go to the core and 
would simply check how many concurrent requests are under way, another 
pair will go to the http module and it will rate the individual client 
IP addresses, the ideas you suggested below can further be explored to 
support more advanced options

Thanks, Sergey

> 2) If you want to do this at an endpoint level via an executor, the CXF schemas do have an “executor” element for the jaxws:endpoint element that can be used to set a specific executor.  There are a couple of “Executor” things that can provide limits that may be able to plug right in here.  That said, I’d discourage this.   When using an executor, when a request comes in on a Jetty (or other transport thread), we have to place the request on the executor and then block the transport thread until the request finishes.  Thus, it ties up two threads and jetty cannot process more while it’s waiting.   That said, there is definitely a possible enhancement here.  If using a transport that supports the CXF continuations, we COULD start a continuation prior to flipping to the executor.   Something to think about a bit more.
>
> 3) Possibly the more “correct” answer to this is that this is a mediation/Camel feature, not a service feature.   CXF is about creating/exposing services.   Placing quality of services requirements around that service is a mediation thing.   That could be considered Camel’s job.   This could be a  from(“jetty://….”).throttle(…).to(“cxf:…”) type thing.  Not sure if the Camel throttling has support for per-ip throttling or not.  Would need to investigate more.
>
> 4) You likely could implement this as a set of CXF interceptors that could use the Continuations to “pause” the request for a few milliseconds or similar if the load is too high.   Would require some extra coding.   Contributions back could be welcome.
>
> 5) Jetty (and likely Tomcat and others) do have some throttling control built in at the servlet engine level.  You may want to investigate that.   Additionally, if you run your web service behind an Apache proxy, I believe the mod_proxy stuff in Apache has some settings for this.
>
> Anyway, lots of thoughts, but haven’t had time to really look into any of them yet.
>

Re: Any existing implemention on limiting calling frequencies by client or by IP

[Daniel Kulp <dk...@apache.org>]

On Nov 19, 2013, at 10:20 PM, Jason Wang <ja...@gmail.com> wrote:

> Hi all,
> 
> I would like to limit the frequencies our APIs can be called. Given that
> they will be public APIs.
> The limit will most likely be done on IP addresses.
> 
> Is there existing mechanism in CXF for this? Otherwise I will create my own
> interceptor to do it.

Currently, no.  I had some similar discussions about this with some folks last week related more about throttling per endpoint instead of per IP.  However, many of the thoughts are the same.   Kind of came up with this list of thoughts to think about:

1) What should happen if more than the needed requests come in?  Should they be queued and processed later?   Should a fault be thrown?  Should some number be queued and then a fault thrown beyond that?   Lots of possible config options here.

2) If you want to do this at an endpoint level via an executor, the CXF schemas do have an “executor” element for the jaxws:endpoint element that can be used to set a specific executor.  There are a couple of “Executor” things that can provide limits that may be able to plug right in here.  That said, I’d discourage this.   When using an executor, when a request comes in on a Jetty (or other transport thread), we have to place the request on the executor and then block the transport thread until the request finishes.  Thus, it ties up two threads and jetty cannot process more while it’s waiting.   That said, there is definitely a possible enhancement here.  If using a transport that supports the CXF continuations, we COULD start a continuation prior to flipping to the executor.   Something to think about a bit more.

3) Possibly the more “correct” answer to this is that this is a mediation/Camel feature, not a service feature.   CXF is about creating/exposing services.   Placing quality of services requirements around that service is a mediation thing.   That could be considered Camel’s job.   This could be a  from(“jetty://….”).throttle(…).to(“cxf:…”) type thing.  Not sure if the Camel throttling has support for per-ip throttling or not.  Would need to investigate more.

4) You likely could implement this as a set of CXF interceptors that could use the Continuations to “pause” the request for a few milliseconds or similar if the load is too high.   Would require some extra coding.   Contributions back could be welcome.

5) Jetty (and likely Tomcat and others) do have some throttling control built in at the servlet engine level.  You may want to investigate that.   Additionally, if you run your web service behind an Apache proxy, I believe the mod_proxy stuff in Apache has some settings for this.

Anyway, lots of thoughts, but haven’t had time to really look into any of them yet.

-- 
Daniel Kulp
dkulp@apache.org - http://dankulp.com/blog
Talend Community Coder - http://coders.talend.com

Re: Any existing implemention on limiting calling frequencies by client or by IP

[Sergey Beryozkin <sb...@gmail.com>]

Hi
On 20/11/13 03:20, Jason Wang wrote:
> Hi all,
>
> I would like to limit the frequencies our APIs can be called. Given that
> they will be public APIs.
> The limit will most likely be done on IP addresses.
>
> Is there existing mechanism in CXF for this? Otherwise I will create my own
> interceptor to do it.
>
At the moment you can try the container specific filters, example, I've 
heard that Jetty offers the one, etc.

That said, it might make sense to offer a couple of CXF utility 
interceptors, supporting the rate limitation at the different levels.

Cheers, Sergey
> Cheers,
> Jason
>