You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by javalishixml <ja...@163.com> on 2015/05/18 17:44:53 UTC
how to block the duplicated requests?
Hi,
I have a website. It is built by apache + tomcat.
Now we make a lottery activity at this website. But we find that some robots always raise the duplicated requests to hit this lottery activity. It causes that robots almost get all the awards.
So we just want to block these kind of duplicated requests at every interval unit.
For example, we set the interval unit is 3 seconds. The if the robot want to hit the lottery activity in 3 seconds, the website could block this action.
So how to do it? I suppose if we do it at tomcat level, is it a very low performance? Can I do it at apache level? how to do it?
If I could not do it apache level, can I do it by setting sth at tomcat?
Thanks in advance,
Java Coder
Re: how to block the duplicated requests?
Posted by David kerber <dc...@verizon.net>.
On 5/19/2015 8:09 AM, javalishixml wrote:
> Just understood you. Really appreciate for your feedback.
>
>
> How do we judge it's a robot?
> item1: we find the request IP is always the same one.
> item2: our page may contains several keep-alive connections. But the attack connection only focus on connection.
>
> Based on these 2 items, we think the client is a robot.
>
>
>
> I think maybe putting these 2 items together to consider it as a robot is a bit complex. Let's do it from the simple point.
> If we alway find there is a same IP request our website the same url for many times, can I block this request at httpd level?
Yes, it's easy to block a request from a specific IP address. You can
do it either with httpd, or further upstream at your firewall. I don't
use httpd, so can't help you with the specific instructions on how to do
it, but I know it's not difficult.
>
>
> Thanks,
>
>
> At 2015-05-19 20:01:00, "David kerber" <dc...@verizon.net> wrote:
>> On 5/19/2015 7:53 AM, javalishixml wrote:
>>>
>>>
>>>
>>>> I doubt you're going to be able to do this in httpd, unless you have a very simple, straight forward way of identifying the robots.
>>> Yes. I just want to have a way to block the duplicated requests at httpd level. After all, my website has to face the the big concurrency issue.
>>
>> I understand that's what you want. What we're telling you is that you
>> probably won't be able to do that.
>>
>> Let me ask the question again, that Chris asked before: how do you tell
>> that a given request is from a robot?
>>
>> The answer to that question will determine if you can block it with httpd.
>>
>>>
>>>
>>>
>>>
>>>
>>> At 2015-05-19 19:35:26, "David kerber" <dc...@verizon.net> wrote:
>>>> On 5/19/2015 1:03 AM, javalishixml wrote:
>>>>> Thanks a lot for your information.
>>>>>
>>>>>
>>>>> This solution is based on tomcat level. If I always handle this issue at java level, I'm afraid it has performance issue. Because this web site afford a very big concurrency access.
>>>>>
>>>>>
>>>>> Taking a consideration on its basic architect tomcat+apache, I think the best way to move this solution from tomcat to apache. So do you have some good solution at apache's configuration? I understand this is a mail list for tomcat.. but just want to get any information
>>>>
>>>> I doubt you're going to be able to do this in httpd, unless you have a
>>>> very simple, straight forward way of identifying the robots.
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>>
>>>>> At 2015-05-19 04:00:28, "Christopher Schultz" <ch...@christopherschultz.net> wrote:
>>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>>> Hash: SHA256
>>>>>>
>>>>>> To whom it may concern,
>>>>>>
>>>>>> On 5/18/15 11:44 AM, javalishixml wrote:
>>>>>>> I have a website. It is built by apache + tomcat.
>>>>>>>
>>>>>>> Now we make a lottery activity at this website. But we find that
>>>>>>> some robots always raise the duplicated requests to hit this
>>>>>>> lottery activity. It causes that robots almost get all the awards.
>>>>>>>
>>>>>>> So we just want to block these kind of duplicated requests at every
>>>>>>> interval unit. For example, we set the interval unit is 3 seconds.
>>>>>>> The if the robot want to hit the lottery activity in 3 seconds, the
>>>>>>> website could block this action.
>>>>>>>
>>>>>>> So how to do it? I suppose if we do it at tomcat level, is it a
>>>>>>> very low performance? Can I do it at apache level? how to do it? If
>>>>>>> I could not do it apache level, can I do it by setting sth at
>>>>>>> tomcat?
>>>>>>
>>>>>> If you have a way to identify a "duplicate" request (e.g. using a
>>>>>> fingerprint of the request that you can check during that 3-second
>>>>>> interval), then this is conceptually very easy.
>>>>>>
>>>>>> It may not be great for performance, but you'll have to weigh that
>>>>>> against your own requirements. (For example, which is worse: poor
>>>>>> performance, or a site where only robots ever win the lottery?)
>>>>>>
>>>>>> This will not be something you can configure in Apache httpd or
>>>>>> Tomcat. This will have to be an application thing (unless you can
>>>>>> describe the fingerprint technique to some httpd module such as
>>>>>> mod_security or mod_qos and then allow it to discard duplicates).
>>>>>>
>>>>>> Back to the solution:
>>>>>>
>>>>>> 1. Take a fingerprint of the request
>>>>>> 2. Lookup the fingerprint in a database of previous requests
>>>>>> ( fingerprint -> latest timestamp )
>>>>>> 3. If the fingerprint appears in your database and the timestamp is
>>>>>> less than 3 seconds ago, discard the request
>>>>>> 4. Otherwise, store the current timestamp and fingerprint in the databas
>>>>>> e
>>>>>>
>>>>>> For a database, I might recommend something like memcached or another
>>>>>> in-memory-style database. An in-memory key-value store is really what
>>>>>> you are looking for. Memcached has a nice feature where values can
>>>>>> automatically time-out (e.g. they are invalid after 3 seconds), so you
>>>>>> can make your application code a bit simpler because you'll never have
>>>>>> a value in the database that is not valid.
>>>>>>
>>>>>> Hope that helps,
>>>>>> - -chris
>>>>>> -----BEGIN PGP SIGNATURE-----
>>>>>> Version: GnuPG v2
>>>>>> Comment: GPGTools - http://gpgtools.org
>>>>>>
>>>>>> iQIcBAEBCAAGBQJVWkTcAAoJEBzwKT+lPKRYnW0QAIeRbfJtsTKtUZHUig9sIRre
>>>>>> y1mgJkPxBXjcRTfoZkZkTPhasYzINE1mb1mTPKfPbQveH+OmpawDREWJxg/6dFeg
>>>>>> af734ZRpBOAs4MtlCyTXgBUWpWka5CcpeIRYeEwx5GKPFLJfTBbGpswV3HwLaoEC
>>>>>> /NqMByVfwHnixBxSTGAM2GIOyrPf+Ii1Z0JpQyDEYcZUS3Dc3IFFeHPTvzQUb1SO
>>>>>> NB84fwjDT6GG/YerrlRV3GHL3WYhAw1n+tQ9cCpSWDvz8/KLUyKXqVjX5s/FbuB+
>>>>>> S+krz2jzKqxG8bdeixW4s0i/9gyA/KcSSDgwmBnRwHsIUDvfF3pzk1Vq7rfGNpmQ
>>>>>> L9V4brxL41H+ZMIDt2NjkVJb/UjgMnL5RpfQ1t+MdNvys/7UYav+vOv8jWqI3Mse
>>>>>> AXNv46mQZAiMFzs/nsR7OIVLLxU70l+wbys4mK6u34uDip5gzxvVSaYKviqgKspx
>>>>>> LT6MUHOpgmBhsiCUxjJ5odA4Q6mYhMfQxOB+6Ej8jRfKMT2uDTlwvU8gZ+/7TcUX
>>>>>> JXngjQLQyjj+gAO+7jS7sWpaakV1ojy8/nFBVWH/3tWoo0YD89DJCRWxA8x8slfx
>>>>>> oI9BGA0T7EwuX1CnqM90OLw7dymMQvwsTlkPAZnIvnWw3Xz29hIRazxQ7NR3AdCk
>>>>>> vNXsseUzO18IJ4n+By1G
>>>>>> =Q/ki
>>>>>> -----END PGP SIGNATURE-----
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re:Re: how to block the duplicated requests?
Posted by javalishixml <ja...@163.com>.
Understood. Really appreciate for your nice time.
Thanks,
At 2015-05-20 21:00:33, "Christopher Schultz" <ch...@christopherschultz.net> wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA256
>
>
>
>On 5/20/15 4:22 AM, javalishixml wrote:
>> More detail information as below:
>>
>> presudo-code step:
>
>This isn't pseudo-code. This is a re-statement of your problem.
>
>> 1. a register page named "http://mywebsite.com/register1.jsp" is
>> set up, and this page contains a CAPTCHA image
>
>You didn't mention that CAPTCHA was already being used. Someone
>mentioned using it as a solution to your problem. What CAPTCHA are you
>using? Perhaps using a more effective one would help more than
>anything else.
>
>> 2. the robot(crackers) could successfully register the thousands
>> different users for this web site during only several minutes.
>>
>> 3. if it is a human beings, these thousands different users should
>> have different IPs. But we find these thousands different users
>> are from same IPs.
>
>No chance these are AOL users? Google for "AOL ip address proxy".
>
>> By the way, we get the IP from HttpServletRequest header.
>
>Where else would you get the remote IP address?
>
>> 4. later, we setup a new register page. We change its url from
>> "http://mywebsite.com/register1.jsp" to
>> "http://mywebsite.com/register2.jsp"
>
>Are you trying to be evasive? Why have you moved your registration page?
>
>> For the first several days, we find everything is good.
>>
>> But after several days, we find the robot(crackers) find this new
>> URL and could successfully register the thousands different users
>> for this web site during only several minutes.
>>
>> It's just reproduced steps for our issue.
>
>So, back to my original question: How are you going to identify a
>"duplicate" request? Show some pseudo-code.
>
>> Our requirements are that: 1. we have a URL for register page. we
>> don't want the thousands different users with same IP could
>> successfully registered during a very short time window.
>
>What about users behind proxies? Are you okay shutting them out? See
>the AOL anecdote above.
>
>> 2. We can have a policy to set an interval time window. Based on
>> this interval time window, the same IP should NOT register users
>> again and again.
>>
>> 3. This policy should manage a group of URLs. We can always add
>> the different URLs for this policy. Because based on our
>> maintaining activities, we may set up many different register page
>> again and again.
>>
>>
>> Is it a DDOS attack?
>
>Are they preventing anyone else from using your site? Or are they just
>raising their numbers quickly enough that statistically, they always
>overwhelm your legitimate users and "win" the "lottery"?
>
>> Is there a good way to resolve it at httpd level?
>
>Seriously, look-up mod_qos, mod_evasive, and mod_security and stop
>asking for solutions. We've already given you a whole bunch of ideas
>that consultants would have already bankrupted you for. Go do some work.
>
>- -chris
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v2
>Comment: GPGTools - http://gpgtools.org
>
>iQIcBAEBCAAGBQJVXIVxAAoJEBzwKT+lPKRY3AMQAIWrelhsrB9WnB8c+Wq7S2ia
>+L1dU+ZTI+VEeFBWy1ARUTtXM/viL7mE7QfofVVEjmMYAxrITrk9Nqn0DzmGBJAG
>JNcPkSHVAvhH9thOJDCfLvD69hV5sGCJdNC6RlYn235IEiai1IhH6ZQudrCXAPjl
>mMjZPX30W65MbA7fBMWG4NUJFi2BBz07zV8/teIwHQ/3w9fTs63o18alRwP5cGUk
>i1yu0lBf63xO5r7xnS5jN9fvklZe6FrCS+6RK2AAj2viF7mGi3kmaco1fdSQmTLY
>rdadMd0M9P6BgowMtBUAVNX4DnqJc2GIo8xlCySC/myvp8y3T9vwOvyRERoSW+8h
>a7oEPV6SKlFYKLHNg0XVgmkT3PHTjqojh2eOlKh8vO3W5YTw2R3xqXa4WUN0dHur
>cbD2RjSm7mA0Ewl+E2YsCbJAdfuPt3w77mIuv3FaV6ZPWdXLtSq0QARfGju0S11x
>bdEBaOzsQsm29qOC5MKMqG0tgHlY1Ya3BnGGxI+GTMat91d8kp92ufWeS5bmda3I
>BqOosM+GkgY9P1DATPXpR5A8Xi5Pp/lgkD4MYVNka2VH7FgKWckXlUhWoilDqFDX
>k4R9z/ZaRrDwqt6lwSAlRN4znwTw0OyP9FSLGr+VIKfKRUyweJss6pVUUGpxd3yQ
>ytK9Cbw2UpbOyFaiA1AE
>=CHtu
>-----END PGP SIGNATURE-----
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org
>
Re: how to block the duplicated requests?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
On 5/20/15 4:22 AM, javalishixml wrote:
> More detail information as below:
>
> presudo-code step:
This isn't pseudo-code. This is a re-statement of your problem.
> 1. a register page named "http://mywebsite.com/register1.jsp" is
> set up, and this page contains a CAPTCHA image
You didn't mention that CAPTCHA was already being used. Someone
mentioned using it as a solution to your problem. What CAPTCHA are you
using? Perhaps using a more effective one would help more than
anything else.
> 2. the robot(crackers) could successfully register the thousands
> different users for this web site during only several minutes.
>
> 3. if it is a human beings, these thousands different users should
> have different IPs. But we find these thousands different users
> are from same IPs.
No chance these are AOL users? Google for "AOL ip address proxy".
> By the way, we get the IP from HttpServletRequest header.
Where else would you get the remote IP address?
> 4. later, we setup a new register page. We change its url from
> "http://mywebsite.com/register1.jsp" to
> "http://mywebsite.com/register2.jsp"
Are you trying to be evasive? Why have you moved your registration page?
> For the first several days, we find everything is good.
>
> But after several days, we find the robot(crackers) find this new
> URL and could successfully register the thousands different users
> for this web site during only several minutes.
>
> It's just reproduced steps for our issue.
So, back to my original question: How are you going to identify a
"duplicate" request? Show some pseudo-code.
> Our requirements are that: 1. we have a URL for register page. we
> don't want the thousands different users with same IP could
> successfully registered during a very short time window.
What about users behind proxies? Are you okay shutting them out? See
the AOL anecdote above.
> 2. We can have a policy to set an interval time window. Based on
> this interval time window, the same IP should NOT register users
> again and again.
>
> 3. This policy should manage a group of URLs. We can always add
> the different URLs for this policy. Because based on our
> maintaining activities, we may set up many different register page
> again and again.
>
>
> Is it a DDOS attack?
Are they preventing anyone else from using your site? Or are they just
raising their numbers quickly enough that statistically, they always
overwhelm your legitimate users and "win" the "lottery"?
> Is there a good way to resolve it at httpd level?
Seriously, look-up mod_qos, mod_evasive, and mod_security and stop
asking for solutions. We've already given you a whole bunch of ideas
that consultants would have already bankrupted you for. Go do some work.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: GPGTools - http://gpgtools.org
iQIcBAEBCAAGBQJVXIVxAAoJEBzwKT+lPKRY3AMQAIWrelhsrB9WnB8c+Wq7S2ia
+L1dU+ZTI+VEeFBWy1ARUTtXM/viL7mE7QfofVVEjmMYAxrITrk9Nqn0DzmGBJAG
JNcPkSHVAvhH9thOJDCfLvD69hV5sGCJdNC6RlYn235IEiai1IhH6ZQudrCXAPjl
mMjZPX30W65MbA7fBMWG4NUJFi2BBz07zV8/teIwHQ/3w9fTs63o18alRwP5cGUk
i1yu0lBf63xO5r7xnS5jN9fvklZe6FrCS+6RK2AAj2viF7mGi3kmaco1fdSQmTLY
rdadMd0M9P6BgowMtBUAVNX4DnqJc2GIo8xlCySC/myvp8y3T9vwOvyRERoSW+8h
a7oEPV6SKlFYKLHNg0XVgmkT3PHTjqojh2eOlKh8vO3W5YTw2R3xqXa4WUN0dHur
cbD2RjSm7mA0Ewl+E2YsCbJAdfuPt3w77mIuv3FaV6ZPWdXLtSq0QARfGju0S11x
bdEBaOzsQsm29qOC5MKMqG0tgHlY1Ya3BnGGxI+GTMat91d8kp92ufWeS5bmda3I
BqOosM+GkgY9P1DATPXpR5A8Xi5Pp/lgkD4MYVNka2VH7FgKWckXlUhWoilDqFDX
k4R9z/ZaRrDwqt6lwSAlRN4znwTw0OyP9FSLGr+VIKfKRUyweJss6pVUUGpxd3yQ
ytK9Cbw2UpbOyFaiA1AE
=CHtu
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Re: how to block the duplicated requests?
Posted by Prabhu Mannu <pr...@fernridge.com>.
Hi
What you explaining now and what you explained before are completely
different story.
Simple and main cause for your issue is inefficiency and poor design.
why don't you simply count number of users registering on particular ip and
stop at threshold for certain amount of time.
A simple request filter servlet would do the job.
And maybe you might want to change the CAPTCHA, to a
not_so_obvious_to_figure_out kind
With regards
Prabhu
-----Original Message-----
From: javalishixml [mailto:javalishixml@163.com]
Sent: Wednesday, 20 May, 2015 4:22 PM
To: Tomcat Users List
Subject: Re:Re: how to block the duplicated requests?
More detail information as below:
presudo-code step:
1. a register page named "http://mywebsite.com/register1.jsp" is set up, and
this page contains a CAPTCHA image 2. the robot(crackers) could successfully
register the thousands different users for this web site during only several
minutes.
3. if it is a human beings, these thousands different users should have
different IPs. But we find these thousands different users are from same
IPs.
By the way, we get the IP from HttpServletRequest header.
4. later, we setup a new register page. We change its url from
"http://mywebsite.com/register1.jsp" to "http://mywebsite.com/register2.jsp"
For the first several days, we find everthing is good.
But after several days, we find the robot(crackers) find this new URL and
could successfully register the thousands different users for this web site
during only several minutes.
It's just reproduced steps for our issue.
Our requirements are that:
1. we have a URL for register page. we don't want the thousands different
users with same IP could successfully registered during a very short time
window.
2. We can have a policy to set an interval time window. Based on this
interval time window, the same IP should NOT register users again and again.
3. This policy should manage a group of URLs. We can always add the
different URLs for this policy. Because based on our maintaining
activities, we may set up many different register page again and again.
Is it a DDOS attack? Is there a good way to resolve it at httpd level?
At 2015-05-19 21:16:29, "Christopher Schultz" <ch...@christopherschultz.net>
wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA256
>
>To whom it may concern,
>
>On 5/19/15 8:09 AM, javalishixml wrote:
>> Just understood you. Really appreciate for your feedback.
>>
>>
>> How do we judge it's a robot? item1: we find the request IP is
>> always the same one. item2: our page may contains several
>> keep-alive connections. But the attack connection only focus on
>> connection.
>
>Based upon the first request, how can you tell that the robot is going
>to make later keep-alive requests?
>
>> Based on these 2 items, we think the client is a robot.
>
>Can you write some pseudo-code that shows the algorithm in its
>simplest form?
>
>> I think maybe putting these 2 items together to consider it as a
>> robot is a bit complex. Let's do it from the simple point.
>>
>> If we always find there is a same IP request our website the same
>> url for many times, can I block this request at httpd level?
>
>This sounds like a job for mod_qos, mod_evasive, or mod_security.
>
>- -chris
>
>> At 2015-05-19 20:01:00, "David kerber" <dc...@verizon.net>
>> wrote:
>>> On 5/19/2015 7:53 AM, javalishixml wrote:
>>>>
>>>>
>>>>
>>>>> I doubt you're going to be able to do this in httpd, unless
>>>>> you have a very simple, straight forward way of identifying
>>>>> the robots.
>>>> Yes. I just want to have a way to block the duplicated requests
>>>> at httpd level. After all, my website has to face the the big
>>>> concurrency issue.
>>>
>>> I understand that's what you want. What we're telling you is
>>> that you probably won't be able to do that.
>>>
>>> Let me ask the question again, that Chris asked before: how do
>>> you tell that a given request is from a robot?
>>>
>>> The answer to that question will determine if you can block it
>>> with httpd.
>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> At 2015-05-19 19:35:26, "David kerber" <dc...@verizon.net>
>>>> wrote:
>>>>> On 5/19/2015 1:03 AM, javalishixml wrote:
>>>>>> Thanks a lot for your information.
>>>>>>
>>>>>>
>>>>>> This solution is based on tomcat level. If I always handle
>>>>>> this issue at java level, I'm afraid it has performance
>>>>>> issue. Because this web site afford a very big concurrency
>>>>>> access.
>>>>>>
>>>>>>
>>>>>> Taking a consideration on its basic architect
>>>>>> tomcat+apache, I think the best way to move this solution
>>>>>> from tomcat to apache. So do you have some good solution at
>>>>>> apache's configuration? I understand this is a mail list
>>>>>> for tomcat.. but just want to get any information
>>>>>
>>>>> I doubt you're going to be able to do this in httpd, unless
>>>>> you have a very simple, straight forward way of identifying
>>>>> the robots.
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>
>>>>>> At 2015-05-19 04:00:28, "Christopher Schultz"
>>>>>> <ch...@christopherschultz.net> wrote:
>> To whom it may concern,
>>
>> On 5/18/15 11:44 AM, javalishixml wrote:
>>>>>>>>> I have a website. It is built by apache + tomcat.
>>>>>>>>>
>>>>>>>>> Now we make a lottery activity at this website. But
>>>>>>>>> we find that some robots always raise the duplicated
>>>>>>>>> requests to hit this lottery activity. It causes that
>>>>>>>>> robots almost get all the awards.
>>>>>>>>>
>>>>>>>>> So we just want to block these kind of duplicated
>>>>>>>>> requests at every interval unit. For example, we set
>>>>>>>>> the interval unit is 3 seconds. The if the robot want
>>>>>>>>> to hit the lottery activity in 3 seconds, the website
>>>>>>>>> could block this action.
>>>>>>>>>
>>>>>>>>> So how to do it? I suppose if we do it at tomcat
>>>>>>>>> level, is it a very low performance? Can I do it at
>>>>>>>>> apache level? how to do it? If I could not do it
>>>>>>>>> apache level, can I do it by setting sth at tomcat?
>>
>> If you have a way to identify a "duplicate" request (e.g. using a
>> fingerprint of the request that you can check during that 3-second
>> interval), then this is conceptually very easy.
>>
>> It may not be great for performance, but you'll have to weigh that
>> against your own requirements. (For example, which is worse: poor
>> performance, or a site where only robots ever win the lottery?)
>>
>> This will not be something you can configure in Apache httpd or
>> Tomcat. This will have to be an application thing (unless you can
>> describe the fingerprint technique to some httpd module such as
>> mod_security or mod_qos and then allow it to discard duplicates).
>>
>> Back to the solution:
>>
>> 1. Take a fingerprint of the request 2. Lookup the fingerprint in a
>> database of previous requests ( fingerprint -> latest timestamp )
>> 3. If the fingerprint appears in your database and the timestamp
>> is less than 3 seconds ago, discard the request 4. Otherwise, store
>> the current timestamp and fingerprint in the databas e
>>
>> For a database, I might recommend something like memcached or
>> another in-memory-style database. An in-memory key-value store is
>> really what you are looking for. Memcached has a nice feature where
>> values can automatically time-out (e.g. they are invalid after 3
>> seconds), so you can make your application code a bit simpler
>> because you'll never have a value in the database that is not
>> valid.
>>
>> Hope that helps, -chris
>>>>>>>
>>>>>>> -----------------------------------------------------------------
>- ----
>>>>>>>
>>>>>>>
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>> For additional commands, e-mail:
>>>>>>> users-help@tomcat.apache.org
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> -------------------------------------------------------------------
>- --
>>>>>
>>>>>
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>> For additional commands, e-mail:
>>>>> users-help@tomcat.apache.org
>>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>>
>>>
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v2
>Comment: GPGTools - http://gpgtools.org
>
>iQIcBAEBCAAGBQJVWzetAAoJEBzwKT+lPKRYh1sP+wQYPO7BY9Stg2XdvzK9GDA0
>/cqRMxIZ2Thq84GzHRKcg2pyC2iN3M+LMCxXCodKdp6+Cl0DfU6H0ijKucy4yHET
>FrwhHT/7A1A7bQ4eT6IYhu1R7dtLhM0o5YRwYDDClKSfvsACTmuLOEmin40bmTeH
>uuREu7EDz1fdNgOpjpDBNw1bC4ZkyYMlhQZ3Ox/jopnxKqRCXcjUx+kZH1UktJu9
>NBBs7rQgvSXFpNtB2DPFBS6pgy97jNSgKgWqoqX0WaQwQkfU1dLIyrH6I2v4/22s
>ldCE7sZDQrRR6PUAyMQVVj36H5WHH+xlxr0zSrkdxvy5bFFqjNoCb8fRh3+J8sGC
>yd0hesq3QW+uSwkwJOg4LsDkGFHCTYNiZZMLBRULGBabFbnHCXKIRjie3h0JGSfb
>uL8diC2sNydKH7Re8WifQ8wqPvqIqkN+oakHed+oLg6BhToJvZJY2mTYuJRdb6gc
>iRYRZ1+dH78PrgwLgovRDQHrnNVQRfUTpEyQOfygBi46o5Hh1t1fIADDMMtQ+yLX
>C/Fg7JO5+vN2AoG3A1UOHUmoTGi7bAOlqp4RUwJXdN9pho++ullP8X8SfAfW52VZ
>A5mZIb3FuI2GstOAcZPIzg63m1dr69d4CY3QCQdYUu5GnmlR9ws/LHwrI0Hbl4DH
>FiXD3fsPSgRYvjCXPISV
>=PWnL
>-----END PGP SIGNATURE-----
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re:Re: how to block the duplicated requests?
Posted by javalishixml <ja...@163.com>.
More detail information as below:
presudo-code step:
1. a register page named "http://mywebsite.com/register1.jsp" is set up, and this page contains a CAPTCHA image
2. the robot(crackers) could successfully register the thousands different users for this web site during only several minutes.
3. if it is a human beings, these thousands different users should have different IPs. But we find these thousands different users are from same IPs.
By the way, we get the IP from HttpServletRequest header.
4. later, we setup a new register page. We change its url from "http://mywebsite.com/register1.jsp" to "http://mywebsite.com/register2.jsp"
For the first several days, we find everthing is good.
But after several days, we find the robot(crackers) find this new URL and could successfully register the thousands different users for this web site during only several minutes.
It's just reproduced steps for our issue.
Our requirements are that:
1. we have a URL for register page. we don't want the thousands different users with same IP could successfully registered during a very short time window.
2. We can have a policy to set an interval time window. Based on this interval time window, the same IP should NOT register users again and again.
3. This policy should manage a group of URLs. We can always add the different URLs for this policy. Because based on our maintaining activities, we may set up many different register page again and again.
Is it a DDOS attack? Is there a good way to resolve it at httpd level?
At 2015-05-19 21:16:29, "Christopher Schultz" <ch...@christopherschultz.net> wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA256
>
>To whom it may concern,
>
>On 5/19/15 8:09 AM, javalishixml wrote:
>> Just understood you. Really appreciate for your feedback.
>>
>>
>> How do we judge it's a robot? item1: we find the request IP is
>> always the same one. item2: our page may contains several
>> keep-alive connections. But the attack connection only focus on
>> connection.
>
>Based upon the first request, how can you tell that the robot is going
>to make later keep-alive requests?
>
>> Based on these 2 items, we think the client is a robot.
>
>Can you write some pseudo-code that shows the algorithm in its
>simplest form?
>
>> I think maybe putting these 2 items together to consider it as a
>> robot is a bit complex. Let's do it from the simple point.
>>
>> If we always find there is a same IP request our website the same
>> url for many times, can I block this request at httpd level?
>
>This sounds like a job for mod_qos, mod_evasive, or mod_security.
>
>- -chris
>
>> At 2015-05-19 20:01:00, "David kerber" <dc...@verizon.net>
>> wrote:
>>> On 5/19/2015 7:53 AM, javalishixml wrote:
>>>>
>>>>
>>>>
>>>>> I doubt you're going to be able to do this in httpd, unless
>>>>> you have a very simple, straight forward way of identifying
>>>>> the robots.
>>>> Yes. I just want to have a way to block the duplicated requests
>>>> at httpd level. After all, my website has to face the the big
>>>> concurrency issue.
>>>
>>> I understand that's what you want. What we're telling you is
>>> that you probably won't be able to do that.
>>>
>>> Let me ask the question again, that Chris asked before: how do
>>> you tell that a given request is from a robot?
>>>
>>> The answer to that question will determine if you can block it
>>> with httpd.
>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> At 2015-05-19 19:35:26, "David kerber" <dc...@verizon.net>
>>>> wrote:
>>>>> On 5/19/2015 1:03 AM, javalishixml wrote:
>>>>>> Thanks a lot for your information.
>>>>>>
>>>>>>
>>>>>> This solution is based on tomcat level. If I always handle
>>>>>> this issue at java level, I'm afraid it has performance
>>>>>> issue. Because this web site afford a very big concurrency
>>>>>> access.
>>>>>>
>>>>>>
>>>>>> Taking a consideration on its basic architect
>>>>>> tomcat+apache, I think the best way to move this solution
>>>>>> from tomcat to apache. So do you have some good solution at
>>>>>> apache's configuration? I understand this is a mail list
>>>>>> for tomcat.. but just want to get any information
>>>>>
>>>>> I doubt you're going to be able to do this in httpd, unless
>>>>> you have a very simple, straight forward way of identifying
>>>>> the robots.
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>
>>>>>> At 2015-05-19 04:00:28, "Christopher Schultz"
>>>>>> <ch...@christopherschultz.net> wrote:
>> To whom it may concern,
>>
>> On 5/18/15 11:44 AM, javalishixml wrote:
>>>>>>>>> I have a website. It is built by apache + tomcat.
>>>>>>>>>
>>>>>>>>> Now we make a lottery activity at this website. But
>>>>>>>>> we find that some robots always raise the duplicated
>>>>>>>>> requests to hit this lottery activity. It causes that
>>>>>>>>> robots almost get all the awards.
>>>>>>>>>
>>>>>>>>> So we just want to block these kind of duplicated
>>>>>>>>> requests at every interval unit. For example, we set
>>>>>>>>> the interval unit is 3 seconds. The if the robot want
>>>>>>>>> to hit the lottery activity in 3 seconds, the website
>>>>>>>>> could block this action.
>>>>>>>>>
>>>>>>>>> So how to do it? I suppose if we do it at tomcat
>>>>>>>>> level, is it a very low performance? Can I do it at
>>>>>>>>> apache level? how to do it? If I could not do it
>>>>>>>>> apache level, can I do it by setting sth at tomcat?
>>
>> If you have a way to identify a "duplicate" request (e.g. using a
>> fingerprint of the request that you can check during that 3-second
>> interval), then this is conceptually very easy.
>>
>> It may not be great for performance, but you'll have to weigh that
>> against your own requirements. (For example, which is worse: poor
>> performance, or a site where only robots ever win the lottery?)
>>
>> This will not be something you can configure in Apache httpd or
>> Tomcat. This will have to be an application thing (unless you can
>> describe the fingerprint technique to some httpd module such as
>> mod_security or mod_qos and then allow it to discard duplicates).
>>
>> Back to the solution:
>>
>> 1. Take a fingerprint of the request 2. Lookup the fingerprint in a
>> database of previous requests ( fingerprint -> latest timestamp )
>> 3. If the fingerprint appears in your database and the timestamp
>> is less than 3 seconds ago, discard the request 4. Otherwise, store
>> the current timestamp and fingerprint in the databas e
>>
>> For a database, I might recommend something like memcached or
>> another in-memory-style database. An in-memory key-value store is
>> really what you are looking for. Memcached has a nice feature where
>> values can automatically time-out (e.g. they are invalid after 3
>> seconds), so you can make your application code a bit simpler
>> because you'll never have a value in the database that is not
>> valid.
>>
>> Hope that helps, -chris
>>>>>>>
>>>>>>> -----------------------------------------------------------------
>- ----
>>>>>>>
>>>>>>>
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>> For additional commands, e-mail:
>>>>>>> users-help@tomcat.apache.org
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> -------------------------------------------------------------------
>- --
>>>>>
>>>>>
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>> For additional commands, e-mail:
>>>>> users-help@tomcat.apache.org
>>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>>
>>>
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v2
>Comment: GPGTools - http://gpgtools.org
>
>iQIcBAEBCAAGBQJVWzetAAoJEBzwKT+lPKRYh1sP+wQYPO7BY9Stg2XdvzK9GDA0
>/cqRMxIZ2Thq84GzHRKcg2pyC2iN3M+LMCxXCodKdp6+Cl0DfU6H0ijKucy4yHET
>FrwhHT/7A1A7bQ4eT6IYhu1R7dtLhM0o5YRwYDDClKSfvsACTmuLOEmin40bmTeH
>uuREu7EDz1fdNgOpjpDBNw1bC4ZkyYMlhQZ3Ox/jopnxKqRCXcjUx+kZH1UktJu9
>NBBs7rQgvSXFpNtB2DPFBS6pgy97jNSgKgWqoqX0WaQwQkfU1dLIyrH6I2v4/22s
>ldCE7sZDQrRR6PUAyMQVVj36H5WHH+xlxr0zSrkdxvy5bFFqjNoCb8fRh3+J8sGC
>yd0hesq3QW+uSwkwJOg4LsDkGFHCTYNiZZMLBRULGBabFbnHCXKIRjie3h0JGSfb
>uL8diC2sNydKH7Re8WifQ8wqPvqIqkN+oakHed+oLg6BhToJvZJY2mTYuJRdb6gc
>iRYRZ1+dH78PrgwLgovRDQHrnNVQRfUTpEyQOfygBi46o5Hh1t1fIADDMMtQ+yLX
>C/Fg7JO5+vN2AoG3A1UOHUmoTGi7bAOlqp4RUwJXdN9pho++ullP8X8SfAfW52VZ
>A5mZIb3FuI2GstOAcZPIzg63m1dr69d4CY3QCQdYUu5GnmlR9ws/LHwrI0Hbl4DH
>FiXD3fsPSgRYvjCXPISV
>=PWnL
>-----END PGP SIGNATURE-----
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org
>
Re: how to block the duplicated requests?
Posted by André Warnier <aw...@ice-sa.com>.
Christopher Schultz wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> To whom it may concern,
>
> On 5/19/15 8:09 AM, javalishixml wrote:
>> Just understood you. Really appreciate for your feedback.
>>
>>
>> How do we judge it's a robot? item1: we find the request IP is
>> always the same one. item2: our page may contains several
>> keep-alive connections. But the attack connection only focus on
>> connection.
>
> Based upon the first request, how can you tell that the robot is going
> to make later keep-alive requests?
>
>> Based on these 2 items, we think the client is a robot.
>
> Can you write some pseudo-code that shows the algorithm in its
> simplest form?
>
>> I think maybe putting these 2 items together to consider it as a
>> robot is a bit complex. Let's do it from the simple point.
>>
>> If we always find there is a same IP request our website the same
>> url for many times, can I block this request at httpd level?
>
> This sounds like a job for mod_qos, mod_evasive, or mod_security.
>
> - -chris
>
+1.
Also, a quick search in Google for "apache filtering unwanted requests", gives a bunch of
results which you may want to explore.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: how to block the duplicated requests?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
To whom it may concern,
On 5/19/15 8:09 AM, javalishixml wrote:
> Just understood you. Really appreciate for your feedback.
>
>
> How do we judge it's a robot? item1: we find the request IP is
> always the same one. item2: our page may contains several
> keep-alive connections. But the attack connection only focus on
> connection.
Based upon the first request, how can you tell that the robot is going
to make later keep-alive requests?
> Based on these 2 items, we think the client is a robot.
Can you write some pseudo-code that shows the algorithm in its
simplest form?
> I think maybe putting these 2 items together to consider it as a
> robot is a bit complex. Let's do it from the simple point.
>
> If we always find there is a same IP request our website the same
> url for many times, can I block this request at httpd level?
This sounds like a job for mod_qos, mod_evasive, or mod_security.
- -chris
> At 2015-05-19 20:01:00, "David kerber" <dc...@verizon.net>
> wrote:
>> On 5/19/2015 7:53 AM, javalishixml wrote:
>>>
>>>
>>>
>>>> I doubt you're going to be able to do this in httpd, unless
>>>> you have a very simple, straight forward way of identifying
>>>> the robots.
>>> Yes. I just want to have a way to block the duplicated requests
>>> at httpd level. After all, my website has to face the the big
>>> concurrency issue.
>>
>> I understand that's what you want. What we're telling you is
>> that you probably won't be able to do that.
>>
>> Let me ask the question again, that Chris asked before: how do
>> you tell that a given request is from a robot?
>>
>> The answer to that question will determine if you can block it
>> with httpd.
>>
>>>
>>>
>>>
>>>
>>>
>>> At 2015-05-19 19:35:26, "David kerber" <dc...@verizon.net>
>>> wrote:
>>>> On 5/19/2015 1:03 AM, javalishixml wrote:
>>>>> Thanks a lot for your information.
>>>>>
>>>>>
>>>>> This solution is based on tomcat level. If I always handle
>>>>> this issue at java level, I'm afraid it has performance
>>>>> issue. Because this web site afford a very big concurrency
>>>>> access.
>>>>>
>>>>>
>>>>> Taking a consideration on its basic architect
>>>>> tomcat+apache, I think the best way to move this solution
>>>>> from tomcat to apache. So do you have some good solution at
>>>>> apache's configuration? I understand this is a mail list
>>>>> for tomcat.. but just want to get any information
>>>>
>>>> I doubt you're going to be able to do this in httpd, unless
>>>> you have a very simple, straight forward way of identifying
>>>> the robots.
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>>
>>>>> At 2015-05-19 04:00:28, "Christopher Schultz"
>>>>> <ch...@christopherschultz.net> wrote:
> To whom it may concern,
>
> On 5/18/15 11:44 AM, javalishixml wrote:
>>>>>>>> I have a website. It is built by apache + tomcat.
>>>>>>>>
>>>>>>>> Now we make a lottery activity at this website. But
>>>>>>>> we find that some robots always raise the duplicated
>>>>>>>> requests to hit this lottery activity. It causes that
>>>>>>>> robots almost get all the awards.
>>>>>>>>
>>>>>>>> So we just want to block these kind of duplicated
>>>>>>>> requests at every interval unit. For example, we set
>>>>>>>> the interval unit is 3 seconds. The if the robot want
>>>>>>>> to hit the lottery activity in 3 seconds, the website
>>>>>>>> could block this action.
>>>>>>>>
>>>>>>>> So how to do it? I suppose if we do it at tomcat
>>>>>>>> level, is it a very low performance? Can I do it at
>>>>>>>> apache level? how to do it? If I could not do it
>>>>>>>> apache level, can I do it by setting sth at tomcat?
>
> If you have a way to identify a "duplicate" request (e.g. using a
> fingerprint of the request that you can check during that 3-second
> interval), then this is conceptually very easy.
>
> It may not be great for performance, but you'll have to weigh that
> against your own requirements. (For example, which is worse: poor
> performance, or a site where only robots ever win the lottery?)
>
> This will not be something you can configure in Apache httpd or
> Tomcat. This will have to be an application thing (unless you can
> describe the fingerprint technique to some httpd module such as
> mod_security or mod_qos and then allow it to discard duplicates).
>
> Back to the solution:
>
> 1. Take a fingerprint of the request 2. Lookup the fingerprint in a
> database of previous requests ( fingerprint -> latest timestamp )
> 3. If the fingerprint appears in your database and the timestamp
> is less than 3 seconds ago, discard the request 4. Otherwise, store
> the current timestamp and fingerprint in the databas e
>
> For a database, I might recommend something like memcached or
> another in-memory-style database. An in-memory key-value store is
> really what you are looking for. Memcached has a nice feature where
> values can automatically time-out (e.g. they are invalid after 3
> seconds), so you can make your application code a bit simpler
> because you'll never have a value in the database that is not
> valid.
>
> Hope that helps, -chris
>>>>>>
>>>>>> -----------------------------------------------------------------
- ----
>>>>>>
>>>>>>
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>> For additional commands, e-mail:
>>>>>> users-help@tomcat.apache.org
>>>>>>
>>>>>
>>>>
>>>>
>>>> -------------------------------------------------------------------
- --
>>>>
>>>>
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>> For additional commands, e-mail:
>>>> users-help@tomcat.apache.org
>>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>>
>>
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: GPGTools - http://gpgtools.org
iQIcBAEBCAAGBQJVWzetAAoJEBzwKT+lPKRYh1sP+wQYPO7BY9Stg2XdvzK9GDA0
/cqRMxIZ2Thq84GzHRKcg2pyC2iN3M+LMCxXCodKdp6+Cl0DfU6H0ijKucy4yHET
FrwhHT/7A1A7bQ4eT6IYhu1R7dtLhM0o5YRwYDDClKSfvsACTmuLOEmin40bmTeH
uuREu7EDz1fdNgOpjpDBNw1bC4ZkyYMlhQZ3Ox/jopnxKqRCXcjUx+kZH1UktJu9
NBBs7rQgvSXFpNtB2DPFBS6pgy97jNSgKgWqoqX0WaQwQkfU1dLIyrH6I2v4/22s
ldCE7sZDQrRR6PUAyMQVVj36H5WHH+xlxr0zSrkdxvy5bFFqjNoCb8fRh3+J8sGC
yd0hesq3QW+uSwkwJOg4LsDkGFHCTYNiZZMLBRULGBabFbnHCXKIRjie3h0JGSfb
uL8diC2sNydKH7Re8WifQ8wqPvqIqkN+oakHed+oLg6BhToJvZJY2mTYuJRdb6gc
iRYRZ1+dH78PrgwLgovRDQHrnNVQRfUTpEyQOfygBi46o5Hh1t1fIADDMMtQ+yLX
C/Fg7JO5+vN2AoG3A1UOHUmoTGi7bAOlqp4RUwJXdN9pho++ullP8X8SfAfW52VZ
A5mZIb3FuI2GstOAcZPIzg63m1dr69d4CY3QCQdYUu5GnmlR9ws/LHwrI0Hbl4DH
FiXD3fsPSgRYvjCXPISV
=PWnL
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re:Re: how to block the duplicated requests?
Posted by javalishixml <ja...@163.com>.
Just understood you. Really appreciate for your feedback.
How do we judge it's a robot?
item1: we find the request IP is always the same one.
item2: our page may contains several keep-alive connections. But the attack connection only focus on connection.
Based on these 2 items, we think the client is a robot.
I think maybe putting these 2 items together to consider it as a robot is a bit complex. Let's do it from the simple point.
If we alway find there is a same IP request our website the same url for many times, can I block this request at httpd level?
Thanks,
At 2015-05-19 20:01:00, "David kerber" <dc...@verizon.net> wrote:
>On 5/19/2015 7:53 AM, javalishixml wrote:
>>
>>
>>
>>> I doubt you're going to be able to do this in httpd, unless you have a very simple, straight forward way of identifying the robots.
>> Yes. I just want to have a way to block the duplicated requests at httpd level. After all, my website has to face the the big concurrency issue.
>
>I understand that's what you want. What we're telling you is that you
>probably won't be able to do that.
>
>Let me ask the question again, that Chris asked before: how do you tell
>that a given request is from a robot?
>
>The answer to that question will determine if you can block it with httpd.
>
>>
>>
>>
>>
>>
>> At 2015-05-19 19:35:26, "David kerber" <dc...@verizon.net> wrote:
>>> On 5/19/2015 1:03 AM, javalishixml wrote:
>>>> Thanks a lot for your information.
>>>>
>>>>
>>>> This solution is based on tomcat level. If I always handle this issue at java level, I'm afraid it has performance issue. Because this web site afford a very big concurrency access.
>>>>
>>>>
>>>> Taking a consideration on its basic architect tomcat+apache, I think the best way to move this solution from tomcat to apache. So do you have some good solution at apache's configuration? I understand this is a mail list for tomcat.. but just want to get any information
>>>
>>> I doubt you're going to be able to do this in httpd, unless you have a
>>> very simple, straight forward way of identifying the robots.
>>>
>>>
>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>>
>>>> At 2015-05-19 04:00:28, "Christopher Schultz" <ch...@christopherschultz.net> wrote:
>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>> Hash: SHA256
>>>>>
>>>>> To whom it may concern,
>>>>>
>>>>> On 5/18/15 11:44 AM, javalishixml wrote:
>>>>>> I have a website. It is built by apache + tomcat.
>>>>>>
>>>>>> Now we make a lottery activity at this website. But we find that
>>>>>> some robots always raise the duplicated requests to hit this
>>>>>> lottery activity. It causes that robots almost get all the awards.
>>>>>>
>>>>>> So we just want to block these kind of duplicated requests at every
>>>>>> interval unit. For example, we set the interval unit is 3 seconds.
>>>>>> The if the robot want to hit the lottery activity in 3 seconds, the
>>>>>> website could block this action.
>>>>>>
>>>>>> So how to do it? I suppose if we do it at tomcat level, is it a
>>>>>> very low performance? Can I do it at apache level? how to do it? If
>>>>>> I could not do it apache level, can I do it by setting sth at
>>>>>> tomcat?
>>>>>
>>>>> If you have a way to identify a "duplicate" request (e.g. using a
>>>>> fingerprint of the request that you can check during that 3-second
>>>>> interval), then this is conceptually very easy.
>>>>>
>>>>> It may not be great for performance, but you'll have to weigh that
>>>>> against your own requirements. (For example, which is worse: poor
>>>>> performance, or a site where only robots ever win the lottery?)
>>>>>
>>>>> This will not be something you can configure in Apache httpd or
>>>>> Tomcat. This will have to be an application thing (unless you can
>>>>> describe the fingerprint technique to some httpd module such as
>>>>> mod_security or mod_qos and then allow it to discard duplicates).
>>>>>
>>>>> Back to the solution:
>>>>>
>>>>> 1. Take a fingerprint of the request
>>>>> 2. Lookup the fingerprint in a database of previous requests
>>>>> ( fingerprint -> latest timestamp )
>>>>> 3. If the fingerprint appears in your database and the timestamp is
>>>>> less than 3 seconds ago, discard the request
>>>>> 4. Otherwise, store the current timestamp and fingerprint in the databas
>>>>> e
>>>>>
>>>>> For a database, I might recommend something like memcached or another
>>>>> in-memory-style database. An in-memory key-value store is really what
>>>>> you are looking for. Memcached has a nice feature where values can
>>>>> automatically time-out (e.g. they are invalid after 3 seconds), so you
>>>>> can make your application code a bit simpler because you'll never have
>>>>> a value in the database that is not valid.
>>>>>
>>>>> Hope that helps,
>>>>> - -chris
>>>>> -----BEGIN PGP SIGNATURE-----
>>>>> Version: GnuPG v2
>>>>> Comment: GPGTools - http://gpgtools.org
>>>>>
>>>>> iQIcBAEBCAAGBQJVWkTcAAoJEBzwKT+lPKRYnW0QAIeRbfJtsTKtUZHUig9sIRre
>>>>> y1mgJkPxBXjcRTfoZkZkTPhasYzINE1mb1mTPKfPbQveH+OmpawDREWJxg/6dFeg
>>>>> af734ZRpBOAs4MtlCyTXgBUWpWka5CcpeIRYeEwx5GKPFLJfTBbGpswV3HwLaoEC
>>>>> /NqMByVfwHnixBxSTGAM2GIOyrPf+Ii1Z0JpQyDEYcZUS3Dc3IFFeHPTvzQUb1SO
>>>>> NB84fwjDT6GG/YerrlRV3GHL3WYhAw1n+tQ9cCpSWDvz8/KLUyKXqVjX5s/FbuB+
>>>>> S+krz2jzKqxG8bdeixW4s0i/9gyA/KcSSDgwmBnRwHsIUDvfF3pzk1Vq7rfGNpmQ
>>>>> L9V4brxL41H+ZMIDt2NjkVJb/UjgMnL5RpfQ1t+MdNvys/7UYav+vOv8jWqI3Mse
>>>>> AXNv46mQZAiMFzs/nsR7OIVLLxU70l+wbys4mK6u34uDip5gzxvVSaYKviqgKspx
>>>>> LT6MUHOpgmBhsiCUxjJ5odA4Q6mYhMfQxOB+6Ej8jRfKMT2uDTlwvU8gZ+/7TcUX
>>>>> JXngjQLQyjj+gAO+7jS7sWpaakV1ojy8/nFBVWH/3tWoo0YD89DJCRWxA8x8slfx
>>>>> oI9BGA0T7EwuX1CnqM90OLw7dymMQvwsTlkPAZnIvnWw3Xz29hIRazxQ7NR3AdCk
>>>>> vNXsseUzO18IJ4n+By1G
>>>>> =Q/ki
>>>>> -----END PGP SIGNATURE-----
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org
>
Re: how to block the duplicated requests?
Posted by David kerber <dc...@verizon.net>.
On 5/19/2015 7:53 AM, javalishixml wrote:
>
>
>
>> I doubt you're going to be able to do this in httpd, unless you have a very simple, straight forward way of identifying the robots.
> Yes. I just want to have a way to block the duplicated requests at httpd level. After all, my website has to face the the big concurrency issue.
I understand that's what you want. What we're telling you is that you
probably won't be able to do that.
Let me ask the question again, that Chris asked before: how do you tell
that a given request is from a robot?
The answer to that question will determine if you can block it with httpd.
>
>
>
>
>
> At 2015-05-19 19:35:26, "David kerber" <dc...@verizon.net> wrote:
>> On 5/19/2015 1:03 AM, javalishixml wrote:
>>> Thanks a lot for your information.
>>>
>>>
>>> This solution is based on tomcat level. If I always handle this issue at java level, I'm afraid it has performance issue. Because this web site afford a very big concurrency access.
>>>
>>>
>>> Taking a consideration on its basic architect tomcat+apache, I think the best way to move this solution from tomcat to apache. So do you have some good solution at apache's configuration? I understand this is a mail list for tomcat.. but just want to get any information
>>
>> I doubt you're going to be able to do this in httpd, unless you have a
>> very simple, straight forward way of identifying the robots.
>>
>>
>>
>>>
>>>
>>> Thanks,
>>>
>>>
>>> At 2015-05-19 04:00:28, "Christopher Schultz" <ch...@christopherschultz.net> wrote:
>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>> Hash: SHA256
>>>>
>>>> To whom it may concern,
>>>>
>>>> On 5/18/15 11:44 AM, javalishixml wrote:
>>>>> I have a website. It is built by apache + tomcat.
>>>>>
>>>>> Now we make a lottery activity at this website. But we find that
>>>>> some robots always raise the duplicated requests to hit this
>>>>> lottery activity. It causes that robots almost get all the awards.
>>>>>
>>>>> So we just want to block these kind of duplicated requests at every
>>>>> interval unit. For example, we set the interval unit is 3 seconds.
>>>>> The if the robot want to hit the lottery activity in 3 seconds, the
>>>>> website could block this action.
>>>>>
>>>>> So how to do it? I suppose if we do it at tomcat level, is it a
>>>>> very low performance? Can I do it at apache level? how to do it? If
>>>>> I could not do it apache level, can I do it by setting sth at
>>>>> tomcat?
>>>>
>>>> If you have a way to identify a "duplicate" request (e.g. using a
>>>> fingerprint of the request that you can check during that 3-second
>>>> interval), then this is conceptually very easy.
>>>>
>>>> It may not be great for performance, but you'll have to weigh that
>>>> against your own requirements. (For example, which is worse: poor
>>>> performance, or a site where only robots ever win the lottery?)
>>>>
>>>> This will not be something you can configure in Apache httpd or
>>>> Tomcat. This will have to be an application thing (unless you can
>>>> describe the fingerprint technique to some httpd module such as
>>>> mod_security or mod_qos and then allow it to discard duplicates).
>>>>
>>>> Back to the solution:
>>>>
>>>> 1. Take a fingerprint of the request
>>>> 2. Lookup the fingerprint in a database of previous requests
>>>> ( fingerprint -> latest timestamp )
>>>> 3. If the fingerprint appears in your database and the timestamp is
>>>> less than 3 seconds ago, discard the request
>>>> 4. Otherwise, store the current timestamp and fingerprint in the databas
>>>> e
>>>>
>>>> For a database, I might recommend something like memcached or another
>>>> in-memory-style database. An in-memory key-value store is really what
>>>> you are looking for. Memcached has a nice feature where values can
>>>> automatically time-out (e.g. they are invalid after 3 seconds), so you
>>>> can make your application code a bit simpler because you'll never have
>>>> a value in the database that is not valid.
>>>>
>>>> Hope that helps,
>>>> - -chris
>>>> -----BEGIN PGP SIGNATURE-----
>>>> Version: GnuPG v2
>>>> Comment: GPGTools - http://gpgtools.org
>>>>
>>>> iQIcBAEBCAAGBQJVWkTcAAoJEBzwKT+lPKRYnW0QAIeRbfJtsTKtUZHUig9sIRre
>>>> y1mgJkPxBXjcRTfoZkZkTPhasYzINE1mb1mTPKfPbQveH+OmpawDREWJxg/6dFeg
>>>> af734ZRpBOAs4MtlCyTXgBUWpWka5CcpeIRYeEwx5GKPFLJfTBbGpswV3HwLaoEC
>>>> /NqMByVfwHnixBxSTGAM2GIOyrPf+Ii1Z0JpQyDEYcZUS3Dc3IFFeHPTvzQUb1SO
>>>> NB84fwjDT6GG/YerrlRV3GHL3WYhAw1n+tQ9cCpSWDvz8/KLUyKXqVjX5s/FbuB+
>>>> S+krz2jzKqxG8bdeixW4s0i/9gyA/KcSSDgwmBnRwHsIUDvfF3pzk1Vq7rfGNpmQ
>>>> L9V4brxL41H+ZMIDt2NjkVJb/UjgMnL5RpfQ1t+MdNvys/7UYav+vOv8jWqI3Mse
>>>> AXNv46mQZAiMFzs/nsR7OIVLLxU70l+wbys4mK6u34uDip5gzxvVSaYKviqgKspx
>>>> LT6MUHOpgmBhsiCUxjJ5odA4Q6mYhMfQxOB+6Ej8jRfKMT2uDTlwvU8gZ+/7TcUX
>>>> JXngjQLQyjj+gAO+7jS7sWpaakV1ojy8/nFBVWH/3tWoo0YD89DJCRWxA8x8slfx
>>>> oI9BGA0T7EwuX1CnqM90OLw7dymMQvwsTlkPAZnIvnWw3Xz29hIRazxQ7NR3AdCk
>>>> vNXsseUzO18IJ4n+By1G
>>>> =Q/ki
>>>> -----END PGP SIGNATURE-----
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re:Re: how to block the duplicated requests?
Posted by javalishixml <ja...@163.com>.
>I doubt you're going to be able to do this in httpd, unless you have a very simple, straight forward way of identifying the robots.
Yes. I just want to have a way to block the duplicated requests at httpd level. After all, my website has to face the the big concurrency issue.
At 2015-05-19 19:35:26, "David kerber" <dc...@verizon.net> wrote:
>On 5/19/2015 1:03 AM, javalishixml wrote:
>> Thanks a lot for your information.
>>
>>
>> This solution is based on tomcat level. If I always handle this issue at java level, I'm afraid it has performance issue. Because this web site afford a very big concurrency access.
>>
>>
>> Taking a consideration on its basic architect tomcat+apache, I think the best way to move this solution from tomcat to apache. So do you have some good solution at apache's configuration? I understand this is a mail list for tomcat.. but just want to get any information
>
>I doubt you're going to be able to do this in httpd, unless you have a
>very simple, straight forward way of identifying the robots.
>
>
>
>>
>>
>> Thanks,
>>
>>
>> At 2015-05-19 04:00:28, "Christopher Schultz" <ch...@christopherschultz.net> wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA256
>>>
>>> To whom it may concern,
>>>
>>> On 5/18/15 11:44 AM, javalishixml wrote:
>>>> I have a website. It is built by apache + tomcat.
>>>>
>>>> Now we make a lottery activity at this website. But we find that
>>>> some robots always raise the duplicated requests to hit this
>>>> lottery activity. It causes that robots almost get all the awards.
>>>>
>>>> So we just want to block these kind of duplicated requests at every
>>>> interval unit. For example, we set the interval unit is 3 seconds.
>>>> The if the robot want to hit the lottery activity in 3 seconds, the
>>>> website could block this action.
>>>>
>>>> So how to do it? I suppose if we do it at tomcat level, is it a
>>>> very low performance? Can I do it at apache level? how to do it? If
>>>> I could not do it apache level, can I do it by setting sth at
>>>> tomcat?
>>>
>>> If you have a way to identify a "duplicate" request (e.g. using a
>>> fingerprint of the request that you can check during that 3-second
>>> interval), then this is conceptually very easy.
>>>
>>> It may not be great for performance, but you'll have to weigh that
>>> against your own requirements. (For example, which is worse: poor
>>> performance, or a site where only robots ever win the lottery?)
>>>
>>> This will not be something you can configure in Apache httpd or
>>> Tomcat. This will have to be an application thing (unless you can
>>> describe the fingerprint technique to some httpd module such as
>>> mod_security or mod_qos and then allow it to discard duplicates).
>>>
>>> Back to the solution:
>>>
>>> 1. Take a fingerprint of the request
>>> 2. Lookup the fingerprint in a database of previous requests
>>> ( fingerprint -> latest timestamp )
>>> 3. If the fingerprint appears in your database and the timestamp is
>>> less than 3 seconds ago, discard the request
>>> 4. Otherwise, store the current timestamp and fingerprint in the databas
>>> e
>>>
>>> For a database, I might recommend something like memcached or another
>>> in-memory-style database. An in-memory key-value store is really what
>>> you are looking for. Memcached has a nice feature where values can
>>> automatically time-out (e.g. they are invalid after 3 seconds), so you
>>> can make your application code a bit simpler because you'll never have
>>> a value in the database that is not valid.
>>>
>>> Hope that helps,
>>> - -chris
>>> -----BEGIN PGP SIGNATURE-----
>>> Version: GnuPG v2
>>> Comment: GPGTools - http://gpgtools.org
>>>
>>> iQIcBAEBCAAGBQJVWkTcAAoJEBzwKT+lPKRYnW0QAIeRbfJtsTKtUZHUig9sIRre
>>> y1mgJkPxBXjcRTfoZkZkTPhasYzINE1mb1mTPKfPbQveH+OmpawDREWJxg/6dFeg
>>> af734ZRpBOAs4MtlCyTXgBUWpWka5CcpeIRYeEwx5GKPFLJfTBbGpswV3HwLaoEC
>>> /NqMByVfwHnixBxSTGAM2GIOyrPf+Ii1Z0JpQyDEYcZUS3Dc3IFFeHPTvzQUb1SO
>>> NB84fwjDT6GG/YerrlRV3GHL3WYhAw1n+tQ9cCpSWDvz8/KLUyKXqVjX5s/FbuB+
>>> S+krz2jzKqxG8bdeixW4s0i/9gyA/KcSSDgwmBnRwHsIUDvfF3pzk1Vq7rfGNpmQ
>>> L9V4brxL41H+ZMIDt2NjkVJb/UjgMnL5RpfQ1t+MdNvys/7UYav+vOv8jWqI3Mse
>>> AXNv46mQZAiMFzs/nsR7OIVLLxU70l+wbys4mK6u34uDip5gzxvVSaYKviqgKspx
>>> LT6MUHOpgmBhsiCUxjJ5odA4Q6mYhMfQxOB+6Ej8jRfKMT2uDTlwvU8gZ+/7TcUX
>>> JXngjQLQyjj+gAO+7jS7sWpaakV1ojy8/nFBVWH/3tWoo0YD89DJCRWxA8x8slfx
>>> oI9BGA0T7EwuX1CnqM90OLw7dymMQvwsTlkPAZnIvnWw3Xz29hIRazxQ7NR3AdCk
>>> vNXsseUzO18IJ4n+By1G
>>> =Q/ki
>>> -----END PGP SIGNATURE-----
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org
>
Re: how to block the duplicated requests?
Posted by David kerber <dc...@verizon.net>.
On 5/19/2015 1:03 AM, javalishixml wrote:
> Thanks a lot for your information.
>
>
> This solution is based on tomcat level. If I always handle this issue at java level, I'm afraid it has performance issue. Because this web site afford a very big concurrency access.
>
>
> Taking a consideration on its basic architect tomcat+apache, I think the best way to move this solution from tomcat to apache. So do you have some good solution at apache's configuration? I understand this is a mail list for tomcat.. but just want to get any information
I doubt you're going to be able to do this in httpd, unless you have a
very simple, straight forward way of identifying the robots.
>
>
> Thanks,
>
>
> At 2015-05-19 04:00:28, "Christopher Schultz" <ch...@christopherschultz.net> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>> To whom it may concern,
>>
>> On 5/18/15 11:44 AM, javalishixml wrote:
>>> I have a website. It is built by apache + tomcat.
>>>
>>> Now we make a lottery activity at this website. But we find that
>>> some robots always raise the duplicated requests to hit this
>>> lottery activity. It causes that robots almost get all the awards.
>>>
>>> So we just want to block these kind of duplicated requests at every
>>> interval unit. For example, we set the interval unit is 3 seconds.
>>> The if the robot want to hit the lottery activity in 3 seconds, the
>>> website could block this action.
>>>
>>> So how to do it? I suppose if we do it at tomcat level, is it a
>>> very low performance? Can I do it at apache level? how to do it? If
>>> I could not do it apache level, can I do it by setting sth at
>>> tomcat?
>>
>> If you have a way to identify a "duplicate" request (e.g. using a
>> fingerprint of the request that you can check during that 3-second
>> interval), then this is conceptually very easy.
>>
>> It may not be great for performance, but you'll have to weigh that
>> against your own requirements. (For example, which is worse: poor
>> performance, or a site where only robots ever win the lottery?)
>>
>> This will not be something you can configure in Apache httpd or
>> Tomcat. This will have to be an application thing (unless you can
>> describe the fingerprint technique to some httpd module such as
>> mod_security or mod_qos and then allow it to discard duplicates).
>>
>> Back to the solution:
>>
>> 1. Take a fingerprint of the request
>> 2. Lookup the fingerprint in a database of previous requests
>> ( fingerprint -> latest timestamp )
>> 3. If the fingerprint appears in your database and the timestamp is
>> less than 3 seconds ago, discard the request
>> 4. Otherwise, store the current timestamp and fingerprint in the databas
>> e
>>
>> For a database, I might recommend something like memcached or another
>> in-memory-style database. An in-memory key-value store is really what
>> you are looking for. Memcached has a nice feature where values can
>> automatically time-out (e.g. they are invalid after 3 seconds), so you
>> can make your application code a bit simpler because you'll never have
>> a value in the database that is not valid.
>>
>> Hope that helps,
>> - -chris
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v2
>> Comment: GPGTools - http://gpgtools.org
>>
>> iQIcBAEBCAAGBQJVWkTcAAoJEBzwKT+lPKRYnW0QAIeRbfJtsTKtUZHUig9sIRre
>> y1mgJkPxBXjcRTfoZkZkTPhasYzINE1mb1mTPKfPbQveH+OmpawDREWJxg/6dFeg
>> af734ZRpBOAs4MtlCyTXgBUWpWka5CcpeIRYeEwx5GKPFLJfTBbGpswV3HwLaoEC
>> /NqMByVfwHnixBxSTGAM2GIOyrPf+Ii1Z0JpQyDEYcZUS3Dc3IFFeHPTvzQUb1SO
>> NB84fwjDT6GG/YerrlRV3GHL3WYhAw1n+tQ9cCpSWDvz8/KLUyKXqVjX5s/FbuB+
>> S+krz2jzKqxG8bdeixW4s0i/9gyA/KcSSDgwmBnRwHsIUDvfF3pzk1Vq7rfGNpmQ
>> L9V4brxL41H+ZMIDt2NjkVJb/UjgMnL5RpfQ1t+MdNvys/7UYav+vOv8jWqI3Mse
>> AXNv46mQZAiMFzs/nsR7OIVLLxU70l+wbys4mK6u34uDip5gzxvVSaYKviqgKspx
>> LT6MUHOpgmBhsiCUxjJ5odA4Q6mYhMfQxOB+6Ej8jRfKMT2uDTlwvU8gZ+/7TcUX
>> JXngjQLQyjj+gAO+7jS7sWpaakV1ojy8/nFBVWH/3tWoo0YD89DJCRWxA8x8slfx
>> oI9BGA0T7EwuX1CnqM90OLw7dymMQvwsTlkPAZnIvnWw3Xz29hIRazxQ7NR3AdCk
>> vNXsseUzO18IJ4n+By1G
>> =Q/ki
>> -----END PGP SIGNATURE-----
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re:Re: how to block the duplicated requests?
Posted by javalishixml <ja...@163.com>.
Thanks a lot for your information.
This solution is based on tomcat level. If I always handle this issue at java level, I'm afraid it has performance issue. Because this web site afford a very big concurrency access.
Taking a consideration on its basic architect tomcat+apache, I think the best way to move this solution from tomcat to apache. So do you have some good solution at apache's configuration? I understand this is a mail list for tomcat.. but just want to get any information
Thanks,
At 2015-05-19 04:00:28, "Christopher Schultz" <ch...@christopherschultz.net> wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA256
>
>To whom it may concern,
>
>On 5/18/15 11:44 AM, javalishixml wrote:
>> I have a website. It is built by apache + tomcat.
>>
>> Now we make a lottery activity at this website. But we find that
>> some robots always raise the duplicated requests to hit this
>> lottery activity. It causes that robots almost get all the awards.
>>
>> So we just want to block these kind of duplicated requests at every
>> interval unit. For example, we set the interval unit is 3 seconds.
>> The if the robot want to hit the lottery activity in 3 seconds, the
>> website could block this action.
>>
>> So how to do it? I suppose if we do it at tomcat level, is it a
>> very low performance? Can I do it at apache level? how to do it? If
>> I could not do it apache level, can I do it by setting sth at
>> tomcat?
>
>If you have a way to identify a "duplicate" request (e.g. using a
>fingerprint of the request that you can check during that 3-second
>interval), then this is conceptually very easy.
>
>It may not be great for performance, but you'll have to weigh that
>against your own requirements. (For example, which is worse: poor
>performance, or a site where only robots ever win the lottery?)
>
>This will not be something you can configure in Apache httpd or
>Tomcat. This will have to be an application thing (unless you can
>describe the fingerprint technique to some httpd module such as
>mod_security or mod_qos and then allow it to discard duplicates).
>
>Back to the solution:
>
>1. Take a fingerprint of the request
>2. Lookup the fingerprint in a database of previous requests
> ( fingerprint -> latest timestamp )
>3. If the fingerprint appears in your database and the timestamp is
>less than 3 seconds ago, discard the request
>4. Otherwise, store the current timestamp and fingerprint in the databas
>e
>
>For a database, I might recommend something like memcached or another
>in-memory-style database. An in-memory key-value store is really what
>you are looking for. Memcached has a nice feature where values can
>automatically time-out (e.g. they are invalid after 3 seconds), so you
>can make your application code a bit simpler because you'll never have
>a value in the database that is not valid.
>
>Hope that helps,
>- -chris
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v2
>Comment: GPGTools - http://gpgtools.org
>
>iQIcBAEBCAAGBQJVWkTcAAoJEBzwKT+lPKRYnW0QAIeRbfJtsTKtUZHUig9sIRre
>y1mgJkPxBXjcRTfoZkZkTPhasYzINE1mb1mTPKfPbQveH+OmpawDREWJxg/6dFeg
>af734ZRpBOAs4MtlCyTXgBUWpWka5CcpeIRYeEwx5GKPFLJfTBbGpswV3HwLaoEC
>/NqMByVfwHnixBxSTGAM2GIOyrPf+Ii1Z0JpQyDEYcZUS3Dc3IFFeHPTvzQUb1SO
>NB84fwjDT6GG/YerrlRV3GHL3WYhAw1n+tQ9cCpSWDvz8/KLUyKXqVjX5s/FbuB+
>S+krz2jzKqxG8bdeixW4s0i/9gyA/KcSSDgwmBnRwHsIUDvfF3pzk1Vq7rfGNpmQ
>L9V4brxL41H+ZMIDt2NjkVJb/UjgMnL5RpfQ1t+MdNvys/7UYav+vOv8jWqI3Mse
>AXNv46mQZAiMFzs/nsR7OIVLLxU70l+wbys4mK6u34uDip5gzxvVSaYKviqgKspx
>LT6MUHOpgmBhsiCUxjJ5odA4Q6mYhMfQxOB+6Ej8jRfKMT2uDTlwvU8gZ+/7TcUX
>JXngjQLQyjj+gAO+7jS7sWpaakV1ojy8/nFBVWH/3tWoo0YD89DJCRWxA8x8slfx
>oI9BGA0T7EwuX1CnqM90OLw7dymMQvwsTlkPAZnIvnWw3Xz29hIRazxQ7NR3AdCk
>vNXsseUzO18IJ4n+By1G
>=Q/ki
>-----END PGP SIGNATURE-----
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org
>
Re: how to block the duplicated requests?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
To whom it may concern,
On 5/18/15 11:44 AM, javalishixml wrote:
> I have a website. It is built by apache + tomcat.
>
> Now we make a lottery activity at this website. But we find that
> some robots always raise the duplicated requests to hit this
> lottery activity. It causes that robots almost get all the awards.
>
> So we just want to block these kind of duplicated requests at every
> interval unit. For example, we set the interval unit is 3 seconds.
> The if the robot want to hit the lottery activity in 3 seconds, the
> website could block this action.
>
> So how to do it? I suppose if we do it at tomcat level, is it a
> very low performance? Can I do it at apache level? how to do it? If
> I could not do it apache level, can I do it by setting sth at
> tomcat?
If you have a way to identify a "duplicate" request (e.g. using a
fingerprint of the request that you can check during that 3-second
interval), then this is conceptually very easy.
It may not be great for performance, but you'll have to weigh that
against your own requirements. (For example, which is worse: poor
performance, or a site where only robots ever win the lottery?)
This will not be something you can configure in Apache httpd or
Tomcat. This will have to be an application thing (unless you can
describe the fingerprint technique to some httpd module such as
mod_security or mod_qos and then allow it to discard duplicates).
Back to the solution:
1. Take a fingerprint of the request
2. Lookup the fingerprint in a database of previous requests
( fingerprint -> latest timestamp )
3. If the fingerprint appears in your database and the timestamp is
less than 3 seconds ago, discard the request
4. Otherwise, store the current timestamp and fingerprint in the databas
e
For a database, I might recommend something like memcached or another
in-memory-style database. An in-memory key-value store is really what
you are looking for. Memcached has a nice feature where values can
automatically time-out (e.g. they are invalid after 3 seconds), so you
can make your application code a bit simpler because you'll never have
a value in the database that is not valid.
Hope that helps,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: GPGTools - http://gpgtools.org
iQIcBAEBCAAGBQJVWkTcAAoJEBzwKT+lPKRYnW0QAIeRbfJtsTKtUZHUig9sIRre
y1mgJkPxBXjcRTfoZkZkTPhasYzINE1mb1mTPKfPbQveH+OmpawDREWJxg/6dFeg
af734ZRpBOAs4MtlCyTXgBUWpWka5CcpeIRYeEwx5GKPFLJfTBbGpswV3HwLaoEC
/NqMByVfwHnixBxSTGAM2GIOyrPf+Ii1Z0JpQyDEYcZUS3Dc3IFFeHPTvzQUb1SO
NB84fwjDT6GG/YerrlRV3GHL3WYhAw1n+tQ9cCpSWDvz8/KLUyKXqVjX5s/FbuB+
S+krz2jzKqxG8bdeixW4s0i/9gyA/KcSSDgwmBnRwHsIUDvfF3pzk1Vq7rfGNpmQ
L9V4brxL41H+ZMIDt2NjkVJb/UjgMnL5RpfQ1t+MdNvys/7UYav+vOv8jWqI3Mse
AXNv46mQZAiMFzs/nsR7OIVLLxU70l+wbys4mK6u34uDip5gzxvVSaYKviqgKspx
LT6MUHOpgmBhsiCUxjJ5odA4Q6mYhMfQxOB+6Ej8jRfKMT2uDTlwvU8gZ+/7TcUX
JXngjQLQyjj+gAO+7jS7sWpaakV1ojy8/nFBVWH/3tWoo0YD89DJCRWxA8x8slfx
oI9BGA0T7EwuX1CnqM90OLw7dymMQvwsTlkPAZnIvnWw3Xz29hIRazxQ7NR3AdCk
vNXsseUzO18IJ4n+By1G
=Q/ki
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re:Re: how to block the duplicated requests?
Posted by javalishixml <ja...@163.com>.
We find some client are always automatically fresh our lottery activity
At 2015-05-18 23:56:51, "David kerber" <dc...@verizon.net> wrote:
>How would you tell that a request is from a robot?
>
>
>On 5/18/2015 11:44 AM, javalishixml wrote:
>> Hi,
>>
>> I have a website. It is built by apache + tomcat.
>>
>> Now we make a lottery activity at this website. But we find that some robots always raise the duplicated requests to hit this lottery activity. It causes that robots almost get all the awards.
>>
>> So we just want to block these kind of duplicated requests at every interval unit.
>> For example, we set the interval unit is 3 seconds. The if the robot want to hit the lottery activity in 3 seconds, the website could block this action.
>>
>> So how to do it? I suppose if we do it at tomcat level, is it a very low performance? Can I do it at apache level? how to do it?
>> If I could not do it apache level, can I do it by setting sth at tomcat?
>>
>> Thanks in advance,
>> Java Coder
>>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org
>
Re: how to block the duplicated requests?
Posted by David kerber <dc...@verizon.net>.
How would you tell that a request is from a robot?
On 5/18/2015 11:44 AM, javalishixml wrote:
> Hi,
>
> I have a website. It is built by apache + tomcat.
>
> Now we make a lottery activity at this website. But we find that some robots always raise the duplicated requests to hit this lottery activity. It causes that robots almost get all the awards.
>
> So we just want to block these kind of duplicated requests at every interval unit.
> For example, we set the interval unit is 3 seconds. The if the robot want to hit the lottery activity in 3 seconds, the website could block this action.
>
> So how to do it? I suppose if we do it at tomcat level, is it a very low performance? Can I do it at apache level? how to do it?
> If I could not do it apache level, can I do it by setting sth at tomcat?
>
> Thanks in advance,
> Java Coder
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org