You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hawq.apache.org by Ruilong Huo <rh...@pivotal.io> on 2016/12/11 00:38:54 UTC
Turning off DoS protection in hawq
Hi hawq community,
Anyone know that why we turn off DoS protection in hawq by setting
net.ipv4.tcp_syncookies to off in /etc/sysctl.conf? Any other reason from
hawq perspective other than below two more from operating system
perspective?
1) increase the number of concurrent tcp clients
2) reduce cpu overhead for creating and processing the syncookies. Though
the overhead is tiny
BTW: HAWQ turn off DoS protection
<http://hdb.docs.pivotal.io/210/hdb/install/install-cli.html> while
Greenplum Database enable DoS protection
<http://gpdb.docs.pivotal.io/4320/install_guide/prep_os_install_gpdb.html> by
default.
Best regards,
Ruilong Huo
Re: Turning off DoS protection in hawq
Posted by Paul Guo <pa...@gmail.com>.
By the way, just checked the doc. At least there is no need to set the
kernel tunable below
since the default value is that.
net.ipv4.ip_forward = 0
2016-12-11 20:13 GMT+08:00 Ruilong Huo <rh...@pivotal.io>:
> Thanks Zhanwei for the clarification.
>
> It is worthwhile to understand the highest tcp connection frequency that
> would cause OS consider the connection as DoS.
> We can then evaluate at which degree of concurrency (partitioned tables,
> column storage, etc) in hawq would cause that problem.
>
> BTW, here is some detail mechanism of protecting DoS with sync cookies:
> http://security.stackexchange.com/questions/20904/using-syn-
> cookies-to-perform-a-dos-attack
> .
>
> Best regards,
> Ruilong Huo
>
> On Sun, Dec 11, 2016 at 10:09 AM, Zhanwei Wang <wa...@apache.org> wrote:
>
> > HI Ruilong
> >
> >
> > According the pervious stress test, under very high workload, OS will
> > consider the connection from HAWQ to HDFS as Dos and reject the connect
> > request and then HAWQ query will fail.
> >
> > After disabling CO table in HAWQ, we have significantly reduced the file
> > write workload, I think it is time to reconsider this OS setting.
> >
> >
> >
> > Best Regards
> >
> > Zhanwei Wang
> > wangzw@apache.org
> >
> >
> >
> > > 在 2016年12月11日，上午8:38，Ruilong Huo <rh...@pivotal.io> 写道：
> > >
> > > Hi hawq community,
> > >
> > > Anyone know that why we turn off DoS protection in hawq by setting
> > > net.ipv4.tcp_syncookies to off in /etc/sysctl.conf? Any other reason
> from
> > > hawq perspective other than below two more from operating system
> > > perspective?
> > >
> > > 1) increase the number of concurrent tcp clients
> > >
> > > 2) reduce cpu overhead for creating and processing the syncookies.
> Though
> > > the overhead is tiny
> > >
> > > BTW: HAWQ turn off DoS protection
> > > <http://hdb.docs.pivotal.io/210/hdb/install/install-cli.html> while
> > > Greenplum Database enable DoS protection
> > > <http://gpdb.docs.pivotal.io/4320/install_guide/prep_os_
> > install_gpdb.html> by
> > > default.
> > >
> > > Best regards,
> > > Ruilong Huo
> >
> >
>
Re: Turning off DoS protection in hawq
Posted by Ruilong Huo <rh...@pivotal.io>.
Thanks Zhanwei for the clarification.
It is worthwhile to understand the highest tcp connection frequency that
would cause OS consider the connection as DoS.
We can then evaluate at which degree of concurrency (partitioned tables,
column storage, etc) in hawq would cause that problem.
BTW, here is some detail mechanism of protecting DoS with sync cookies:
http://security.stackexchange.com/questions/20904/using-syn-cookies-to-perform-a-dos-attack
.
Best regards,
Ruilong Huo
On Sun, Dec 11, 2016 at 10:09 AM, Zhanwei Wang <wa...@apache.org> wrote:
> HI Ruilong
>
>
> According the pervious stress test, under very high workload, OS will
> consider the connection from HAWQ to HDFS as Dos and reject the connect
> request and then HAWQ query will fail.
>
> After disabling CO table in HAWQ, we have significantly reduced the file
> write workload, I think it is time to reconsider this OS setting.
>
>
>
> Best Regards
>
> Zhanwei Wang
> wangzw@apache.org
>
>
>
> > 在 2016年12月11日，上午8:38，Ruilong Huo <rh...@pivotal.io> 写道：
> >
> > Hi hawq community,
> >
> > Anyone know that why we turn off DoS protection in hawq by setting
> > net.ipv4.tcp_syncookies to off in /etc/sysctl.conf? Any other reason from
> > hawq perspective other than below two more from operating system
> > perspective?
> >
> > 1) increase the number of concurrent tcp clients
> >
> > 2) reduce cpu overhead for creating and processing the syncookies. Though
> > the overhead is tiny
> >
> > BTW: HAWQ turn off DoS protection
> > <http://hdb.docs.pivotal.io/210/hdb/install/install-cli.html> while
> > Greenplum Database enable DoS protection
> > <http://gpdb.docs.pivotal.io/4320/install_guide/prep_os_
> install_gpdb.html> by
> > default.
> >
> > Best regards,
> > Ruilong Huo
>
>
Re: Turning off DoS protection in hawq
Posted by Zhanwei Wang <wa...@apache.org>.
HI Ruilong
According the pervious stress test, under very high workload, OS will consider the connection from HAWQ to HDFS as Dos and reject the connect request and then HAWQ query will fail.
After disabling CO table in HAWQ, we have significantly reduced the file write workload, I think it is time to reconsider this OS setting.
Best Regards
Zhanwei Wang
wangzw@apache.org
> 在 2016年12月11日，上午8:38，Ruilong Huo <rh...@pivotal.io> 写道：
>
> Hi hawq community,
>
> Anyone know that why we turn off DoS protection in hawq by setting
> net.ipv4.tcp_syncookies to off in /etc/sysctl.conf? Any other reason from
> hawq perspective other than below two more from operating system
> perspective?
>
> 1) increase the number of concurrent tcp clients
>
> 2) reduce cpu overhead for creating and processing the syncookies. Though
> the overhead is tiny
>
> BTW: HAWQ turn off DoS protection
> <http://hdb.docs.pivotal.io/210/hdb/install/install-cli.html> while
> Greenplum Database enable DoS protection
> <http://gpdb.docs.pivotal.io/4320/install_guide/prep_os_install_gpdb.html> by
> default.
>
> Best regards,
> Ruilong Huo