You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@druid.apache.org by Samarth Jain <sa...@gmail.com> on 2019/06/06 16:22:19 UTC
Service protection and Druid SQL
Hello Druid devs,
With Druid sql slowly becoming the de-facto way of issuing Druid queries,
chances of user queries unintentionally doing too much and potentially
putting a lot of load on the the Druid cluster goes up. One such case being
users forgetting to limit the amount of data they are scanning by
forgetting to add appropriate filter on the __time column.
Does it make sense to start investing in service protection features which
provides operators ways to throttle/block such queries from running.
At my work, for example, we are enabling druid sql on our platform. And to
make sure users get a good experience, I am adding a check to block queries
if the "intervals" part of query is unbounded.
- Samarth
Re: Service protection and Druid SQL
Posted by Samarth Jain <sa...@gmail.com>.
Thanks for the pointer! I had no idea that there was already an existing
config to fail queries without time filter.
On Thu, Jun 6, 2019 at 12:43 PM restful blue <ax...@gmail.com> wrote:
> It looks like such configurations already exists
> For example, there is druid.sql.planner.requireTimeCondition which makes
> it necessary for query to have a time filter.
> Also there is druid.server.http.maxScatterGatherBytes which is restricting
> max bytes gathered from historicals and
> druid.server.http.defaultQueryTimeout for restricting maximum query time
>
>
>
> От: Jihoon Son
> Отправлено: 6 июня 2019 г. в 22:18
> Кому: dev@druid.apache.org
> Тема: Re: Service protection and Druid SQL
>
> +1, I think it's a good idea.
>
> Jihoon
>
> On Thu, Jun 6, 2019 at 9:22 AM Samarth Jain <sa...@gmail.com>
> wrote:
>
> > Hello Druid devs,
> >
> > With Druid sql slowly becoming the de-facto way of issuing Druid queries,
> > chances of user queries unintentionally doing too much and potentially
> > putting a lot of load on the the Druid cluster goes up. One such case
> being
> > users forgetting to limit the amount of data they are scanning by
> > forgetting to add appropriate filter on the __time column.
> >
> > Does it make sense to start investing in service protection features
> which
> > provides operators ways to throttle/block such queries from running.
> >
> > At my work, for example, we are enabling druid sql on our platform. And
> to
> > make sure users get a good experience, I am adding a check to block
> queries
> > if the "intervals" part of query is unbounded.
> >
> > - Samarth
> >
>
>
RE: Service protection and Druid SQL
Posted by restful blue <ax...@gmail.com>.
It looks like such configurations already exists
For example, there is druid.sql.planner.requireTimeCondition which makes it necessary for query to have a time filter.
Also there is druid.server.http.maxScatterGatherBytes which is restricting max bytes gathered from historicals and druid.server.http.defaultQueryTimeout for restricting maximum query time
От: Jihoon Son
Отправлено: 6 июня 2019 г. в 22:18
Кому: dev@druid.apache.org
Тема: Re: Service protection and Druid SQL
+1, I think it's a good idea.
Jihoon
On Thu, Jun 6, 2019 at 9:22 AM Samarth Jain <sa...@gmail.com> wrote:
> Hello Druid devs,
>
> With Druid sql slowly becoming the de-facto way of issuing Druid queries,
> chances of user queries unintentionally doing too much and potentially
> putting a lot of load on the the Druid cluster goes up. One such case being
> users forgetting to limit the amount of data they are scanning by
> forgetting to add appropriate filter on the __time column.
>
> Does it make sense to start investing in service protection features which
> provides operators ways to throttle/block such queries from running.
>
> At my work, for example, we are enabling druid sql on our platform. And to
> make sure users get a good experience, I am adding a check to block queries
> if the "intervals" part of query is unbounded.
>
> - Samarth
>
Re: Service protection and Druid SQL
Posted by Jihoon Son <ji...@apache.org>.
+1, I think it's a good idea.
Jihoon
On Thu, Jun 6, 2019 at 9:22 AM Samarth Jain <sa...@gmail.com> wrote:
> Hello Druid devs,
>
> With Druid sql slowly becoming the de-facto way of issuing Druid queries,
> chances of user queries unintentionally doing too much and potentially
> putting a lot of load on the the Druid cluster goes up. One such case being
> users forgetting to limit the amount of data they are scanning by
> forgetting to add appropriate filter on the __time column.
>
> Does it make sense to start investing in service protection features which
> provides operators ways to throttle/block such queries from running.
>
> At my work, for example, we are enabling druid sql on our platform. And to
> make sure users get a good experience, I am adding a check to block queries
> if the "intervals" part of query is unbounded.
>
> - Samarth
>