You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Dave Challis <da...@aistemos.com> on 2018/06/26 14:45:19 UTC
Web console responsiveness under heavy load
Are there any recommended Drill settings to configure in order to ensure
that the web console (running on 8047) remains responsive even under heavy
load?
Currently, if I execute a large/complex query (that e.g. takes 5m to
complete), all queries to 8047 just block until the query completes.
I'd like to use it to keep an eye on the query (on the profiles) page.
Re: Web console responsiveness under heavy load
Posted by Dave Challis <da...@aistemos.com>.
Yup, running the query through the web UI. If I avoid doing that, would the
UI be more responsive?
Drillbit settings are:
DRILL_HEAP=8G
DRILL_MAX_DIRECT_MEMORY=16G
DRILLBIT_CODE_CACHE_SIZE=2G
The query itself uses a peak of ~3.5Gb memory while it's running.
On 30 June 2018 at 08:18, Kunal Khatua <ku...@apache.org> wrote:
> Are you running the query through the WebUI? What are the memory settings
> of your Drillbit?
>
> On 6/26/2018 7:45:30 AM, Dave Challis <da...@aistemos.com> wrote:
> Are there any recommended Drill settings to configure in order to ensure
> that the web console (running on 8047) remains responsive even under heavy
> load?
>
> Currently, if I execute a large/complex query (that e.g. takes 5m to
> complete), all queries to 8047 just block until the query completes.
>
> I'd like to use it to keep an eye on the query (on the profiles) page.
>
Re: Web console responsiveness under heavy load
Posted by Dave Challis <da...@aistemos.com>.
Many thanks, that's good to know, will move to doing that.
On 3 July 2018 at 05:29, Kunal Khatua <ku...@apache.org> wrote:
> Yes. The reason you see the sluggishness is because the embedded webserver
> within the Drillbit also acts as a proxy client, which receives the entire
> result set, before converting it into a JSON response for your browser.
>
> This can lead to rather high memory requirements.
>
> I'd recommend using tools like DBeaver (which has a nice feature of
> automatically downloading the latest JDBC drivers) or Squirrel.
>
> You can then use the WebUI to monitor the queries in flight, etc.
>
> On 6/30/2018 12:18:23 AM, Kunal Khatua <ku...@apache.org> wrote:
> Are you running the query through the WebUI? What are the memory settings
> of your Drillbit?
>
> On 6/26/2018 7:45:30 AM, Dave Challis <da...@aistemos.com> wrote:
> Are there any recommended Drill settings to configure in order to ensure
> that the web console (running on 8047) remains responsive even under heavy
> load?
>
> Currently, if I execute a large/complex query (that e.g. takes 5m to
> complete), all queries to 8047 just block until the query completes.
>
> I'd like to use it to keep an eye on the query (on the profiles) page.
>
Re: Web console responsiveness under heavy load
Posted by Kunal Khatua <ku...@apache.org>.
Yes. The reason you see the sluggishness is because the embedded webserver within the Drillbit also acts as a proxy client, which receives the entire result set, before converting it into a JSON response for your browser.
This can lead to rather high memory requirements.
I'd recommend using tools like DBeaver (which has a nice feature of automatically downloading the latest JDBC drivers) or Squirrel.
You can then use the WebUI to monitor the queries in flight, etc.
On 6/30/2018 12:18:23 AM, Kunal Khatua <ku...@apache.org> wrote:
Are you running the query through the WebUI? What are the memory settings of your Drillbit?
On 6/26/2018 7:45:30 AM, Dave Challis <da...@aistemos.com> wrote:
Are there any recommended Drill settings to configure in order to ensure
that the web console (running on 8047) remains responsive even under heavy
load?
Currently, if I execute a large/complex query (that e.g. takes 5m to
complete), all queries to 8047 just block until the query completes.
I'd like to use it to keep an eye on the query (on the profiles) page.
Re: Web console responsiveness under heavy load
Posted by Kunal Khatua <ku...@apache.org>.
Are you running the query through the WebUI? What are the memory settings of your Drillbit?
On 6/26/2018 7:45:30 AM, Dave Challis <da...@aistemos.com> wrote:
Are there any recommended Drill settings to configure in order to ensure
that the web console (running on 8047) remains responsive even under heavy
load?
Currently, if I execute a large/complex query (that e.g. takes 5m to
complete), all queries to 8047 just block until the query completes.
I'd like to use it to keep an eye on the query (on the profiles) page.