You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by ja...@accenture.com on 2017/05/03 12:28:48 UTC
Drill query are stuck in ENQUEUED mode
Hi,
I am having queries in which I am fetching just 1 row from the parquet file using LIMIT 1, and even these queries are stuck in ENQUEUED state in DRILL.
I am using exec.queue.enable = true and have default settings for the rest.
Can you help me out in this.
Regards,
Jasbir Singh
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com
Re: Drill query are stuck in ENQUEUED mode
Posted by Paul Rogers <pr...@mapr.com>.
Hi All,
Looks like queues are enabled. The numbers used for the queue are cost which are only very lightly related to your row count. Cost is in what I call “planning units”: units that make sense to produce good plans, but which don’t directly map to real-world unit such as rows, memory or time.
When queuing is enabled, queries start in the Foreman in the Enqueued state. During that time, we do query planning and parallelization.
Once planning is complete, the Foreman waits on ZooKeeper to get a run slot. ZooKeeper implements a distributed semaphore. I like to think of it as a “take a number” schema like in old-time delis: Each query takes a number and starts to run when its number comes up.
If queries are stuck in the Enqueued state, it could therefore be due to either 1) planning is taking a long time, or 2) the query is waiting for its slot from ZK. A jstack will tell us which is the case.
BTW: The default queue sizes are probably too large for all but the largest clusters. You will want to adjust the numbers for your load to best make use of the cores and memory you have available.
Given your number of cores and memory, running 110 concurrent queries is probably asking too much of your hardware. Try a much smaller number: maybe 2 and 5. Remember that all queries will be parallelized to 70% of your CPU count (20 in your case) and each will launch multiple threads per CPU. Running even 7 queries, with a couple of fragments each, will cause Drill to run 7 * 2 * 20 = 280 threads on your 24 cores which may still be too large.
So, it may be that the reason queries are stuck in Enqueued state is that your cluster is overloaded: queries are competing for resources, running slowly and blocking incoming queries.
- Paul
> On May 4, 2017, at 5:52 AM, jasbir.sing@accenture.com wrote:
>
> Hi,
>
> Thanks for the updates.
>
> Just want to update you that we are already following this parameter. Value of below parameters are -
> 1. exec.queue.enable = TRUE
> 2. exec.queue.large = 10 (Default)
> 3. exec.queue.small = 100 (Default)
> 4. exec.queue.threshold = 30000000(Default)
> Exec.queue.timeout_millis = 300000(Default)
>
> Even after following queries are getting stuck. As in our case parquet record count is less than 1000, do I still need to set this threshold or I need to change this value?
> As it doesn't tells me on how much record count it decides to queue the query in SMALL section. What is that record count?
>
> Description of my cluster is -
> 1. Nodes = 1 - Local mode
> 2. Memory = 64 GB
> 3. Cores = 24
> 4. It's a linux box
>
> Please do let me know how to resolve query stuck issue as it is hampering products performance.
>
> Regards,
> Jasbir Singh
>
> -----Original Message-----
> From: Kunal Khatua [mailto:kkhatua@mapr.com]
> Sent: Wednesday, May 03, 2017 10:43 PM
> To: user@drill.apache.org
> Cc: Kothari, Maneesh <ma...@accenture.com>; Kumar, H. P. <h....@accenture.com>
> Subject: Re: Drill query are stuck in ENQUEUED mode
>
> You could try experimenting with these parameters:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__drill.apache.org_docs_enabling-2Dquery-2Dqueuing_&d=DwIFAg&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=7scIIjM0jY9x3fjvY6a_yERLxMA2NwA8l0DnuyrL6yA&m=0506fa1JfLmFVj-y5MXeRZgSyJyeYwWoQYArxFhAFpc&s=Dr7zjb2Guf7ZBOsBVweE4jbrxUv_RRxAMGk9NBqL5Zo&e= when you have Queuing enabled.
>
>
> However, in a false state, there should be no queuing (i.e. no ENQUEUED mode).
>
>
> Can you provide a description of your cluster? (e.g. nodes, cores, memory... is it a VM, etc)
>
>
> One way to understand why a query is 'stuck' with queuing disabled would be to use JStack and identify the state of the foreman and fragment threads executing that query.
>
>
> For this, you first need to know what is the query ID (the part that follows the "https://urldefense.proofpoint.com/v2/url?u=http-3A__hostname-3A8047_profiles_&d=DwIFAg&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=7scIIjM0jY9x3fjvY6a_yERLxMA2NwA8l0DnuyrL6yA&m=0506fa1JfLmFVj-y5MXeRZgSyJyeYwWoQYArxFhAFpc&s=WL1ut57ugpifNnM-IVN1E-NzFBOigrMvDwHw4Uj3etg&e= " part of the query's web UI page). Use that as a reference to identify the corresponding threads.
>
>
> Usually, a large number of threads will carry this as a part of their thread names. However, if it is stuck, my hunch is that only 1 fragment thread is stuck, while the others completed. So you should be able to find that easily.
>
>
> Kunal Khatua
>
> Engineering
>
> [1490734684477_mapr.png]
>
> www.mapr.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.mapr.com_&d=DwIFAg&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=7scIIjM0jY9x3fjvY6a_yERLxMA2NwA8l0DnuyrL6yA&m=0506fa1JfLmFVj-y5MXeRZgSyJyeYwWoQYArxFhAFpc&s=sYU_DLIUmwFTX0OxodY-x06c7dD7RrXjNex3CQFiQVk&e= >
>
> ________________________________
> From: jasbir.sing@accenture.com <ja...@accenture.com>
> Sent: Wednesday, May 3, 2017 9:37:44 AM
> To: user@drill.apache.org
> Cc: maneesh.kothari@accenture.com; h.p.kumar@accenture.com
> Subject: RE: Drill query are stuck in ENQUEUED mode
>
> Few things to update -
>
> My parquet files are having less than 1000 records. I create around 250 parquet files from my application and fetch data from them using Apache Drill.
> When I restart the drill instance all 250 queries on parquet files would execute, but after sometime out of 250 only 1 query would be stuck. And this process repeats thereafter after every execution.
>
> And can someone also let me know about the number of records which determine that it will be in exec.queue.small?
>
>
> Regards,
> Jasbir Singh
>
> -----Original Message-----
> From: Sing, Jasbir
> Sent: Wednesday, May 03, 2017 6:40 PM
> To: user@drill.apache.org
> Cc: Kothari, Maneesh <ma...@accenture.com>
> Subject: RE: Drill query are stuck in ENQUEUED mode
>
> IN false state, over the period of time queries were getting stuck in ENQUEUED state because of which I had turned this to TRUE and now they are even getting stuck with this property as TRUE.
>
> -----Original Message-----
> From: Khurram Faraaz [mailto:kfaraaz@mapr.com]
> Sent: Wednesday, May 03, 2017 6:35 PM
> To: user@drill.apache.org
> Cc: Kothari, Maneesh <ma...@accenture.com>
> Subject: Re: Drill query are stuck in ENQUEUED mode
>
> Does your query execute and complete when you set exec.queue.enable = false ?
>
> The default is to set exec.queue.enable to false.
>
>
> Thanks,
>
> Khurram
>
> ________________________________
> From: jasbir.sing@accenture.com <ja...@accenture.com>
> Sent: Wednesday, May 3, 2017 5:58:48 PM
> To: user@drill.apache.org
> Cc: maneesh.kothari@accenture.com
> Subject: Drill query are stuck in ENQUEUED mode
>
> Hi,
>
> I am having queries in which I am fetching just 1 row from the parquet file using LIMIT 1, and even these queries are stuck in ENQUEUED state in DRILL.
>
> I am using exec.queue.enable = true and have default settings for the rest.
>
> Can you help me out in this.
>
> Regards,
> Jasbir Singh
>
> ________________________________
>
> This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
> ______________________________________________________________________________________
>
> www.accenture.com<http://www.accenture.com>
Re: Drill query are stuck in ENQUEUED mode
Posted by Kunal Khatua <kk...@mapr.com>.
What does the jstack command (located in the same dir as the java executable) say when a query is hung?
Also, what is the JStack when the Drillbit is running with the exec.queue.enable=FALSE ?
________________________________
From: jasbir.sing@accenture.com <ja...@accenture.com>
Sent: Thursday, May 4, 2017 5:52:43 AM
To: user@drill.apache.org
Cc: maneesh.kothari@accenture.com; h.p.kumar@accenture.com
Subject: RE: Drill query are stuck in ENQUEUED mode
Hi,
Thanks for the updates.
Just want to update you that we are already following this parameter. Value of below parameters are -
1. exec.queue.enable = TRUE
2. exec.queue.large = 10 (Default)
3. exec.queue.small = 100 (Default)
4. exec.queue.threshold = 30000000(Default)
Exec.queue.timeout_millis = 300000(Default)
Even after following queries are getting stuck. As in our case parquet record count is less than 1000, do I still need to set this threshold or I need to change this value?
As it doesn't tells me on how much record count it decides to queue the query in SMALL section. What is that record count?
Description of my cluster is -
1. Nodes = 1 - Local mode
2. Memory = 64 GB
3. Cores = 24
4. It's a linux box
Please do let me know how to resolve query stuck issue as it is hampering products performance.
Regards,
Jasbir Singh
-----Original Message-----
From: Kunal Khatua [mailto:kkhatua@mapr.com]
Sent: Wednesday, May 03, 2017 10:43 PM
To: user@drill.apache.org
Cc: Kothari, Maneesh <ma...@accenture.com>; Kumar, H. P. <h....@accenture.com>
Subject: Re: Drill query are stuck in ENQUEUED mode
You could try experimenting with these parameters:
https://urldefense.proofpoint.com/v2/url?u=https-3A__drill.apache.org_docs_enabling-2Dquery-2Dqueuing_&d=DwIFAg&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=7scIIjM0jY9x3fjvY6a_yERLxMA2NwA8l0DnuyrL6yA&m=0506fa1JfLmFVj-y5MXeRZgSyJyeYwWoQYArxFhAFpc&s=Dr7zjb2Guf7ZBOsBVweE4jbrxUv_RRxAMGk9NBqL5Zo&e= when you have Queuing enabled.
However, in a false state, there should be no queuing (i.e. no ENQUEUED mode).
Can you provide a description of your cluster? (e.g. nodes, cores, memory... is it a VM, etc)
One way to understand why a query is 'stuck' with queuing disabled would be to use JStack and identify the state of the foreman and fragment threads executing that query.
For this, you first need to know what is the query ID (the part that follows the "https://urldefense.proofpoint.com/v2/url?u=http-3A__hostname-3A8047_profiles_&d=DwIFAg&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=7scIIjM0jY9x3fjvY6a_yERLxMA2NwA8l0DnuyrL6yA&m=0506fa1JfLmFVj-y5MXeRZgSyJyeYwWoQYArxFhAFpc&s=WL1ut57ugpifNnM-IVN1E-NzFBOigrMvDwHw4Uj3etg&e= " part of the query's web UI page). Use that as a reference to identify the corresponding threads.
Usually, a large number of threads will carry this as a part of their thread names. However, if it is stuck, my hunch is that only 1 fragment thread is stuck, while the others completed. So you should be able to find that easily.
Kunal Khatua
Engineering
[1490734684477_mapr.png]
www.mapr.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.mapr.com_&d=DwIFAg&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=7scIIjM0jY9x3fjvY6a_yERLxMA2NwA8l0DnuyrL6yA&m=0506fa1JfLmFVj-y5MXeRZgSyJyeYwWoQYArxFhAFpc&s=sYU_DLIUmwFTX0OxodY-x06c7dD7RrXjNex3CQFiQVk&e= >
________________________________
From: jasbir.sing@accenture.com <ja...@accenture.com>
Sent: Wednesday, May 3, 2017 9:37:44 AM
To: user@drill.apache.org
Cc: maneesh.kothari@accenture.com; h.p.kumar@accenture.com
Subject: RE: Drill query are stuck in ENQUEUED mode
Few things to update -
My parquet files are having less than 1000 records. I create around 250 parquet files from my application and fetch data from them using Apache Drill.
When I restart the drill instance all 250 queries on parquet files would execute, but after sometime out of 250 only 1 query would be stuck. And this process repeats thereafter after every execution.
And can someone also let me know about the number of records which determine that it will be in exec.queue.small?
Regards,
Jasbir Singh
-----Original Message-----
From: Sing, Jasbir
Sent: Wednesday, May 03, 2017 6:40 PM
To: user@drill.apache.org
Cc: Kothari, Maneesh <ma...@accenture.com>
Subject: RE: Drill query are stuck in ENQUEUED mode
IN false state, over the period of time queries were getting stuck in ENQUEUED state because of which I had turned this to TRUE and now they are even getting stuck with this property as TRUE.
-----Original Message-----
From: Khurram Faraaz [mailto:kfaraaz@mapr.com]
Sent: Wednesday, May 03, 2017 6:35 PM
To: user@drill.apache.org
Cc: Kothari, Maneesh <ma...@accenture.com>
Subject: Re: Drill query are stuck in ENQUEUED mode
Does your query execute and complete when you set exec.queue.enable = false ?
The default is to set exec.queue.enable to false.
Thanks,
Khurram
________________________________
From: jasbir.sing@accenture.com <ja...@accenture.com>
Sent: Wednesday, May 3, 2017 5:58:48 PM
To: user@drill.apache.org
Cc: maneesh.kothari@accenture.com
Subject: Drill query are stuck in ENQUEUED mode
Hi,
I am having queries in which I am fetching just 1 row from the parquet file using LIMIT 1, and even these queries are stuck in ENQUEUED state in DRILL.
I am using exec.queue.enable = true and have default settings for the rest.
Can you help me out in this.
Regards,
Jasbir Singh
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com>
RE: Drill query are stuck in ENQUEUED mode
Posted by ja...@accenture.com.
Hi,
Thanks for the updates.
Just want to update you that we are already following this parameter. Value of below parameters are -
1. exec.queue.enable = TRUE
2. exec.queue.large = 10 (Default)
3. exec.queue.small = 100 (Default)
4. exec.queue.threshold = 30000000(Default)
Exec.queue.timeout_millis = 300000(Default)
Even after following queries are getting stuck. As in our case parquet record count is less than 1000, do I still need to set this threshold or I need to change this value?
As it doesn't tells me on how much record count it decides to queue the query in SMALL section. What is that record count?
Description of my cluster is -
1. Nodes = 1 - Local mode
2. Memory = 64 GB
3. Cores = 24
4. It's a linux box
Please do let me know how to resolve query stuck issue as it is hampering products performance.
Regards,
Jasbir Singh
-----Original Message-----
From: Kunal Khatua [mailto:kkhatua@mapr.com]
Sent: Wednesday, May 03, 2017 10:43 PM
To: user@drill.apache.org
Cc: Kothari, Maneesh <ma...@accenture.com>; Kumar, H. P. <h....@accenture.com>
Subject: Re: Drill query are stuck in ENQUEUED mode
You could try experimenting with these parameters:
https://urldefense.proofpoint.com/v2/url?u=https-3A__drill.apache.org_docs_enabling-2Dquery-2Dqueuing_&d=DwIFAg&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=7scIIjM0jY9x3fjvY6a_yERLxMA2NwA8l0DnuyrL6yA&m=0506fa1JfLmFVj-y5MXeRZgSyJyeYwWoQYArxFhAFpc&s=Dr7zjb2Guf7ZBOsBVweE4jbrxUv_RRxAMGk9NBqL5Zo&e= when you have Queuing enabled.
However, in a false state, there should be no queuing (i.e. no ENQUEUED mode).
Can you provide a description of your cluster? (e.g. nodes, cores, memory... is it a VM, etc)
One way to understand why a query is 'stuck' with queuing disabled would be to use JStack and identify the state of the foreman and fragment threads executing that query.
For this, you first need to know what is the query ID (the part that follows the "https://urldefense.proofpoint.com/v2/url?u=http-3A__hostname-3A8047_profiles_&d=DwIFAg&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=7scIIjM0jY9x3fjvY6a_yERLxMA2NwA8l0DnuyrL6yA&m=0506fa1JfLmFVj-y5MXeRZgSyJyeYwWoQYArxFhAFpc&s=WL1ut57ugpifNnM-IVN1E-NzFBOigrMvDwHw4Uj3etg&e= " part of the query's web UI page). Use that as a reference to identify the corresponding threads.
Usually, a large number of threads will carry this as a part of their thread names. However, if it is stuck, my hunch is that only 1 fragment thread is stuck, while the others completed. So you should be able to find that easily.
Kunal Khatua
Engineering
[1490734684477_mapr.png]
www.mapr.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.mapr.com_&d=DwIFAg&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=7scIIjM0jY9x3fjvY6a_yERLxMA2NwA8l0DnuyrL6yA&m=0506fa1JfLmFVj-y5MXeRZgSyJyeYwWoQYArxFhAFpc&s=sYU_DLIUmwFTX0OxodY-x06c7dD7RrXjNex3CQFiQVk&e= >
________________________________
From: jasbir.sing@accenture.com <ja...@accenture.com>
Sent: Wednesday, May 3, 2017 9:37:44 AM
To: user@drill.apache.org
Cc: maneesh.kothari@accenture.com; h.p.kumar@accenture.com
Subject: RE: Drill query are stuck in ENQUEUED mode
Few things to update -
My parquet files are having less than 1000 records. I create around 250 parquet files from my application and fetch data from them using Apache Drill.
When I restart the drill instance all 250 queries on parquet files would execute, but after sometime out of 250 only 1 query would be stuck. And this process repeats thereafter after every execution.
And can someone also let me know about the number of records which determine that it will be in exec.queue.small?
Regards,
Jasbir Singh
-----Original Message-----
From: Sing, Jasbir
Sent: Wednesday, May 03, 2017 6:40 PM
To: user@drill.apache.org
Cc: Kothari, Maneesh <ma...@accenture.com>
Subject: RE: Drill query are stuck in ENQUEUED mode
IN false state, over the period of time queries were getting stuck in ENQUEUED state because of which I had turned this to TRUE and now they are even getting stuck with this property as TRUE.
-----Original Message-----
From: Khurram Faraaz [mailto:kfaraaz@mapr.com]
Sent: Wednesday, May 03, 2017 6:35 PM
To: user@drill.apache.org
Cc: Kothari, Maneesh <ma...@accenture.com>
Subject: Re: Drill query are stuck in ENQUEUED mode
Does your query execute and complete when you set exec.queue.enable = false ?
The default is to set exec.queue.enable to false.
Thanks,
Khurram
________________________________
From: jasbir.sing@accenture.com <ja...@accenture.com>
Sent: Wednesday, May 3, 2017 5:58:48 PM
To: user@drill.apache.org
Cc: maneesh.kothari@accenture.com
Subject: Drill query are stuck in ENQUEUED mode
Hi,
I am having queries in which I am fetching just 1 row from the parquet file using LIMIT 1, and even these queries are stuck in ENQUEUED state in DRILL.
I am using exec.queue.enable = true and have default settings for the rest.
Can you help me out in this.
Regards,
Jasbir Singh
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com>
Re: Drill query are stuck in ENQUEUED mode
Posted by Kunal Khatua <kk...@mapr.com>.
You could try experimenting with these parameters:
https://drill.apache.org/docs/enabling-query-queuing/ when you have Queuing enabled.
However, in a false state, there should be no queuing (i.e. no ENQUEUED mode).
Can you provide a description of your cluster? (e.g. nodes, cores, memory... is it a VM, etc)
One way to understand why a query is 'stuck' with queuing disabled would be to use JStack and identify the state of the foreman and fragment threads executing that query.
For this, you first need to know what is the query ID (the part that follows the "http://hostname:8047/profiles/" part of the query's web UI page). Use that as a reference to identify the corresponding threads.
Usually, a large number of threads will carry this as a part of their thread names. However, if it is stuck, my hunch is that only 1 fragment thread is stuck, while the others completed. So you should be able to find that easily.
Kunal Khatua
Engineering
[1490734684477_mapr.png]
www.mapr.com<http://www.mapr.com/>
________________________________
From: jasbir.sing@accenture.com <ja...@accenture.com>
Sent: Wednesday, May 3, 2017 9:37:44 AM
To: user@drill.apache.org
Cc: maneesh.kothari@accenture.com; h.p.kumar@accenture.com
Subject: RE: Drill query are stuck in ENQUEUED mode
Few things to update -
My parquet files are having less than 1000 records. I create around 250 parquet files from my application and fetch data from them using Apache Drill.
When I restart the drill instance all 250 queries on parquet files would execute, but after sometime out of 250 only 1 query would be stuck. And this process repeats thereafter after every execution.
And can someone also let me know about the number of records which determine that it will be in exec.queue.small?
Regards,
Jasbir Singh
-----Original Message-----
From: Sing, Jasbir
Sent: Wednesday, May 03, 2017 6:40 PM
To: user@drill.apache.org
Cc: Kothari, Maneesh <ma...@accenture.com>
Subject: RE: Drill query are stuck in ENQUEUED mode
IN false state, over the period of time queries were getting stuck in ENQUEUED state because of which I had turned this to TRUE and now they are even getting stuck with this property as TRUE.
-----Original Message-----
From: Khurram Faraaz [mailto:kfaraaz@mapr.com]
Sent: Wednesday, May 03, 2017 6:35 PM
To: user@drill.apache.org
Cc: Kothari, Maneesh <ma...@accenture.com>
Subject: Re: Drill query are stuck in ENQUEUED mode
Does your query execute and complete when you set exec.queue.enable = false ?
The default is to set exec.queue.enable to false.
Thanks,
Khurram
________________________________
From: jasbir.sing@accenture.com <ja...@accenture.com>
Sent: Wednesday, May 3, 2017 5:58:48 PM
To: user@drill.apache.org
Cc: maneesh.kothari@accenture.com
Subject: Drill query are stuck in ENQUEUED mode
Hi,
I am having queries in which I am fetching just 1 row from the parquet file using LIMIT 1, and even these queries are stuck in ENQUEUED state in DRILL.
I am using exec.queue.enable = true and have default settings for the rest.
Can you help me out in this.
Regards,
Jasbir Singh
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com>
RE: Drill query are stuck in ENQUEUED mode
Posted by ja...@accenture.com.
Few things to update -
My parquet files are having less than 1000 records. I create around 250 parquet files from my application and fetch data from them using Apache Drill.
When I restart the drill instance all 250 queries on parquet files would execute, but after sometime out of 250 only 1 query would be stuck. And this process repeats thereafter after every execution.
And can someone also let me know about the number of records which determine that it will be in exec.queue.small?
Regards,
Jasbir Singh
-----Original Message-----
From: Sing, Jasbir
Sent: Wednesday, May 03, 2017 6:40 PM
To: user@drill.apache.org
Cc: Kothari, Maneesh <ma...@accenture.com>
Subject: RE: Drill query are stuck in ENQUEUED mode
IN false state, over the period of time queries were getting stuck in ENQUEUED state because of which I had turned this to TRUE and now they are even getting stuck with this property as TRUE.
-----Original Message-----
From: Khurram Faraaz [mailto:kfaraaz@mapr.com]
Sent: Wednesday, May 03, 2017 6:35 PM
To: user@drill.apache.org
Cc: Kothari, Maneesh <ma...@accenture.com>
Subject: Re: Drill query are stuck in ENQUEUED mode
Does your query execute and complete when you set exec.queue.enable = false ?
The default is to set exec.queue.enable to false.
Thanks,
Khurram
________________________________
From: jasbir.sing@accenture.com <ja...@accenture.com>
Sent: Wednesday, May 3, 2017 5:58:48 PM
To: user@drill.apache.org
Cc: maneesh.kothari@accenture.com
Subject: Drill query are stuck in ENQUEUED mode
Hi,
I am having queries in which I am fetching just 1 row from the parquet file using LIMIT 1, and even these queries are stuck in ENQUEUED state in DRILL.
I am using exec.queue.enable = true and have default settings for the rest.
Can you help me out in this.
Regards,
Jasbir Singh
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com>
RE: Drill query are stuck in ENQUEUED mode
Posted by ja...@accenture.com.
IN false state, over the period of time queries were getting stuck in ENQUEUED state because of which I had turned this to TRUE and now they are even getting stuck with this property as TRUE.
-----Original Message-----
From: Khurram Faraaz [mailto:kfaraaz@mapr.com]
Sent: Wednesday, May 03, 2017 6:35 PM
To: user@drill.apache.org
Cc: Kothari, Maneesh <ma...@accenture.com>
Subject: Re: Drill query are stuck in ENQUEUED mode
Does your query execute and complete when you set exec.queue.enable = false ?
The default is to set exec.queue.enable to false.
Thanks,
Khurram
________________________________
From: jasbir.sing@accenture.com <ja...@accenture.com>
Sent: Wednesday, May 3, 2017 5:58:48 PM
To: user@drill.apache.org
Cc: maneesh.kothari@accenture.com
Subject: Drill query are stuck in ENQUEUED mode
Hi,
I am having queries in which I am fetching just 1 row from the parquet file using LIMIT 1, and even these queries are stuck in ENQUEUED state in DRILL.
I am using exec.queue.enable = true and have default settings for the rest.
Can you help me out in this.
Regards,
Jasbir Singh
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com>
Re: Drill query are stuck in ENQUEUED mode
Posted by Khurram Faraaz <kf...@mapr.com>.
Does your query execute and complete when you set exec.queue.enable = false ?
The default is to set exec.queue.enable to false.
Thanks,
Khurram
________________________________
From: jasbir.sing@accenture.com <ja...@accenture.com>
Sent: Wednesday, May 3, 2017 5:58:48 PM
To: user@drill.apache.org
Cc: maneesh.kothari@accenture.com
Subject: Drill query are stuck in ENQUEUED mode
Hi,
I am having queries in which I am fetching just 1 row from the parquet file using LIMIT 1, and even these queries are stuck in ENQUEUED state in DRILL.
I am using exec.queue.enable = true and have default settings for the rest.
Can you help me out in this.
Regards,
Jasbir Singh
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com>