You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by Me...@lamresearch.com on 2017/05/24 19:58:30 UTC
Phoenix driver query rejection - note change in config?
I have recently set up a JDBC driver to connect to hdfs using Apache Phoenix. Basic queries on Squirrel have worked well (for example, "select * from datafile"), but as soon as I ask a slightly more complicated query (ie, "select column1 from datafile where column2 = 'filter1'", I encounter this error:
org.apache.phoenix.exception.PhoenixIOException: Task
org.apache.phoenix.job.JobManager$InstrumentedJobFutureTask rejected from
org.apache.phoenix.job.JobManager[Running, pool size = 128, active threads =
128, queued tasks = 5000, completed tasks = 5132]
From some searching, it seems that I should increase the ThreadPoolSize in the Apache Phoenix hbase.xml configuration file in order to avoid this error, which I have done, increasing it from 128 to 512. However, it does not seem to have noticed this change. The error persists and the "pool size" is still given as 128 within the error.
On the Phoenix Driver settings in Squirrel, I have indicated the location of hbase and hdfs directories containing the .xml config files under "Extra Class Path" in setup.
Is there any way to make the driver "notice" that the ThreadPoolSize has changed?
Thank you!
-----
Megan McClarty
Process Engineer 2 | Lam Research
E-mail<ma...@lamresearch.com> | LinkedIn<https://www.linkedin.com/in/meganmcclarty>
Re: Phoenix driver query rejection - note change in config?
Posted by Josh Elser <el...@apache.org>.
Megan,
Not quite sure I understand your question. The queues are local to your
client, not shared by anyone else. In other words, you're only competing
against yourself (there's nothing to gain from changing priorities).
I'm surprised that you're seeing issues running such a simple query
(select with single predicate). Can you share the updated error message
you see, and your table's DDL statement? Perhaps the output of an
EXPLAIN on your query, too?
On 6/2/17 4:47 PM, Megan.McClarty@lamresearch.com wrote:
> Hello,
>
> Thank you for your reply. I did do the restarts and I just deleted and re-added the jar in the Squirrel lib and it has now noticed the change! So thank you. However, the error persists even with increases ThreadPoolSize and QueueSize. Could this be a queuing priority error? Is there any way on the client side to bump my tasks up in priority?
>
> Thank you!
>
>
> -----Original Message-----
> From: Josh Elser [mailto:elserj@apache.org]
> Sent: Thursday, May 25, 2017 9:30 AM
> To: user@phoenix.apache.org
> Subject: Re: Phoenix driver query rejection - note change in config?
>
> Hi Megan,
>
> Did you happen to restart Squirrel and/or re-connect to Phoenix after making the change? The steps you took (Squirrel, notwithstanding) should have sufficiently fixed the issue you described.
>
> Another sanity check would be to make sure your didn't have any mis-typing of the configuration key in hbase-site.xml.
>
> On 5/24/17 3:58 PM, Megan.McClarty@lamresearch.com wrote:
>> I have recently set up a JDBC driver to connect to hdfs using Apache
>> Phoenix. Basic queries on Squirrel have worked well (for example,
>> "select * from datafile"), but as soon as I ask a slightly more
>> complicated query (ie, "select column1 from datafile where column2 =
>> 'filter1'", I encounter this error:
>>
>> org.apache.phoenix.exception.PhoenixIOException: Task
>>
>> org.apache.phoenix.job.JobManager$InstrumentedJobFutureTask rejected
>> from
>>
>> org.apache.phoenix.job.JobManager[Running, pool size = 128, active
>> threads =
>>
>> 128, queued tasks = 5000, completed tasks = 5132]
>>
>> From some searching, it seems that I should increase the
>> ThreadPoolSize in the Apache Phoenix hbase.xml configuration file in
>> order to avoid this error, which I have done, increasing it from 128
>> to 512. However, it does not seem to have noticed this change. The
>> error persists and the "pool size" is still given as 128 within the error.
>>
>> On the Phoenix Driver settings in Squirrel, I have indicated the
>> location of hbase and hdfs directories containing the .xml config
>> files under "Extra Class Path" in setup.
>>
>> Is there any way to make the driver "notice" that the ThreadPoolSize
>> has changed?
>>
>> Thank you!
>>
>> *-----*
>>
>> *Megan McClarty*
>>
>> Process Engineer 2 | Lam Research
>>
>> E-mail <ma...@lamresearch.com> | LinkedIn
>> <https://www.linkedin.com/in/meganmcclarty>
>>
RE: Phoenix driver query rejection - note change in config?
Posted by Me...@lamresearch.com.
Hello,
Thank you for your reply. I did do the restarts and I just deleted and re-added the jar in the Squirrel lib and it has now noticed the change! So thank you. However, the error persists even with increases ThreadPoolSize and QueueSize. Could this be a queuing priority error? Is there any way on the client side to bump my tasks up in priority?
Thank you!
-----Original Message-----
From: Josh Elser [mailto:elserj@apache.org]
Sent: Thursday, May 25, 2017 9:30 AM
To: user@phoenix.apache.org
Subject: Re: Phoenix driver query rejection - note change in config?
Hi Megan,
Did you happen to restart Squirrel and/or re-connect to Phoenix after making the change? The steps you took (Squirrel, notwithstanding) should have sufficiently fixed the issue you described.
Another sanity check would be to make sure your didn't have any mis-typing of the configuration key in hbase-site.xml.
On 5/24/17 3:58 PM, Megan.McClarty@lamresearch.com wrote:
> I have recently set up a JDBC driver to connect to hdfs using Apache
> Phoenix. Basic queries on Squirrel have worked well (for example,
> "select * from datafile"), but as soon as I ask a slightly more
> complicated query (ie, "select column1 from datafile where column2 =
> 'filter1'", I encounter this error:
>
> org.apache.phoenix.exception.PhoenixIOException: Task
>
> org.apache.phoenix.job.JobManager$InstrumentedJobFutureTask rejected
> from
>
> org.apache.phoenix.job.JobManager[Running, pool size = 128, active
> threads =
>
> 128, queued tasks = 5000, completed tasks = 5132]
>
> From some searching, it seems that I should increase the
> ThreadPoolSize in the Apache Phoenix hbase.xml configuration file in
> order to avoid this error, which I have done, increasing it from 128
> to 512. However, it does not seem to have noticed this change. The
> error persists and the "pool size" is still given as 128 within the error.
>
> On the Phoenix Driver settings in Squirrel, I have indicated the
> location of hbase and hdfs directories containing the .xml config
> files under "Extra Class Path" in setup.
>
> Is there any way to make the driver "notice" that the ThreadPoolSize
> has changed?
>
> Thank you!
>
> *-----*
>
> *Megan McClarty*
>
> Process Engineer 2 | Lam Research
>
> E-mail <ma...@lamresearch.com> | LinkedIn
> <https://www.linkedin.com/in/meganmcclarty>
>
Re: Phoenix driver query rejection - note change in config?
Posted by Josh Elser <el...@apache.org>.
Hi Megan,
Did you happen to restart Squirrel and/or re-connect to Phoenix after
making the change? The steps you took (Squirrel, notwithstanding) should
have sufficiently fixed the issue you described.
Another sanity check would be to make sure your didn't have any
mis-typing of the configuration key in hbase-site.xml.
On 5/24/17 3:58 PM, Megan.McClarty@lamresearch.com wrote:
> I have recently set up a JDBC driver to connect to hdfs using Apache
> Phoenix. Basic queries on Squirrel have worked well (for example,
> "select * from datafile"), but as soon as I ask a slightly more
> complicated query (ie, "select column1 from datafile where column2 =
> 'filter1'", I encounter this error:
>
> org.apache.phoenix.exception.PhoenixIOException: Task
>
> org.apache.phoenix.job.JobManager$InstrumentedJobFutureTask rejected from
>
> org.apache.phoenix.job.JobManager[Running, pool size = 128, active
> threads =
>
> 128, queued tasks = 5000, completed tasks = 5132]
>
> From some searching, it seems that I should increase the ThreadPoolSize
> in the Apache Phoenix hbase.xml configuration file in order to avoid
> this error, which I have done, increasing it from 128 to 512. However,
> it does not seem to have noticed this change. The error persists and the
> "pool size" is still given as 128 within the error.
>
> On the Phoenix Driver settings in Squirrel, I have indicated the
> location of hbase and hdfs directories containing the .xml config files
> under "Extra Class Path" in setup.
>
> Is there any way to make the driver "notice" that the ThreadPoolSize has
> changed?
>
> Thank you!
>
> *-----*
>
> *Megan McClarty*
>
> Process Engineer 2 | Lam Research
>
> E-mail <ma...@lamresearch.com> | LinkedIn
> <https://www.linkedin.com/in/meganmcclarty>
>