You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by William Shen <wi...@marinsoftware.com> on 2019/01/31 05:37:12 UTC

Phoenix JDBC Connection Warmup

Hi there,

I have a component that makes Phoenix queries via the Phoenix JDBC
Connection. I noticed that consistently, the Phoenix Client takes longer to
execute a PreparedStatement and it takes longer to read through the
ResultSet for a period of time (~15m) after a restart of the component. It
seems like there is a warmup period for the JDBC connection. Is this to be
expected?

Thanks!

Re: Phoenix JDBC Connection Warmup

Posted by Thomas D'Silva <td...@salesforce.com>.
As James suggested if you set the UPDATE_CACHE_FREQUENCY table property,
the server will not be pinged for the latest metadata until the update
frequency.
Check out the altering section (https://phoenix.apache.org/)

On Sun, Feb 3, 2019 at 5:57 PM William Shen <wi...@marinsoftware.com>
wrote:

> Thanks for the suggestions!
>
> Jaanai - do you mean to enable trace logging on the client side or on the
> server side?
>
> James - no I have not tried setting it. I have not heard about the
> configuration. Do you have a ball park suggestion on how to approach
> setting this value?
> On Sat, Feb 2, 2019 at 8:53 AM James Taylor <ja...@apache.org>
> wrote:
>
>> Have you tried setting UPDATE_CACHE_FREQUENCY on your tables?
>>
>> On Fri, Feb 1, 2019 at 6:28 PM Jaanai Zhang <cl...@gmail.com>
>> wrote:
>>
>>>  we experimented with issuing the same query repeatedly, and we observed
>>>> a slow down not only on the first query
>>>
>>> I am not sure what the reasons are, perhaps you can enable TRACE log to
>>> find what leads to slow,  I guess that some meta information is reloaded
>>> under highly write workload.
>>>
>>> ----------------------------------------
>>>    Jaanai Zhang
>>>    Best regards!
>>>
>>>
>>>
>>> William Shen <wi...@marinsoftware.com> 于2019年2月1日周五 上午2:09写道:
>>>
>>>> Thanks Jaanai. Do you know if that is expected only on the first query
>>>> against a table? For us, we experimented with issuing the same query
>>>> repeatedly, and we observed a slow down not only on the first query. Does
>>>> it make sense to preemptively load table metadata on start up to warm up
>>>> the system to reduce latency during the actual query time (if it is
>>>> possible to do so)?
>>>>
>>>> On Wed, Jan 30, 2019 at 10:54 PM Jaanai Zhang <cl...@gmail.com>
>>>> wrote:
>>>>
>>>>> It is expected when firstly query tables after establishing the
>>>>> connection. Something likes loads some meta information into local cache
>>>>> that need take some time,  mainly including two aspects: 1. access
>>>>> SYSTEM.CATALOG table to get schema information of the table  2. access the
>>>>> meta table of HBase to get regions information of the table
>>>>>
>>>>> ----------------------------------------
>>>>>    Jaanai Zhang
>>>>>    Best regards!
>>>>>
>>>>>
>>>>>
>>>>> William Shen <wi...@marinsoftware.com> 于2019年1月31日周四 下午1:37写道:
>>>>>
>>>>>> Hi there,
>>>>>>
>>>>>> I have a component that makes Phoenix queries via the Phoenix JDBC
>>>>>> Connection. I noticed that consistently, the Phoenix Client takes longer to
>>>>>> execute a PreparedStatement and it takes longer to read through the
>>>>>> ResultSet for a period of time (~15m) after a restart of the component. It
>>>>>> seems like there is a warmup period for the JDBC connection. Is this to be
>>>>>> expected?
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>

Re: Phoenix JDBC Connection Warmup

Posted by William Shen <wi...@marinsoftware.com>.
Thanks for the suggestions!

Jaanai - do you mean to enable trace logging on the client side or on the
server side?

James - no I have not tried setting it. I have not heard about the
configuration. Do you have a ball park suggestion on how to approach
setting this value?
On Sat, Feb 2, 2019 at 8:53 AM James Taylor <ja...@apache.org> wrote:

> Have you tried setting UPDATE_CACHE_FREQUENCY on your tables?
>
> On Fri, Feb 1, 2019 at 6:28 PM Jaanai Zhang <cl...@gmail.com>
> wrote:
>
>>  we experimented with issuing the same query repeatedly, and we observed
>>> a slow down not only on the first query
>>
>> I am not sure what the reasons are, perhaps you can enable TRACE log to
>> find what leads to slow,  I guess that some meta information is reloaded
>> under highly write workload.
>>
>> ----------------------------------------
>>    Jaanai Zhang
>>    Best regards!
>>
>>
>>
>> William Shen <wi...@marinsoftware.com> 于2019年2月1日周五 上午2:09写道:
>>
>>> Thanks Jaanai. Do you know if that is expected only on the first query
>>> against a table? For us, we experimented with issuing the same query
>>> repeatedly, and we observed a slow down not only on the first query. Does
>>> it make sense to preemptively load table metadata on start up to warm up
>>> the system to reduce latency during the actual query time (if it is
>>> possible to do so)?
>>>
>>> On Wed, Jan 30, 2019 at 10:54 PM Jaanai Zhang <cl...@gmail.com>
>>> wrote:
>>>
>>>> It is expected when firstly query tables after establishing the
>>>> connection. Something likes loads some meta information into local cache
>>>> that need take some time,  mainly including two aspects: 1. access
>>>> SYSTEM.CATALOG table to get schema information of the table  2. access the
>>>> meta table of HBase to get regions information of the table
>>>>
>>>> ----------------------------------------
>>>>    Jaanai Zhang
>>>>    Best regards!
>>>>
>>>>
>>>>
>>>> William Shen <wi...@marinsoftware.com> 于2019年1月31日周四 下午1:37写道:
>>>>
>>>>> Hi there,
>>>>>
>>>>> I have a component that makes Phoenix queries via the Phoenix JDBC
>>>>> Connection. I noticed that consistently, the Phoenix Client takes longer to
>>>>> execute a PreparedStatement and it takes longer to read through the
>>>>> ResultSet for a period of time (~15m) after a restart of the component. It
>>>>> seems like there is a warmup period for the JDBC connection. Is this to be
>>>>> expected?
>>>>>
>>>>> Thanks!
>>>>>
>>>>

Re: Phoenix JDBC Connection Warmup

Posted by James Taylor <ja...@apache.org>.
Have you tried setting UPDATE_CACHE_FREQUENCY on your tables?

On Fri, Feb 1, 2019 at 6:28 PM Jaanai Zhang <cl...@gmail.com> wrote:

>  we experimented with issuing the same query repeatedly, and we observed a
>> slow down not only on the first query
>
> I am not sure what the reasons are, perhaps you can enable TRACE log to
> find what leads to slow,  I guess that some meta information is reloaded
> under highly write workload.
>
> ----------------------------------------
>    Jaanai Zhang
>    Best regards!
>
>
>
> William Shen <wi...@marinsoftware.com> 于2019年2月1日周五 上午2:09写道:
>
>> Thanks Jaanai. Do you know if that is expected only on the first query
>> against a table? For us, we experimented with issuing the same query
>> repeatedly, and we observed a slow down not only on the first query. Does
>> it make sense to preemptively load table metadata on start up to warm up
>> the system to reduce latency during the actual query time (if it is
>> possible to do so)?
>>
>> On Wed, Jan 30, 2019 at 10:54 PM Jaanai Zhang <cl...@gmail.com>
>> wrote:
>>
>>> It is expected when firstly query tables after establishing the
>>> connection. Something likes loads some meta information into local cache
>>> that need take some time,  mainly including two aspects: 1. access
>>> SYSTEM.CATALOG table to get schema information of the table  2. access the
>>> meta table of HBase to get regions information of the table
>>>
>>> ----------------------------------------
>>>    Jaanai Zhang
>>>    Best regards!
>>>
>>>
>>>
>>> William Shen <wi...@marinsoftware.com> 于2019年1月31日周四 下午1:37写道:
>>>
>>>> Hi there,
>>>>
>>>> I have a component that makes Phoenix queries via the Phoenix JDBC
>>>> Connection. I noticed that consistently, the Phoenix Client takes longer to
>>>> execute a PreparedStatement and it takes longer to read through the
>>>> ResultSet for a period of time (~15m) after a restart of the component. It
>>>> seems like there is a warmup period for the JDBC connection. Is this to be
>>>> expected?
>>>>
>>>> Thanks!
>>>>
>>>

Re: Phoenix JDBC Connection Warmup

Posted by Jaanai Zhang <cl...@gmail.com>.
>
>  we experimented with issuing the same query repeatedly, and we observed a
> slow down not only on the first query

I am not sure what the reasons are, perhaps you can enable TRACE log to
find what leads to slow,  I guess that some meta information is reloaded
under highly write workload.

----------------------------------------
   Jaanai Zhang
   Best regards!



William Shen <wi...@marinsoftware.com> 于2019年2月1日周五 上午2:09写道:

> Thanks Jaanai. Do you know if that is expected only on the first query
> against a table? For us, we experimented with issuing the same query
> repeatedly, and we observed a slow down not only on the first query. Does
> it make sense to preemptively load table metadata on start up to warm up
> the system to reduce latency during the actual query time (if it is
> possible to do so)?
>
> On Wed, Jan 30, 2019 at 10:54 PM Jaanai Zhang <cl...@gmail.com>
> wrote:
>
>> It is expected when firstly query tables after establishing the
>> connection. Something likes loads some meta information into local cache
>> that need take some time,  mainly including two aspects: 1. access
>> SYSTEM.CATALOG table to get schema information of the table  2. access the
>> meta table of HBase to get regions information of the table
>>
>> ----------------------------------------
>>    Jaanai Zhang
>>    Best regards!
>>
>>
>>
>> William Shen <wi...@marinsoftware.com> 于2019年1月31日周四 下午1:37写道:
>>
>>> Hi there,
>>>
>>> I have a component that makes Phoenix queries via the Phoenix JDBC
>>> Connection. I noticed that consistently, the Phoenix Client takes longer to
>>> execute a PreparedStatement and it takes longer to read through the
>>> ResultSet for a period of time (~15m) after a restart of the component. It
>>> seems like there is a warmup period for the JDBC connection. Is this to be
>>> expected?
>>>
>>> Thanks!
>>>
>>

Re: Phoenix JDBC Connection Warmup

Posted by William Shen <wi...@marinsoftware.com>.
Thanks Jaanai. Do you know if that is expected only on the first query
against a table? For us, we experimented with issuing the same query
repeatedly, and we observed a slow down not only on the first query. Does
it make sense to preemptively load table metadata on start up to warm up
the system to reduce latency during the actual query time (if it is
possible to do so)?

On Wed, Jan 30, 2019 at 10:54 PM Jaanai Zhang <cl...@gmail.com>
wrote:

> It is expected when firstly query tables after establishing the
> connection. Something likes loads some meta information into local cache
> that need take some time,  mainly including two aspects: 1. access
> SYSTEM.CATALOG table to get schema information of the table  2. access the
> meta table of HBase to get regions information of the table
>
> ----------------------------------------
>    Jaanai Zhang
>    Best regards!
>
>
>
> William Shen <wi...@marinsoftware.com> 于2019年1月31日周四 下午1:37写道:
>
>> Hi there,
>>
>> I have a component that makes Phoenix queries via the Phoenix JDBC
>> Connection. I noticed that consistently, the Phoenix Client takes longer to
>> execute a PreparedStatement and it takes longer to read through the
>> ResultSet for a period of time (~15m) after a restart of the component. It
>> seems like there is a warmup period for the JDBC connection. Is this to be
>> expected?
>>
>> Thanks!
>>
>

Re: Phoenix JDBC Connection Warmup

Posted by Jaanai Zhang <cl...@gmail.com>.
It is expected when firstly query tables after establishing the connection.
Something likes loads some meta information into local cache that need take
some time,  mainly including two aspects: 1. access SYSTEM.CATALOG table to
get schema information of the table  2. access the meta table of HBase to
get regions information of the table

----------------------------------------
   Jaanai Zhang
   Best regards!



William Shen <wi...@marinsoftware.com> 于2019年1月31日周四 下午1:37写道:

> Hi there,
>
> I have a component that makes Phoenix queries via the Phoenix JDBC
> Connection. I noticed that consistently, the Phoenix Client takes longer to
> execute a PreparedStatement and it takes longer to read through the
> ResultSet for a period of time (~15m) after a restart of the component. It
> seems like there is a warmup period for the JDBC connection. Is this to be
> expected?
>
> Thanks!
>