You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by Jaanai Zhang <cl...@gmail.com> on 2019/02/02 02:28:03 UTC
Re: Phoenix JDBC Connection Warmup
>
> we experimented with issuing the same query repeatedly, and we observed a
> slow down not only on the first query
I am not sure what the reasons are, perhaps you can enable TRACE log to
find what leads to slow, I guess that some meta information is reloaded
under highly write workload.
----------------------------------------
Jaanai Zhang
Best regards!
William Shen <wi...@marinsoftware.com> 于2019年2月1日周五 上午2:09写道：
> Thanks Jaanai. Do you know if that is expected only on the first query
> against a table? For us, we experimented with issuing the same query
> repeatedly, and we observed a slow down not only on the first query. Does
> it make sense to preemptively load table metadata on start up to warm up
> the system to reduce latency during the actual query time (if it is
> possible to do so)?
>
> On Wed, Jan 30, 2019 at 10:54 PM Jaanai Zhang <cl...@gmail.com>
> wrote:
>
>> It is expected when firstly query tables after establishing the
>> connection. Something likes loads some meta information into local cache
>> that need take some time, mainly including two aspects: 1. access
>> SYSTEM.CATALOG table to get schema information of the table 2. access the
>> meta table of HBase to get regions information of the table
>>
>> ----------------------------------------
>> Jaanai Zhang
>> Best regards!
>>
>>
>>
>> William Shen <wi...@marinsoftware.com> 于2019年1月31日周四 下午1:37写道：
>>
>>> Hi there,
>>>
>>> I have a component that makes Phoenix queries via the Phoenix JDBC
>>> Connection. I noticed that consistently, the Phoenix Client takes longer to
>>> execute a PreparedStatement and it takes longer to read through the
>>> ResultSet for a period of time (~15m) after a restart of the component. It
>>> seems like there is a warmup period for the JDBC connection. Is this to be
>>> expected?
>>>
>>> Thanks!
>>>
>>
Re: Phoenix JDBC Connection Warmup
Posted by Thomas D'Silva <td...@salesforce.com>.
As James suggested if you set the UPDATE_CACHE_FREQUENCY table property,
the server will not be pinged for the latest metadata until the update
frequency.
Check out the altering section (https://phoenix.apache.org/)
On Sun, Feb 3, 2019 at 5:57 PM William Shen <wi...@marinsoftware.com>
wrote:
> Thanks for the suggestions!
>
> Jaanai - do you mean to enable trace logging on the client side or on the
> server side?
>
> James - no I have not tried setting it. I have not heard about the
> configuration. Do you have a ball park suggestion on how to approach
> setting this value?
> On Sat, Feb 2, 2019 at 8:53 AM James Taylor <ja...@apache.org>
> wrote:
>
>> Have you tried setting UPDATE_CACHE_FREQUENCY on your tables?
>>
>> On Fri, Feb 1, 2019 at 6:28 PM Jaanai Zhang <cl...@gmail.com>
>> wrote:
>>
>>> we experimented with issuing the same query repeatedly, and we observed
>>>> a slow down not only on the first query
>>>
>>> I am not sure what the reasons are, perhaps you can enable TRACE log to
>>> find what leads to slow, I guess that some meta information is reloaded
>>> under highly write workload.
>>>
>>> ----------------------------------------
>>> Jaanai Zhang
>>> Best regards!
>>>
>>>
>>>
>>> William Shen <wi...@marinsoftware.com> 于2019年2月1日周五 上午2:09写道：
>>>
>>>> Thanks Jaanai. Do you know if that is expected only on the first query
>>>> against a table? For us, we experimented with issuing the same query
>>>> repeatedly, and we observed a slow down not only on the first query. Does
>>>> it make sense to preemptively load table metadata on start up to warm up
>>>> the system to reduce latency during the actual query time (if it is
>>>> possible to do so)?
>>>>
>>>> On Wed, Jan 30, 2019 at 10:54 PM Jaanai Zhang <cl...@gmail.com>
>>>> wrote:
>>>>
>>>>> It is expected when firstly query tables after establishing the
>>>>> connection. Something likes loads some meta information into local cache
>>>>> that need take some time, mainly including two aspects: 1. access
>>>>> SYSTEM.CATALOG table to get schema information of the table 2. access the
>>>>> meta table of HBase to get regions information of the table
>>>>>
>>>>> ----------------------------------------
>>>>> Jaanai Zhang
>>>>> Best regards!
>>>>>
>>>>>
>>>>>
>>>>> William Shen <wi...@marinsoftware.com> 于2019年1月31日周四 下午1:37写道：
>>>>>
>>>>>> Hi there,
>>>>>>
>>>>>> I have a component that makes Phoenix queries via the Phoenix JDBC
>>>>>> Connection. I noticed that consistently, the Phoenix Client takes longer to
>>>>>> execute a PreparedStatement and it takes longer to read through the
>>>>>> ResultSet for a period of time (~15m) after a restart of the component. It
>>>>>> seems like there is a warmup period for the JDBC connection. Is this to be
>>>>>> expected?
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>
Re: Phoenix JDBC Connection Warmup
Posted by William Shen <wi...@marinsoftware.com>.
Thanks for the suggestions!
Jaanai - do you mean to enable trace logging on the client side or on the
server side?
James - no I have not tried setting it. I have not heard about the
configuration. Do you have a ball park suggestion on how to approach
setting this value?
On Sat, Feb 2, 2019 at 8:53 AM James Taylor <ja...@apache.org> wrote:
> Have you tried setting UPDATE_CACHE_FREQUENCY on your tables?
>
> On Fri, Feb 1, 2019 at 6:28 PM Jaanai Zhang <cl...@gmail.com>
> wrote:
>
>> we experimented with issuing the same query repeatedly, and we observed
>>> a slow down not only on the first query
>>
>> I am not sure what the reasons are, perhaps you can enable TRACE log to
>> find what leads to slow, I guess that some meta information is reloaded
>> under highly write workload.
>>
>> ----------------------------------------
>> Jaanai Zhang
>> Best regards!
>>
>>
>>
>> William Shen <wi...@marinsoftware.com> 于2019年2月1日周五 上午2:09写道：
>>
>>> Thanks Jaanai. Do you know if that is expected only on the first query
>>> against a table? For us, we experimented with issuing the same query
>>> repeatedly, and we observed a slow down not only on the first query. Does
>>> it make sense to preemptively load table metadata on start up to warm up
>>> the system to reduce latency during the actual query time (if it is
>>> possible to do so)?
>>>
>>> On Wed, Jan 30, 2019 at 10:54 PM Jaanai Zhang <cl...@gmail.com>
>>> wrote:
>>>
>>>> It is expected when firstly query tables after establishing the
>>>> connection. Something likes loads some meta information into local cache
>>>> that need take some time, mainly including two aspects: 1. access
>>>> SYSTEM.CATALOG table to get schema information of the table 2. access the
>>>> meta table of HBase to get regions information of the table
>>>>
>>>> ----------------------------------------
>>>> Jaanai Zhang
>>>> Best regards!
>>>>
>>>>
>>>>
>>>> William Shen <wi...@marinsoftware.com> 于2019年1月31日周四 下午1:37写道：
>>>>
>>>>> Hi there,
>>>>>
>>>>> I have a component that makes Phoenix queries via the Phoenix JDBC
>>>>> Connection. I noticed that consistently, the Phoenix Client takes longer to
>>>>> execute a PreparedStatement and it takes longer to read through the
>>>>> ResultSet for a period of time (~15m) after a restart of the component. It
>>>>> seems like there is a warmup period for the JDBC connection. Is this to be
>>>>> expected?
>>>>>
>>>>> Thanks!
>>>>>
>>>>
Re: Phoenix JDBC Connection Warmup
Posted by James Taylor <ja...@apache.org>.
Have you tried setting UPDATE_CACHE_FREQUENCY on your tables?
On Fri, Feb 1, 2019 at 6:28 PM Jaanai Zhang <cl...@gmail.com> wrote:
> we experimented with issuing the same query repeatedly, and we observed a
>> slow down not only on the first query
>
> I am not sure what the reasons are, perhaps you can enable TRACE log to
> find what leads to slow, I guess that some meta information is reloaded
> under highly write workload.
>
> ----------------------------------------
> Jaanai Zhang
> Best regards!
>
>
>
> William Shen <wi...@marinsoftware.com> 于2019年2月1日周五 上午2:09写道：
>
>> Thanks Jaanai. Do you know if that is expected only on the first query
>> against a table? For us, we experimented with issuing the same query
>> repeatedly, and we observed a slow down not only on the first query. Does
>> it make sense to preemptively load table metadata on start up to warm up
>> the system to reduce latency during the actual query time (if it is
>> possible to do so)?
>>
>> On Wed, Jan 30, 2019 at 10:54 PM Jaanai Zhang <cl...@gmail.com>
>> wrote:
>>
>>> It is expected when firstly query tables after establishing the
>>> connection. Something likes loads some meta information into local cache
>>> that need take some time, mainly including two aspects: 1. access
>>> SYSTEM.CATALOG table to get schema information of the table 2. access the
>>> meta table of HBase to get regions information of the table
>>>
>>> ----------------------------------------
>>> Jaanai Zhang
>>> Best regards!
>>>
>>>
>>>
>>> William Shen <wi...@marinsoftware.com> 于2019年1月31日周四 下午1:37写道：
>>>
>>>> Hi there,
>>>>
>>>> I have a component that makes Phoenix queries via the Phoenix JDBC
>>>> Connection. I noticed that consistently, the Phoenix Client takes longer to
>>>> execute a PreparedStatement and it takes longer to read through the
>>>> ResultSet for a period of time (~15m) after a restart of the component. It
>>>> seems like there is a warmup period for the JDBC connection. Is this to be
>>>> expected?
>>>>
>>>> Thanks!
>>>>
>>>