You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by anupama agarwal <an...@gmail.com> on 2016/08/02 09:35:23 UTC
Phoenix upsert query time
Hi,
We need to insert to do 10K upserts per second in our phoenix table. We are
using phoenix 4.5 with CDH5.4.4. Problem is each upsert query is making
rpc call to read metadata about the table.
"o.a.phoenix.compile.FromCompiler.createTableRef 446 - Re-resolved stale
table"
Phoenix have added feature to specify "UPDATE_CACHE_FREQUENCY" in 4.7
version. Any way to apply this in Phoenix 4.5 by caching metadata at client
side? My table schema will change only once in 6 months. This would be
really helpful for me.
Regards
Anupama
Re: Phoenix upsert query time
Posted by James Taylor <ja...@apache.org>.
On Wed, Aug 3, 2016 at 11:17 AM, Heather, James (ELS) <
james.heather@elsevier.com> wrote:
> Can there be disastrous consequences of a high value for
> UPDATE_CACHE_FREQUENCY? What happens if a node pushes a change when it's
> got a stale cache?
>
There's some discussion of this on PHOENIX-2885.
Re: Phoenix upsert query time
Posted by "Heather, James (ELS)" <ja...@elsevier.com>.
Can there be disastrous consequences of a high value for UPDATE_CACHE_FREQUENCY? What happens if a node pushes a change when it's got a stale cache?
On 3 August 2016 16:23:15 James Taylor <ja...@apache.org> wrote:
Short of upgrading to 4.7 to leverage the UPDATE_CACHE_FREQUENCY feature, you can try setting the CURRENT_SCN property to Long.MAX_VALUE when you connect. Another alternative would be to set it to one more than the creation time of your tables. You can control the timestamp your tables are created at by setting the CURRENT_SCN property on the connection making the CREATE TABLE call. More on that here: https://phoenix.apache.org/faq.html#Can_phoenix_work_on_tables_with_arbitrary_timestamp_as_flexible_as_HBase_API
Thanks,
James
On Tue, Aug 2, 2016 at 2:59 PM, anil gupta <an...@gmail.com>> wrote:
How many nodes you have in cluster? How many regions in that phoenix table? Can you do batch upserts?
If Phoenix is querying for MetaData for every upsert in a preparedStatement then it definitely sounds like a bug/performance problem.
IMO, 30 ms is not really that horrible of a performance given that you also have a secondary index. Have you tried increasing number of write clients?
On Tue, Aug 2, 2016 at 10:54 AM, anupama agarwal <an...@gmail.com>> wrote:
I don't have option to update my CDH5.7. My upsert query is taking 30ms with one fully covered index on table.
I am using Spring JDBC template which uses prepared statement internally.
<https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
NamedParameterJdbcTemplate jdbcTemplate = jdbcTemplateMap.get(event.getNamespaceName());
jdbcTemplate.update(upsertQuery, event.getFieldMapPair()); Do I need to use explicit prepared statement? Link to code<https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
On Tue, Aug 2, 2016 at 10:31 PM, Anil Gupta <an...@gmail.com>> wrote:
Are you using a prepared statement for upserts? IMO, query should be compiled only once when prepared statement is used.
Sent from my iPhone
On Aug 2, 2016, at 7:56 AM, Samarth Jain <sa...@apache.org>> wrote:
Best bet is to updgrade your cloudera version to cdh5.7. It supports phoenix 4.7. See -
http://community.cloudera.com/t5/Cloudera-Labs/ANNOUNCE-Third-installment-of-Cloudera-Labs-packaging-of-Apache/m-p/42351#U42351
On Tuesday, August 2, 2016, anupama agarwal <an...@gmail.com>> wrote:
Hi,
We need to insert to do 10K upserts per second in our phoenix table. We are using phoenix 4.5 with CDH5.4.4. Problem is each upsert query is making rpc call to read metadata about the table.
"o.a.phoenix.compile.FromCompiler.createTableRef 446 - Re-resolved stale table"
Phoenix have added feature to specify "UPDATE_CACHE_FREQUENCY" in 4.7 version. Any way to apply this in Phoenix 4.5 by caching metadata at client side? My table schema will change only once in 6 months. This would be really helpful for me.
Regards
Anupama
--
Thanks & Regards,
Anil Gupta
________________________________
Elsevier Limited. Registered Office: The Boulevard, Langford Lane, Kidlington, Oxford, OX5 1GB, United Kingdom, Registration No. 1982084, Registered in England and Wales.
Re: Phoenix upsert query time
Posted by James Taylor <ja...@apache.org>.
I'm not sure it will help, but it's worth a try. When you need to update
the table schema, you wouldn't connect using Long.MAX_VALUE as the
CURRENT_SCN. These are all just ideas to experiment with until you can
upgrade to 4.7 or higher.
On Wed, Aug 3, 2016 at 12:30 PM, anupama agarwal <an...@gmail.com> wrote:
> Hi James,
>
> Thanks for your suggestion. I am a noob, I didn't understand how "you can
> try setting the CURRENT_SCN property to Long.MAX_VALUE when you connect."
> will help. Will this prevent phoenix from loading metadata about table at
> each call? How will I proceed when I actually need to update the table
> schema?
>
> On Wed, Aug 3, 2016 at 8:53 PM, James Taylor <ja...@apache.org>
> wrote:
>
>> Short of upgrading to 4.7 to leverage the UPDATE_CACHE_FREQUENCY
>> feature, you can try setting the CURRENT_SCN property to Long.MAX_VALUE
>> when you connect. Another alternative would be to set it to one more than
>> the creation time of your tables. You can control the timestamp your tables
>> are created at by setting the CURRENT_SCN property on the connection making
>> the CREATE TABLE call. More on that here:
>> https://phoenix.apache.org/faq.html#Can_phoenix_work_on_tables_with_arbitrary_timestamp_as_flexible_as_HBase_API
>>
>> Thanks,
>> James
>>
>> On Tue, Aug 2, 2016 at 2:59 PM, anil gupta <an...@gmail.com> wrote:
>>
>>> How many nodes you have in cluster? How many regions in that phoenix
>>> table? Can you do batch upserts?
>>> If Phoenix is querying for MetaData for every upsert in a
>>> preparedStatement then it definitely sounds like a bug/performance problem.
>>> IMO, 30 ms is not really that horrible of a performance given that you
>>> also have a secondary index. Have you tried increasing number of write
>>> clients?
>>>
>>> On Tue, Aug 2, 2016 at 10:54 AM, anupama agarwal <an...@gmail.com>
>>> wrote:
>>>
>>>> I don't have option to update my CDH5.7. My upsert query is taking 30ms
>>>> with one fully covered index on table.
>>>>
>>>> I am using Spring JDBC template which uses prepared statement
>>>> internally.
>>>>
>>>>
>>>> <https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
>>>>
>>>> NamedParameterJdbcTemplate jdbcTemplate = jdbcTemplateMap.get(event.
>>>> getNamespaceName());
>>>> jdbcTemplate.update(upsertQuery, event.getFieldMapPair()); Do I need
>>>> to use explicit prepared statement? Link to code
>>>> <https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
>>>>
>>>> On Tue, Aug 2, 2016 at 10:31 PM, Anil Gupta <an...@gmail.com>
>>>> wrote:
>>>>
>>>>> Are you using a prepared statement for upserts? IMO, query should be
>>>>> compiled only once when prepared statement is used.
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>> On Aug 2, 2016, at 7:56 AM, Samarth Jain <sa...@apache.org> wrote:
>>>>>
>>>>> Best bet is to updgrade your cloudera version to cdh5.7. It supports
>>>>> phoenix 4.7. See -
>>>>>
>>>>> http://community.cloudera.com/t5/Cloudera-Labs/ANNOUNCE-Third-installment-of-Cloudera-Labs-packaging-of-Apache/m-p/42351#U42351
>>>>>
>>>>>
>>>>> On Tuesday, August 2, 2016, anupama agarwal <an...@gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> We need to insert to do 10K upserts per second in our phoenix table.
>>>>>> We are using phoenix 4.5 with CDH5.4.4. Problem is each upsert query is
>>>>>> making rpc call to read metadata about the table.
>>>>>>
>>>>>> "o.a.phoenix.compile.FromCompiler.createTableRef 446 - Re-resolved
>>>>>> stale table"
>>>>>>
>>>>>> Phoenix have added feature to specify "UPDATE_CACHE_FREQUENCY" in 4.7
>>>>>> version. Any way to apply this in Phoenix 4.5 by caching metadata at client
>>>>>> side? My table schema will change only once in 6 months. This would be
>>>>>> really helpful for me.
>>>>>>
>>>>>> Regards
>>>>>> Anupama
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks & Regards,
>>> Anil Gupta
>>>
>>
>>
>
Re: Phoenix upsert query time
Posted by anupama agarwal <an...@gmail.com>.
Hi James,
Thanks for your suggestion. I am a noob, I didn't understand how "you can
try setting the CURRENT_SCN property to Long.MAX_VALUE when you connect."
will help. Will this prevent phoenix from loading metadata about table at
each call? How will I proceed when I actually need to update the table
schema?
On Wed, Aug 3, 2016 at 8:53 PM, James Taylor <ja...@apache.org> wrote:
> Short of upgrading to 4.7 to leverage the UPDATE_CACHE_FREQUENCY feature,
> you can try setting the CURRENT_SCN property to Long.MAX_VALUE when you
> connect. Another alternative would be to set it to one more than the
> creation time of your tables. You can control the timestamp your tables are
> created at by setting the CURRENT_SCN property on the connection making the
> CREATE TABLE call. More on that here:
> https://phoenix.apache.org/faq.html#Can_phoenix_work_on_tables_with_arbitrary_timestamp_as_flexible_as_HBase_API
>
> Thanks,
> James
>
> On Tue, Aug 2, 2016 at 2:59 PM, anil gupta <an...@gmail.com> wrote:
>
>> How many nodes you have in cluster? How many regions in that phoenix
>> table? Can you do batch upserts?
>> If Phoenix is querying for MetaData for every upsert in a
>> preparedStatement then it definitely sounds like a bug/performance problem.
>> IMO, 30 ms is not really that horrible of a performance given that you
>> also have a secondary index. Have you tried increasing number of write
>> clients?
>>
>> On Tue, Aug 2, 2016 at 10:54 AM, anupama agarwal <an...@gmail.com>
>> wrote:
>>
>>> I don't have option to update my CDH5.7. My upsert query is taking 30ms
>>> with one fully covered index on table.
>>>
>>> I am using Spring JDBC template which uses prepared statement
>>> internally.
>>>
>>>
>>> <https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
>>>
>>> NamedParameterJdbcTemplate jdbcTemplate = jdbcTemplateMap.get(event.
>>> getNamespaceName());
>>> jdbcTemplate.update(upsertQuery, event.getFieldMapPair()); Do I need to
>>> use explicit prepared statement? Link to code
>>> <https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
>>>
>>> On Tue, Aug 2, 2016 at 10:31 PM, Anil Gupta <an...@gmail.com>
>>> wrote:
>>>
>>>> Are you using a prepared statement for upserts? IMO, query should be
>>>> compiled only once when prepared statement is used.
>>>>
>>>> Sent from my iPhone
>>>>
>>>> On Aug 2, 2016, at 7:56 AM, Samarth Jain <sa...@apache.org> wrote:
>>>>
>>>> Best bet is to updgrade your cloudera version to cdh5.7. It supports
>>>> phoenix 4.7. See -
>>>>
>>>> http://community.cloudera.com/t5/Cloudera-Labs/ANNOUNCE-Third-installment-of-Cloudera-Labs-packaging-of-Apache/m-p/42351#U42351
>>>>
>>>>
>>>> On Tuesday, August 2, 2016, anupama agarwal <an...@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> We need to insert to do 10K upserts per second in our phoenix table.
>>>>> We are using phoenix 4.5 with CDH5.4.4. Problem is each upsert query is
>>>>> making rpc call to read metadata about the table.
>>>>>
>>>>> "o.a.phoenix.compile.FromCompiler.createTableRef 446 - Re-resolved
>>>>> stale table"
>>>>>
>>>>> Phoenix have added feature to specify "UPDATE_CACHE_FREQUENCY" in 4.7
>>>>> version. Any way to apply this in Phoenix 4.5 by caching metadata at client
>>>>> side? My table schema will change only once in 6 months. This would be
>>>>> really helpful for me.
>>>>>
>>>>> Regards
>>>>> Anupama
>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>
>>
>> --
>> Thanks & Regards,
>> Anil Gupta
>>
>
>
Re: Phoenix upsert query time
Posted by James Taylor <ja...@apache.org>.
Short of upgrading to 4.7 to leverage the UPDATE_CACHE_FREQUENCY feature,
you can try setting the CURRENT_SCN property to Long.MAX_VALUE when you
connect. Another alternative would be to set it to one more than the
creation time of your tables. You can control the timestamp your tables are
created at by setting the CURRENT_SCN property on the connection making the
CREATE TABLE call. More on that here:
https://phoenix.apache.org/faq.html#Can_phoenix_work_on_tables_with_arbitrary_timestamp_as_flexible_as_HBase_API
Thanks,
James
On Tue, Aug 2, 2016 at 2:59 PM, anil gupta <an...@gmail.com> wrote:
> How many nodes you have in cluster? How many regions in that phoenix
> table? Can you do batch upserts?
> If Phoenix is querying for MetaData for every upsert in a
> preparedStatement then it definitely sounds like a bug/performance problem.
> IMO, 30 ms is not really that horrible of a performance given that you
> also have a secondary index. Have you tried increasing number of write
> clients?
>
> On Tue, Aug 2, 2016 at 10:54 AM, anupama agarwal <an...@gmail.com>
> wrote:
>
>> I don't have option to update my CDH5.7. My upsert query is taking 30ms
>> with one fully covered index on table.
>>
>> I am using Spring JDBC template which uses prepared statement internally.
>>
>>
>> <https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
>>
>> NamedParameterJdbcTemplate jdbcTemplate = jdbcTemplateMap.get(event.
>> getNamespaceName());
>> jdbcTemplate.update(upsertQuery, event.getFieldMapPair()); Do I need to
>> use explicit prepared statement? Link to code
>> <https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
>>
>> On Tue, Aug 2, 2016 at 10:31 PM, Anil Gupta <an...@gmail.com>
>> wrote:
>>
>>> Are you using a prepared statement for upserts? IMO, query should be
>>> compiled only once when prepared statement is used.
>>>
>>> Sent from my iPhone
>>>
>>> On Aug 2, 2016, at 7:56 AM, Samarth Jain <sa...@apache.org> wrote:
>>>
>>> Best bet is to updgrade your cloudera version to cdh5.7. It supports
>>> phoenix 4.7. See -
>>>
>>> http://community.cloudera.com/t5/Cloudera-Labs/ANNOUNCE-Third-installment-of-Cloudera-Labs-packaging-of-Apache/m-p/42351#U42351
>>>
>>>
>>> On Tuesday, August 2, 2016, anupama agarwal <an...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> We need to insert to do 10K upserts per second in our phoenix table. We
>>>> are using phoenix 4.5 with CDH5.4.4. Problem is each upsert query is
>>>> making rpc call to read metadata about the table.
>>>>
>>>> "o.a.phoenix.compile.FromCompiler.createTableRef 446 - Re-resolved
>>>> stale table"
>>>>
>>>> Phoenix have added feature to specify "UPDATE_CACHE_FREQUENCY" in 4.7
>>>> version. Any way to apply this in Phoenix 4.5 by caching metadata at client
>>>> side? My table schema will change only once in 6 months. This would be
>>>> really helpful for me.
>>>>
>>>> Regards
>>>> Anupama
>>>>
>>>>
>>>>
>>>>
>>
>
>
> --
> Thanks & Regards,
> Anil Gupta
>
Re: Phoenix upsert query time
Posted by anupama agarwal <an...@gmail.com>.
I have 10 region servers, 16 regions in the table. I cannot do batch
upserts. Yes, phoenix is querying metadata for every upsert. 30 ms is not a
bad response, but I wanted to understand if I want to remove these extra
queries, what options I have?
On Wed, Aug 3, 2016 at 3:29 AM, anil gupta <an...@gmail.com> wrote:
> How many nodes you have in cluster? How many regions in that phoenix
> table? Can you do batch upserts?
> If Phoenix is querying for MetaData for every upsert in a
> preparedStatement then it definitely sounds like a bug/performance problem.
> IMO, 30 ms is not really that horrible of a performance given that you
> also have a secondary index. Have you tried increasing number of write
> clients?
>
> On Tue, Aug 2, 2016 at 10:54 AM, anupama agarwal <an...@gmail.com>
> wrote:
>
>> I don't have option to update my CDH5.7. My upsert query is taking 30ms
>> with one fully covered index on table.
>>
>> I am using Spring JDBC template which uses prepared statement internally.
>>
>>
>> <https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
>>
>> NamedParameterJdbcTemplate jdbcTemplate = jdbcTemplateMap.get(event.
>> getNamespaceName());
>> jdbcTemplate.update(upsertQuery, event.getFieldMapPair()); Do I need to
>> use explicit prepared statement? Link to code
>> <https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
>>
>> On Tue, Aug 2, 2016 at 10:31 PM, Anil Gupta <an...@gmail.com>
>> wrote:
>>
>>> Are you using a prepared statement for upserts? IMO, query should be
>>> compiled only once when prepared statement is used.
>>>
>>> Sent from my iPhone
>>>
>>> On Aug 2, 2016, at 7:56 AM, Samarth Jain <sa...@apache.org> wrote:
>>>
>>> Best bet is to updgrade your cloudera version to cdh5.7. It supports
>>> phoenix 4.7. See -
>>>
>>> http://community.cloudera.com/t5/Cloudera-Labs/ANNOUNCE-Third-installment-of-Cloudera-Labs-packaging-of-Apache/m-p/42351#U42351
>>>
>>>
>>> On Tuesday, August 2, 2016, anupama agarwal <an...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> We need to insert to do 10K upserts per second in our phoenix table. We
>>>> are using phoenix 4.5 with CDH5.4.4. Problem is each upsert query is
>>>> making rpc call to read metadata about the table.
>>>>
>>>> "o.a.phoenix.compile.FromCompiler.createTableRef 446 - Re-resolved
>>>> stale table"
>>>>
>>>> Phoenix have added feature to specify "UPDATE_CACHE_FREQUENCY" in 4.7
>>>> version. Any way to apply this in Phoenix 4.5 by caching metadata at client
>>>> side? My table schema will change only once in 6 months. This would be
>>>> really helpful for me.
>>>>
>>>> Regards
>>>> Anupama
>>>>
>>>>
>>>>
>>>>
>>
>
>
> --
> Thanks & Regards,
> Anil Gupta
>
Re: Phoenix upsert query time
Posted by anil gupta <an...@gmail.com>.
How many nodes you have in cluster? How many regions in that phoenix table?
Can you do batch upserts?
If Phoenix is querying for MetaData for every upsert in a preparedStatement
then it definitely sounds like a bug/performance problem.
IMO, 30 ms is not really that horrible of a performance given that you also
have a secondary index. Have you tried increasing number of write clients?
On Tue, Aug 2, 2016 at 10:54 AM, anupama agarwal <an...@gmail.com> wrote:
> I don't have option to update my CDH5.7. My upsert query is taking 30ms
> with one fully covered index on table.
>
> I am using Spring JDBC template which uses prepared statement internally.
>
>
> <https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
>
> NamedParameterJdbcTemplate jdbcTemplate = jdbcTemplateMap.get(event.
> getNamespaceName());
> jdbcTemplate.update(upsertQuery, event.getFieldMapPair()); Do I need to
> use explicit prepared statement? Link to code
> <https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
>
> On Tue, Aug 2, 2016 at 10:31 PM, Anil Gupta <an...@gmail.com> wrote:
>
>> Are you using a prepared statement for upserts? IMO, query should be
>> compiled only once when prepared statement is used.
>>
>> Sent from my iPhone
>>
>> On Aug 2, 2016, at 7:56 AM, Samarth Jain <sa...@apache.org> wrote:
>>
>> Best bet is to updgrade your cloudera version to cdh5.7. It supports
>> phoenix 4.7. See -
>>
>> http://community.cloudera.com/t5/Cloudera-Labs/ANNOUNCE-Third-installment-of-Cloudera-Labs-packaging-of-Apache/m-p/42351#U42351
>>
>>
>> On Tuesday, August 2, 2016, anupama agarwal <an...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> We need to insert to do 10K upserts per second in our phoenix table. We
>>> are using phoenix 4.5 with CDH5.4.4. Problem is each upsert query is
>>> making rpc call to read metadata about the table.
>>>
>>> "o.a.phoenix.compile.FromCompiler.createTableRef 446 - Re-resolved stale
>>> table"
>>>
>>> Phoenix have added feature to specify "UPDATE_CACHE_FREQUENCY" in 4.7
>>> version. Any way to apply this in Phoenix 4.5 by caching metadata at client
>>> side? My table schema will change only once in 6 months. This would be
>>> really helpful for me.
>>>
>>> Regards
>>> Anupama
>>>
>>>
>>>
>>>
>
--
Thanks & Regards,
Anil Gupta
Re: Phoenix upsert query time
Posted by anupama agarwal <an...@gmail.com>.
I don't have option to update my CDH5.7. My upsert query is taking 30ms
with one fully covered index on table.
I am using Spring JDBC template which uses prepared statement internally.
<https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
NamedParameterJdbcTemplate jdbcTemplate = jdbcTemplateMap.get(event.
getNamespaceName());
jdbcTemplate.update(upsertQuery, event.getFieldMapPair()); Do I need to use
explicit prepared statement? Link to code
<https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
On Tue, Aug 2, 2016 at 10:31 PM, Anil Gupta <an...@gmail.com> wrote:
> Are you using a prepared statement for upserts? IMO, query should be
> compiled only once when prepared statement is used.
>
> Sent from my iPhone
>
> On Aug 2, 2016, at 7:56 AM, Samarth Jain <sa...@apache.org> wrote:
>
> Best bet is to updgrade your cloudera version to cdh5.7. It supports
> phoenix 4.7. See -
>
> http://community.cloudera.com/t5/Cloudera-Labs/ANNOUNCE-Third-installment-of-Cloudera-Labs-packaging-of-Apache/m-p/42351#U42351
>
>
> On Tuesday, August 2, 2016, anupama agarwal <an...@gmail.com> wrote:
>
>> Hi,
>>
>> We need to insert to do 10K upserts per second in our phoenix table. We
>> are using phoenix 4.5 with CDH5.4.4. Problem is each upsert query is
>> making rpc call to read metadata about the table.
>>
>> "o.a.phoenix.compile.FromCompiler.createTableRef 446 - Re-resolved stale
>> table"
>>
>> Phoenix have added feature to specify "UPDATE_CACHE_FREQUENCY" in 4.7
>> version. Any way to apply this in Phoenix 4.5 by caching metadata at client
>> side? My table schema will change only once in 6 months. This would be
>> really helpful for me.
>>
>> Regards
>> Anupama
>>
>>
>>
>>
Re: Phoenix upsert query time
Posted by Anil Gupta <an...@gmail.com>.
Are you using a prepared statement for upserts? IMO, query should be compiled only once when prepared statement is used.
Sent from my iPhone
> On Aug 2, 2016, at 7:56 AM, Samarth Jain <sa...@apache.org> wrote:
>
> Best bet is to updgrade your cloudera version to cdh5.7. It supports phoenix 4.7. See -
> http://community.cloudera.com/t5/Cloudera-Labs/ANNOUNCE-Third-installment-of-Cloudera-Labs-packaging-of-Apache/m-p/42351#U42351
>
>
>> On Tuesday, August 2, 2016, anupama agarwal <an...@gmail.com> wrote:
>> Hi,
>>
>> We need to insert to do 10K upserts per second in our phoenix table. We are using phoenix 4.5 with CDH5.4.4. Problem is each upsert query is making rpc call to read metadata about the table.
>>
>> "o.a.phoenix.compile.FromCompiler.createTableRef 446 - Re-resolved stale table"
>>
>> Phoenix have added feature to specify "UPDATE_CACHE_FREQUENCY" in 4.7 version. Any way to apply this in Phoenix 4.5 by caching metadata at client side? My table schema will change only once in 6 months. This would be really helpful for me.
>>
>> Regards
>> Anupama
Re: Phoenix upsert query time
Posted by Samarth Jain <sa...@apache.org>.
Best bet is to updgrade your cloudera version to cdh5.7. It supports
phoenix 4.7. See -
http://community.cloudera.com/t5/Cloudera-Labs/ANNOUNCE-Third-installment-of-Cloudera-Labs-packaging-of-Apache/m-p/42351#U42351
On Tuesday, August 2, 2016, anupama agarwal <an...@gmail.com> wrote:
> Hi,
>
> We need to insert to do 10K upserts per second in our phoenix table. We
> are using phoenix 4.5 with CDH5.4.4. Problem is each upsert query is
> making rpc call to read metadata about the table.
>
> "o.a.phoenix.compile.FromCompiler.createTableRef 446 - Re-resolved stale
> table"
>
> Phoenix have added feature to specify "UPDATE_CACHE_FREQUENCY" in 4.7
> version. Any way to apply this in Phoenix 4.5 by caching metadata at client
> side? My table schema will change only once in 6 months. This would be
> really helpful for me.
>
> Regards
> Anupama
>
>
>
>