You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by Manthosh Kumar T <ma...@gmail.com> on 2014/03/17 13:39:20 UTC
Fwd: Improve upsert performance - JDBC
Hi All,
Currently when I upsert values via JDBC, each upsertion takes
14-20 ms on an average. Auto commit is set to true. Is there a way to
improve this without batch insert?.
--
Cheers,
Manthosh Kumar. T
Re: Improve upsert performance - JDBC
Posted by Manthosh Kumar T <ma...@gmail.com>.
Hi James,
Thanks again. That's my problem. If I have a threshold, at the
time of reading I may miss few Objects which has been received at that
time. So I thought time is a better option, please correct me if I'm wrong.
On 20 March 2014 10:14, James Taylor <ja...@apache.org> wrote:
> Best would be to track how many rows you've received and commit once a
> threshold is reached (as it sounds like the rate at which objects are
> received varies pretty greatly).
> Thanks,
> James
>
>
> On Wed, Mar 19, 2014 at 6:12 PM, Manthosh Kumar T <ma...@gmail.com>wrote:
>
>> Hi James,
>> Thanks. I'm using Phoenix with Storm. Say I need to
>> store objects in Phoenix from a source. The rate at which this source emits
>> the object ranges from 4000/second to 4/second. Another application needs
>> to read from this table concurrently and needs to know the total objects
>> received till that time. That's why I had set auto commit as true. So to
>> improve performance, is it efficient to perform commit based on time (say
>> for every second)?
>> On 17 Mar 2014 22:24, "James Taylor" <ja...@apache.org> wrote:
>>
>>> Hi Manthosh,
>>> Probably not a lot you can do. If it's the same upsert statement, you
>>> can cache the PreparedStatement to avoid the parse cost (but I doubt
>>> that'll help).
>>>
>>> Any reason why you can't turn auto commit off and commit in batches?
>>>
>>> Thanks,
>>> James
>>>
>>>
>>> On Mon, Mar 17, 2014 at 5:39 AM, Manthosh Kumar T <ma...@gmail.com>wrote:
>>>
>>>> Hi All,
>>>> Currently when I upsert values via JDBC, each upsertion takes
>>>> 14-20 ms on an average. Auto commit is set to true. Is there a way to
>>>> improve this without batch insert?.
>>>>
>>>> --
>>>> Cheers,
>>>> Manthosh Kumar. T
>>>>
>>>
>>>
>
--
Cheers,
Manthosh Kumar. T
Re: Improve upsert performance - JDBC
Posted by James Taylor <ja...@apache.org>.
Best would be to track how many rows you've received and commit once a
threshold is reached (as it sounds like the rate at which objects are
received varies pretty greatly).
Thanks,
James
On Wed, Mar 19, 2014 at 6:12 PM, Manthosh Kumar T <ma...@gmail.com>wrote:
> Hi James,
> Thanks. I'm using Phoenix with Storm. Say I need to store
> objects in Phoenix from a source. The rate at which this source emits the
> object ranges from 4000/second to 4/second. Another application needs to
> read from this table concurrently and needs to know the total objects
> received till that time. That's why I had set auto commit as true. So to
> improve performance, is it efficient to perform commit based on time (say
> for every second)?
> On 17 Mar 2014 22:24, "James Taylor" <ja...@apache.org> wrote:
>
>> Hi Manthosh,
>> Probably not a lot you can do. If it's the same upsert statement, you can
>> cache the PreparedStatement to avoid the parse cost (but I doubt that'll
>> help).
>>
>> Any reason why you can't turn auto commit off and commit in batches?
>>
>> Thanks,
>> James
>>
>>
>> On Mon, Mar 17, 2014 at 5:39 AM, Manthosh Kumar T <ma...@gmail.com>wrote:
>>
>>> Hi All,
>>> Currently when I upsert values via JDBC, each upsertion takes
>>> 14-20 ms on an average. Auto commit is set to true. Is there a way to
>>> improve this without batch insert?.
>>>
>>> --
>>> Cheers,
>>> Manthosh Kumar. T
>>>
>>
>>
Re: Improve upsert performance - JDBC
Posted by Manthosh Kumar T <ma...@gmail.com>.
Hi James,
Thanks. I'm using Phoenix with Storm. Say I need to store
objects in Phoenix from a source. The rate at which this source emits the
object ranges from 4000/second to 4/second. Another application needs to
read from this table concurrently and needs to know the total objects
received till that time. That's why I had set auto commit as true. So to
improve performance, is it efficient to perform commit based on time (say
for every second)?
On 17 Mar 2014 22:24, "James Taylor" <ja...@apache.org> wrote:
> Hi Manthosh,
> Probably not a lot you can do. If it's the same upsert statement, you can
> cache the PreparedStatement to avoid the parse cost (but I doubt that'll
> help).
>
> Any reason why you can't turn auto commit off and commit in batches?
>
> Thanks,
> James
>
>
> On Mon, Mar 17, 2014 at 5:39 AM, Manthosh Kumar T <ma...@gmail.com>wrote:
>
>> Hi All,
>> Currently when I upsert values via JDBC, each upsertion takes
>> 14-20 ms on an average. Auto commit is set to true. Is there a way to
>> improve this without batch insert?.
>>
>> --
>> Cheers,
>> Manthosh Kumar. T
>>
>
>
Re: Improve upsert performance - JDBC
Posted by James Taylor <ja...@apache.org>.
Hi Manthosh,
Probably not a lot you can do. If it's the same upsert statement, you can
cache the PreparedStatement to avoid the parse cost (but I doubt that'll
help).
Any reason why you can't turn auto commit off and commit in batches?
Thanks,
James
On Mon, Mar 17, 2014 at 5:39 AM, Manthosh Kumar T <ma...@gmail.com>wrote:
> Hi All,
> Currently when I upsert values via JDBC, each upsertion takes
> 14-20 ms on an average. Auto commit is set to true. Is there a way to
> improve this without batch insert?.
>
> --
> Cheers,
> Manthosh Kumar. T
>