You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Chris Paterson <cj...@googlemail.com> on 2008/08/12 15:09:07 UTC
Slow inserts
Hey all,
I'm trying to insert a large number of rows into my table (about 5000)
and it's proving very slow - ~2 seconds / row. The data's very simple,
with IP addresses being used as the row key and a single 10-character
string as the data (under a single column family, "Details", and
column, "PhoneNo"). There's also a timestamp (long) representing time
since the epoch.
Below's the program I'm running to insert:
Text text = new Text(rowKey);
long lockid = tbl.startUpdate(new Text(rowKey));
tbl.put(lockid, new Text("Details:PhoneNo"), new Text(data));
tbl.commit(lockid, timestamp);
If anyone can see anything wrong with that I'm doing, please let me know!
Cheers.
Re: Slow inserts
Posted by Jean-Daniel Cryans <jd...@gmail.com>.
Chris,
Glad to see it works good for you. If there is anything you need that you
think HBase should offer, feel free to fill a ticket in our Jira:
http://issues.apache.org/jira/browse/HBASE
J-D
On Sat, Aug 16, 2008 at 11:11 AM, Chris Paterson <cjrpaterson@googlemail.com
> wrote:
> Hey,
>
> Cheers for that, it works with the => instead of a = sign
>
> Chris
>
> On Fri, Aug 15, 2008 at 8:24 PM, stack <st...@duboce.net> wrote:
>
> > Chris Paterson wrote:
> >
> >> Hmm, just as a follow up, using this command:
> >>
> >> get 'iptable', '192.168.0.1', {COLUMN => 'details:phone', TIMESTAMP =>
> >> 1218532708269, VERSIONS = 4}
> >>
> >>
> >
> > If you do 'VERSIONS => 4' instead of 'VERSIONS = 4', does it work? (I
> > fixed the hirb help where our example looked like the above).
> > St.Ack
> >
> >
> >
> >
> > Returns "SyntaxError: (hbase):8: , unexpected tRCURLY" -- is there
> >> something I'm doing wrong?
> >>
> >> Cheers.
> >>
> >> On 8/12/08, Chris Paterson <cj...@googlemail.com> wrote:
> >>
> >>
> >>> Thanks Jean, much obliged. I don't suppose you know if this will be
> >>> changed
> >>> in the future, so you're not required to enter a column?
> >>>
> >>> Cheers.
> >>>
> >>>
> >>> On Tue, Aug 12, 2008 at 6:14 PM, Jean-Daniel Cryans
> >>> <jd...@gmail.com>wrote:
> >>>
> >>>
> >>>
> >>>> Chris,
> >>>>
> >>>> API docs :
> >>>> http://hadoop.apache.org/hbase/docs/current/api/index.htmltaken
> >>>> from http://hadoop.apache.org/hbase/docs/current/
> >>>>
> >>>> For your command, for the moment you really need to specify columns if
> >>>> you
> >>>> want versions.
> >>>>
> >>>> J-D
> >>>>
> >>>> On Tue, Aug 12, 2008 at 1:03 PM, Chris Paterson
> >>>> <cj...@googlemail.com>wrote:
> >>>>
> >>>>
> >>>>
> >>>>> Hi Jean,
> >>>>>
> >>>>> I was running Hadoop 16.4 / HBase 0.1.3 on a single VMWare node
> >>>>> (Ubuntu).
> >>>>> However having upgraded to Hadoop 17.1 / HBase 0.2.0 on the same hw
> it
> >>>>> inserts all of the rows in < 2 seconds - problem solved!
> >>>>>
> >>>>> Also, do you know where can I find the API docs for 0.2.0?
> >>>>>
> >>>>> Finally I'm having trouble with the syntax in the new shell for
> >>>>>
> >>>>>
> >>>> retrieving
> >>>>
> >>>>
> >>>>> the latest 5 versions of a row. Off the top of my head (I'm away from
> >>>>> my
> >>>>> desk now) I'm using "get 'iptable', '192.168.1.1', VERSIONS = 5" but
> >>>>>
> >>>>>
> >>>> this
> >>>>
> >>>>
> >>>>> doesn't seem to work (removing the versions part does).
> >>>>>
> >>>>> Cheers,
> >>>>> Chris
> >>>>>
> >>>>>
> >>>>> On Tue, Aug 12, 2008 at 2:28 PM, Jean-Daniel Cryans <
> >>>>> jdcryans@gmail.com
> >>>>>
> >>>>>
> >>>>>> wrote:
> >>>>>> Chris,
> >>>>>>
> >>>>>> It really seems slow yes. What's your setup like? # of machines, how
> >>>>>>
> >>>>>>
> >>>>> the
> >>>>
> >>>>
> >>>>> soft is installed, which version of hadoop/hbase, etc?
> >>>>>>
> >>>>>> Thx,
> >>>>>>
> >>>>>> J-D
> >>>>>>
> >>>>>> On Tue, Aug 12, 2008 at 9:09 AM, Chris Paterson
> >>>>>> <cj...@googlemail.com>wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> Hey all,
> >>>>>>>
> >>>>>>> I'm trying to insert a large number of rows into my table (about
> >>>>>>>
> >>>>>>>
> >>>>>> 5000)
> >>>>
> >>>>
> >>>>> and it's proving very slow - ~2 seconds / row. The data's very
> >>>>>>>
> >>>>>>>
> >>>>>> simple,
> >>>>
> >>>>
> >>>>> with IP addresses being used as the row key and a single
> >>>>>>> 10-character
> >>>>>>> string as the data (under a single column family, "Details", and
> >>>>>>> column, "PhoneNo"). There's also a timestamp (long) representing
> >>>>>>> time
> >>>>>>> since the epoch.
> >>>>>>>
> >>>>>>> Below's the program I'm running to insert:
> >>>>>>>
> >>>>>>> Text text = new Text(rowKey);
> >>>>>>> long lockid = tbl.startUpdate(new Text(rowKey));
> >>>>>>> tbl.put(lockid, new Text("Details:PhoneNo"), new Text(data));
> >>>>>>> tbl.commit(lockid, timestamp);
> >>>>>>>
> >>>>>>> If anyone can see anything wrong with that I'm doing, please let me
> >>>>>>>
> >>>>>>>
> >>>>>> know!
> >>>>>
> >>>>>
> >>>>>> Cheers.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >
>
Re: Slow inserts
Posted by Chris Paterson <cj...@googlemail.com>.
Hey,
Cheers for that, it works with the => instead of a = sign.
Chris
On Fri, Aug 15, 2008 at 8:24 PM, stack <st...@duboce.net> wrote:
> Chris Paterson wrote:
>
>> Hmm, just as a follow up, using this command:
>>
>> get 'iptable', '192.168.0.1', {COLUMN => 'details:phone', TIMESTAMP =>
>> 1218532708269, VERSIONS = 4}
>>
>>
>
> If you do 'VERSIONS => 4' instead of 'VERSIONS = 4', does it work? (I
> fixed the hirb help where our example looked like the above).
> St.Ack
>
>
>
>
> Returns "SyntaxError: (hbase):8: , unexpected tRCURLY" -- is there
>> something I'm doing wrong?
>>
>> Cheers.
>>
>> On 8/12/08, Chris Paterson <cj...@googlemail.com> wrote:
>>
>>
>>> Thanks Jean, much obliged. I don't suppose you know if this will be
>>> changed
>>> in the future, so you're not required to enter a column?
>>>
>>> Cheers.
>>>
>>>
>>> On Tue, Aug 12, 2008 at 6:14 PM, Jean-Daniel Cryans
>>> <jd...@gmail.com>wrote:
>>>
>>>
>>>
>>>> Chris,
>>>>
>>>> API docs :
>>>> http://hadoop.apache.org/hbase/docs/current/api/index.htmltaken
>>>> from http://hadoop.apache.org/hbase/docs/current/
>>>>
>>>> For your command, for the moment you really need to specify columns if
>>>> you
>>>> want versions.
>>>>
>>>> J-D
>>>>
>>>> On Tue, Aug 12, 2008 at 1:03 PM, Chris Paterson
>>>> <cj...@googlemail.com>wrote:
>>>>
>>>>
>>>>
>>>>> Hi Jean,
>>>>>
>>>>> I was running Hadoop 16.4 / HBase 0.1.3 on a single VMWare node
>>>>> (Ubuntu).
>>>>> However having upgraded to Hadoop 17.1 / HBase 0.2.0 on the same hw it
>>>>> inserts all of the rows in < 2 seconds - problem solved!
>>>>>
>>>>> Also, do you know where can I find the API docs for 0.2.0?
>>>>>
>>>>> Finally I'm having trouble with the syntax in the new shell for
>>>>>
>>>>>
>>>> retrieving
>>>>
>>>>
>>>>> the latest 5 versions of a row. Off the top of my head (I'm away from
>>>>> my
>>>>> desk now) I'm using "get 'iptable', '192.168.1.1', VERSIONS = 5" but
>>>>>
>>>>>
>>>> this
>>>>
>>>>
>>>>> doesn't seem to work (removing the versions part does).
>>>>>
>>>>> Cheers,
>>>>> Chris
>>>>>
>>>>>
>>>>> On Tue, Aug 12, 2008 at 2:28 PM, Jean-Daniel Cryans <
>>>>> jdcryans@gmail.com
>>>>>
>>>>>
>>>>>> wrote:
>>>>>> Chris,
>>>>>>
>>>>>> It really seems slow yes. What's your setup like? # of machines, how
>>>>>>
>>>>>>
>>>>> the
>>>>
>>>>
>>>>> soft is installed, which version of hadoop/hbase, etc?
>>>>>>
>>>>>> Thx,
>>>>>>
>>>>>> J-D
>>>>>>
>>>>>> On Tue, Aug 12, 2008 at 9:09 AM, Chris Paterson
>>>>>> <cj...@googlemail.com>wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Hey all,
>>>>>>>
>>>>>>> I'm trying to insert a large number of rows into my table (about
>>>>>>>
>>>>>>>
>>>>>> 5000)
>>>>
>>>>
>>>>> and it's proving very slow - ~2 seconds / row. The data's very
>>>>>>>
>>>>>>>
>>>>>> simple,
>>>>
>>>>
>>>>> with IP addresses being used as the row key and a single
>>>>>>> 10-character
>>>>>>> string as the data (under a single column family, "Details", and
>>>>>>> column, "PhoneNo"). There's also a timestamp (long) representing
>>>>>>> time
>>>>>>> since the epoch.
>>>>>>>
>>>>>>> Below's the program I'm running to insert:
>>>>>>>
>>>>>>> Text text = new Text(rowKey);
>>>>>>> long lockid = tbl.startUpdate(new Text(rowKey));
>>>>>>> tbl.put(lockid, new Text("Details:PhoneNo"), new Text(data));
>>>>>>> tbl.commit(lockid, timestamp);
>>>>>>>
>>>>>>> If anyone can see anything wrong with that I'm doing, please let me
>>>>>>>
>>>>>>>
>>>>>> know!
>>>>>
>>>>>
>>>>>> Cheers.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>
Re: Slow inserts
Posted by stack <st...@duboce.net>.
Chris Paterson wrote:
> Hmm, just as a follow up, using this command:
>
> get 'iptable', '192.168.0.1', {COLUMN => 'details:phone', TIMESTAMP =>
> 1218532708269, VERSIONS = 4}
>
If you do 'VERSIONS => 4' instead of 'VERSIONS = 4', does it work? (I
fixed the hirb help where our example looked like the above).
St.Ack
> Returns "SyntaxError: (hbase):8: , unexpected tRCURLY" -- is there
> something I'm doing wrong?
>
> Cheers.
>
> On 8/12/08, Chris Paterson <cj...@googlemail.com> wrote:
>
>> Thanks Jean, much obliged. I don't suppose you know if this will be changed
>> in the future, so you're not required to enter a column?
>>
>> Cheers.
>>
>>
>> On Tue, Aug 12, 2008 at 6:14 PM, Jean-Daniel Cryans
>> <jd...@gmail.com>wrote:
>>
>>
>>> Chris,
>>>
>>> API docs : http://hadoop.apache.org/hbase/docs/current/api/index.htmltaken
>>> from http://hadoop.apache.org/hbase/docs/current/
>>>
>>> For your command, for the moment you really need to specify columns if you
>>> want versions.
>>>
>>> J-D
>>>
>>> On Tue, Aug 12, 2008 at 1:03 PM, Chris Paterson
>>> <cj...@googlemail.com>wrote:
>>>
>>>
>>>> Hi Jean,
>>>>
>>>> I was running Hadoop 16.4 / HBase 0.1.3 on a single VMWare node
>>>> (Ubuntu).
>>>> However having upgraded to Hadoop 17.1 / HBase 0.2.0 on the same hw it
>>>> inserts all of the rows in < 2 seconds - problem solved!
>>>>
>>>> Also, do you know where can I find the API docs for 0.2.0?
>>>>
>>>> Finally I'm having trouble with the syntax in the new shell for
>>>>
>>> retrieving
>>>
>>>> the latest 5 versions of a row. Off the top of my head (I'm away from my
>>>> desk now) I'm using "get 'iptable', '192.168.1.1', VERSIONS = 5" but
>>>>
>>> this
>>>
>>>> doesn't seem to work (removing the versions part does).
>>>>
>>>> Cheers,
>>>> Chris
>>>>
>>>>
>>>> On Tue, Aug 12, 2008 at 2:28 PM, Jean-Daniel Cryans <jdcryans@gmail.com
>>>>
>>>>> wrote:
>>>>>
>>>>> Chris,
>>>>>
>>>>> It really seems slow yes. What's your setup like? # of machines, how
>>>>>
>>> the
>>>
>>>>> soft is installed, which version of hadoop/hbase, etc?
>>>>>
>>>>> Thx,
>>>>>
>>>>> J-D
>>>>>
>>>>> On Tue, Aug 12, 2008 at 9:09 AM, Chris Paterson
>>>>> <cj...@googlemail.com>wrote:
>>>>>
>>>>>
>>>>>> Hey all,
>>>>>>
>>>>>> I'm trying to insert a large number of rows into my table (about
>>>>>>
>>> 5000)
>>>
>>>>>> and it's proving very slow - ~2 seconds / row. The data's very
>>>>>>
>>> simple,
>>>
>>>>>> with IP addresses being used as the row key and a single
>>>>>> 10-character
>>>>>> string as the data (under a single column family, "Details", and
>>>>>> column, "PhoneNo"). There's also a timestamp (long) representing
>>>>>> time
>>>>>> since the epoch.
>>>>>>
>>>>>> Below's the program I'm running to insert:
>>>>>>
>>>>>> Text text = new Text(rowKey);
>>>>>> long lockid = tbl.startUpdate(new Text(rowKey));
>>>>>> tbl.put(lockid, new Text("Details:PhoneNo"), new Text(data));
>>>>>> tbl.commit(lockid, timestamp);
>>>>>>
>>>>>> If anyone can see anything wrong with that I'm doing, please let me
>>>>>>
>>>> know!
>>>>
>>>>>> Cheers.
>>>>>>
>>>>>>
Re: Slow inserts
Posted by Chris Paterson <cj...@googlemail.com>.
Hmm, just as a follow up, using this command:
get 'iptable', '192.168.0.1', {COLUMN => 'details:phone', TIMESTAMP =>
1218532708269, VERSIONS = 4}
Returns "SyntaxError: (hbase):8: , unexpected tRCURLY" -- is there
something I'm doing wrong?
Cheers.
On 8/12/08, Chris Paterson <cj...@googlemail.com> wrote:
> Thanks Jean, much obliged. I don't suppose you know if this will be changed
> in the future, so you're not required to enter a column?
>
> Cheers.
>
>
> On Tue, Aug 12, 2008 at 6:14 PM, Jean-Daniel Cryans
> <jd...@gmail.com>wrote:
>
>> Chris,
>>
>> API docs : http://hadoop.apache.org/hbase/docs/current/api/index.htmltaken
>> from http://hadoop.apache.org/hbase/docs/current/
>>
>> For your command, for the moment you really need to specify columns if you
>> want versions.
>>
>> J-D
>>
>> On Tue, Aug 12, 2008 at 1:03 PM, Chris Paterson
>> <cj...@googlemail.com>wrote:
>>
>> > Hi Jean,
>> >
>> > I was running Hadoop 16.4 / HBase 0.1.3 on a single VMWare node
>> > (Ubuntu).
>> > However having upgraded to Hadoop 17.1 / HBase 0.2.0 on the same hw it
>> > inserts all of the rows in < 2 seconds - problem solved!
>> >
>> > Also, do you know where can I find the API docs for 0.2.0?
>> >
>> > Finally I'm having trouble with the syntax in the new shell for
>> retrieving
>> > the latest 5 versions of a row. Off the top of my head (I'm away from my
>> > desk now) I'm using "get 'iptable', '192.168.1.1', VERSIONS = 5" but
>> this
>> > doesn't seem to work (removing the versions part does).
>> >
>> > Cheers,
>> > Chris
>> >
>> >
>> > On Tue, Aug 12, 2008 at 2:28 PM, Jean-Daniel Cryans <jdcryans@gmail.com
>> > >wrote:
>> >
>> > > Chris,
>> > >
>> > > It really seems slow yes. What's your setup like? # of machines, how
>> the
>> > > soft is installed, which version of hadoop/hbase, etc?
>> > >
>> > > Thx,
>> > >
>> > > J-D
>> > >
>> > > On Tue, Aug 12, 2008 at 9:09 AM, Chris Paterson
>> > > <cj...@googlemail.com>wrote:
>> > >
>> > > > Hey all,
>> > > >
>> > > > I'm trying to insert a large number of rows into my table (about
>> 5000)
>> > > > and it's proving very slow - ~2 seconds / row. The data's very
>> simple,
>> > > > with IP addresses being used as the row key and a single
>> > > > 10-character
>> > > > string as the data (under a single column family, "Details", and
>> > > > column, "PhoneNo"). There's also a timestamp (long) representing
>> > > > time
>> > > > since the epoch.
>> > > >
>> > > > Below's the program I'm running to insert:
>> > > >
>> > > > Text text = new Text(rowKey);
>> > > > long lockid = tbl.startUpdate(new Text(rowKey));
>> > > > tbl.put(lockid, new Text("Details:PhoneNo"), new Text(data));
>> > > > tbl.commit(lockid, timestamp);
>> > > >
>> > > > If anyone can see anything wrong with that I'm doing, please let me
>> > know!
>> > > > Cheers.
>> > > >
>> > >
>> >
>>
>
Re: Slow inserts
Posted by Chris Paterson <cj...@googlemail.com>.
Thanks Jean, much obliged. I don't suppose you know if this will be changed
in the future, so you're not required to enter a column?
Cheers.
On Tue, Aug 12, 2008 at 6:14 PM, Jean-Daniel Cryans <jd...@gmail.com>wrote:
> Chris,
>
> API docs : http://hadoop.apache.org/hbase/docs/current/api/index.htmltaken
> from http://hadoop.apache.org/hbase/docs/current/
>
> For your command, for the moment you really need to specify columns if you
> want versions.
>
> J-D
>
> On Tue, Aug 12, 2008 at 1:03 PM, Chris Paterson
> <cj...@googlemail.com>wrote:
>
> > Hi Jean,
> >
> > I was running Hadoop 16.4 / HBase 0.1.3 on a single VMWare node (Ubuntu).
> > However having upgraded to Hadoop 17.1 / HBase 0.2.0 on the same hw it
> > inserts all of the rows in < 2 seconds - problem solved!
> >
> > Also, do you know where can I find the API docs for 0.2.0?
> >
> > Finally I'm having trouble with the syntax in the new shell for
> retrieving
> > the latest 5 versions of a row. Off the top of my head (I'm away from my
> > desk now) I'm using "get 'iptable', '192.168.1.1', VERSIONS = 5" but
> this
> > doesn't seem to work (removing the versions part does).
> >
> > Cheers,
> > Chris
> >
> >
> > On Tue, Aug 12, 2008 at 2:28 PM, Jean-Daniel Cryans <jdcryans@gmail.com
> > >wrote:
> >
> > > Chris,
> > >
> > > It really seems slow yes. What's your setup like? # of machines, how
> the
> > > soft is installed, which version of hadoop/hbase, etc?
> > >
> > > Thx,
> > >
> > > J-D
> > >
> > > On Tue, Aug 12, 2008 at 9:09 AM, Chris Paterson
> > > <cj...@googlemail.com>wrote:
> > >
> > > > Hey all,
> > > >
> > > > I'm trying to insert a large number of rows into my table (about
> 5000)
> > > > and it's proving very slow - ~2 seconds / row. The data's very
> simple,
> > > > with IP addresses being used as the row key and a single 10-character
> > > > string as the data (under a single column family, "Details", and
> > > > column, "PhoneNo"). There's also a timestamp (long) representing time
> > > > since the epoch.
> > > >
> > > > Below's the program I'm running to insert:
> > > >
> > > > Text text = new Text(rowKey);
> > > > long lockid = tbl.startUpdate(new Text(rowKey));
> > > > tbl.put(lockid, new Text("Details:PhoneNo"), new Text(data));
> > > > tbl.commit(lockid, timestamp);
> > > >
> > > > If anyone can see anything wrong with that I'm doing, please let me
> > know!
> > > > Cheers.
> > > >
> > >
> >
>
Re: Slow inserts
Posted by Jean-Daniel Cryans <jd...@gmail.com>.
Chris,
API docs : http://hadoop.apache.org/hbase/docs/current/api/index.html taken
from http://hadoop.apache.org/hbase/docs/current/
For your command, for the moment you really need to specify columns if you
want versions.
J-D
On Tue, Aug 12, 2008 at 1:03 PM, Chris Paterson
<cj...@googlemail.com>wrote:
> Hi Jean,
>
> I was running Hadoop 16.4 / HBase 0.1.3 on a single VMWare node (Ubuntu).
> However having upgraded to Hadoop 17.1 / HBase 0.2.0 on the same hw it
> inserts all of the rows in < 2 seconds - problem solved!
>
> Also, do you know where can I find the API docs for 0.2.0?
>
> Finally I'm having trouble with the syntax in the new shell for retrieving
> the latest 5 versions of a row. Off the top of my head (I'm away from my
> desk now) I'm using "get 'iptable', '192.168.1.1', VERSIONS = 5" but this
> doesn't seem to work (removing the versions part does).
>
> Cheers,
> Chris
>
>
> On Tue, Aug 12, 2008 at 2:28 PM, Jean-Daniel Cryans <jdcryans@gmail.com
> >wrote:
>
> > Chris,
> >
> > It really seems slow yes. What's your setup like? # of machines, how the
> > soft is installed, which version of hadoop/hbase, etc?
> >
> > Thx,
> >
> > J-D
> >
> > On Tue, Aug 12, 2008 at 9:09 AM, Chris Paterson
> > <cj...@googlemail.com>wrote:
> >
> > > Hey all,
> > >
> > > I'm trying to insert a large number of rows into my table (about 5000)
> > > and it's proving very slow - ~2 seconds / row. The data's very simple,
> > > with IP addresses being used as the row key and a single 10-character
> > > string as the data (under a single column family, "Details", and
> > > column, "PhoneNo"). There's also a timestamp (long) representing time
> > > since the epoch.
> > >
> > > Below's the program I'm running to insert:
> > >
> > > Text text = new Text(rowKey);
> > > long lockid = tbl.startUpdate(new Text(rowKey));
> > > tbl.put(lockid, new Text("Details:PhoneNo"), new Text(data));
> > > tbl.commit(lockid, timestamp);
> > >
> > > If anyone can see anything wrong with that I'm doing, please let me
> know!
> > > Cheers.
> > >
> >
>
Re: Slow inserts
Posted by Chris Paterson <cj...@googlemail.com>.
Hi Jean,
I was running Hadoop 16.4 / HBase 0.1.3 on a single VMWare node (Ubuntu).
However having upgraded to Hadoop 17.1 / HBase 0.2.0 on the same hw it
inserts all of the rows in < 2 seconds - problem solved!
Also, do you know where can I find the API docs for 0.2.0?
Finally I'm having trouble with the syntax in the new shell for retrieving
the latest 5 versions of a row. Off the top of my head (I'm away from my
desk now) I'm using "get 'iptable', '192.168.1.1', VERSIONS = 5" but this
doesn't seem to work (removing the versions part does).
Cheers,
Chris
On Tue, Aug 12, 2008 at 2:28 PM, Jean-Daniel Cryans <jd...@gmail.com>wrote:
> Chris,
>
> It really seems slow yes. What's your setup like? # of machines, how the
> soft is installed, which version of hadoop/hbase, etc?
>
> Thx,
>
> J-D
>
> On Tue, Aug 12, 2008 at 9:09 AM, Chris Paterson
> <cj...@googlemail.com>wrote:
>
> > Hey all,
> >
> > I'm trying to insert a large number of rows into my table (about 5000)
> > and it's proving very slow - ~2 seconds / row. The data's very simple,
> > with IP addresses being used as the row key and a single 10-character
> > string as the data (under a single column family, "Details", and
> > column, "PhoneNo"). There's also a timestamp (long) representing time
> > since the epoch.
> >
> > Below's the program I'm running to insert:
> >
> > Text text = new Text(rowKey);
> > long lockid = tbl.startUpdate(new Text(rowKey));
> > tbl.put(lockid, new Text("Details:PhoneNo"), new Text(data));
> > tbl.commit(lockid, timestamp);
> >
> > If anyone can see anything wrong with that I'm doing, please let me know!
> > Cheers.
> >
>
Re: Slow inserts
Posted by Jean-Daniel Cryans <jd...@gmail.com>.
Chris,
It really seems slow yes. What's your setup like? # of machines, how the
soft is installed, which version of hadoop/hbase, etc?
Thx,
J-D
On Tue, Aug 12, 2008 at 9:09 AM, Chris Paterson
<cj...@googlemail.com>wrote:
> Hey all,
>
> I'm trying to insert a large number of rows into my table (about 5000)
> and it's proving very slow - ~2 seconds / row. The data's very simple,
> with IP addresses being used as the row key and a single 10-character
> string as the data (under a single column family, "Details", and
> column, "PhoneNo"). There's also a timestamp (long) representing time
> since the epoch.
>
> Below's the program I'm running to insert:
>
> Text text = new Text(rowKey);
> long lockid = tbl.startUpdate(new Text(rowKey));
> tbl.put(lockid, new Text("Details:PhoneNo"), new Text(data));
> tbl.commit(lockid, timestamp);
>
> If anyone can see anything wrong with that I'm doing, please let me know!
> Cheers.
>