You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Qiang Tian <ti...@gmail.com> on 2014/10/21 11:28:24 UTC
YCSB load data stucked
Hi Guys,
I am running YCSB 0.1.4 against hbase 0.98.5,
"bin/ycsb load hbase -P workloads/workloada -p columnfamily=f1 -p
recordcount=1000 -p threadcount=4 -s | tee -a workloada.dat" stucked as
below:
10 sec: 0 operations;
20 sec: 0 operations;
30 sec: 0 operations;
40 sec: 0 operations;
50 sec: 0 operations;
...
stacktrace shows YCSB is locating meta region:
at java/lang/Object.wait(Object.java:201)
at
org/apache/hadoop/hbase/zookeeper/ZooKeeperNodeTracker.blockUntilAvailable(ZooKeeperNodeTracker.java:122)
(entered lock:
org/apache/hadoop/hbase/zookeeper/RootRegionTracker@0x00000000E01A02B0,
entry count: 1)
at
org/apache/hadoop/hbase/zookeeper/RootRegionTracker.waitRootRegionLocation(RootRegionTracker.java:73)
at
org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:786)
at
org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
at
org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
at
org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:797)
at
org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
at
org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
at
org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
at
org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:189)
at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:163)
at com/yahoo/ycsb/db/HBaseClient.getHTable(HBaseClient.java:111)
(entered lock: java/lang/Object@0x00000000E006AD20, entry count: 1)
at com/yahoo/ycsb/db/HBaseClient.update(HBaseClient.java:295)
at com/yahoo/ycsb/db/HBaseClient.insert(HBaseClient.java:350)
at com/yahoo/ycsb/DBWrapper.insert(DBWrapper.java:145)
at com/yahoo/ycsb/workloads/CoreWorkload.doInsert(CoreWorkload.java:461)
at com/yahoo/ycsb/ClientThread.run(Client.java:269)
however looking at the RS log, the meta region is up successfully, hbase
shell works fine. I can also see it in zk cli:
[zk: localhost:2181(CONNECTED) 2] get /hbase/meta-region-server
?regionserver:60020??|~md"PBUF
ihc028???????)
cZxid = 0xc57
ctime = Tue Oct 21 16:23:41 CST 2014
mZxid = 0xc57
mtime = Tue Oct 21 16:23:41 CST 2014
pZxid = 0xc57
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 71
numChildren = 0
is it a configure issue? anything wrong with my steps?
1. on hbase master node, download ycsb-0.1.4.tar.gz
2. copy all jars in hbase lib dir to hbase-binding/lib
3. copy hbase-site.xml to hbase-binding/conf
4. create table in hbase shell: create 'usertable','f1','f2','f3'
5. load data via above command.
thanks.
Re: YCSB load data stucked
Posted by Sean Busbey <bu...@cloudera.com>.
As an aside, if there are changes we'd like to see in YCSB upstream has
started taking patches again with a bit of prodding.
On Tue, Oct 21, 2014 at 9:23 PM, Ted Yu <yu...@gmail.com> wrote:
> Once you clone ycsb, you should build it with your choice of 0.98
>
> Here's thread where Andrew mentioned his ycsb repo:
>
>
> http://search-hadoop.com/m/DHED4NaxYb1/andrew+purtell+ycsb+2014&subj=Re+Performance+oddity+between+AWS+instance+sizes
>
> Cheers
>
> On Tue, Oct 21, 2014 at 7:15 PM, Qiang Tian <ti...@gmail.com> wrote:
>
> > Thanks Ted.
> > do you mean I should rebuild the ycsb? could you point me Andrew's repo?
> >
> >
> > On Tue, Oct 21, 2014 at 5:37 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > Cycling bits:
> > > http://search-hadoop.com/m/DHED4N0syk1
> > >
> > > Andrew has his ycsb repo as well.
> > >
> > > Cheers
> > >
> > > On Oct 21, 2014, at 2:28 AM, Qiang Tian <ti...@gmail.com> wrote:
> > >
> > > > Hi Guys,
> > > > I am running YCSB 0.1.4 against hbase 0.98.5,
> > > >
> > > > "bin/ycsb load hbase -P workloads/workloada -p columnfamily=f1 -p
> > > > recordcount=1000 -p threadcount=4 -s | tee -a workloada.dat" stucked
> as
> > > > below:
> > > >
> > > > 10 sec: 0 operations;
> > > > 20 sec: 0 operations;
> > > > 30 sec: 0 operations;
> > > > 40 sec: 0 operations;
> > > > 50 sec: 0 operations;
> > > > ...
> > > >
> > > > stacktrace shows YCSB is locating meta region:
> > > >
> > > > at java/lang/Object.wait(Object.java:201)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/zookeeper/ZooKeeperNodeTracker.blockUntilAvailable(ZooKeeperNodeTracker.java:122)
> > > > (entered lock:
> > > >
> org/apache/hadoop/hbase/zookeeper/RootRegionTracker@0x00000000E01A02B0
> > ,
> > > > entry count: 1)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/zookeeper/RootRegionTracker.waitRootRegionLocation(RootRegionTracker.java:73)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:786)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:797)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
> > > > at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:189)
> > > > at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:163)
> > > > at com/yahoo/ycsb/db/HBaseClient.getHTable(HBaseClient.java:111)
> > > > (entered lock: java/lang/Object@0x00000000E006AD20, entry count: 1)
> > > > at com/yahoo/ycsb/db/HBaseClient.update(HBaseClient.java:295)
> > > > at com/yahoo/ycsb/db/HBaseClient.insert(HBaseClient.java:350)
> > > > at com/yahoo/ycsb/DBWrapper.insert(DBWrapper.java:145)
> > > > at
> > com/yahoo/ycsb/workloads/CoreWorkload.doInsert(CoreWorkload.java:461)
> > > > at com/yahoo/ycsb/ClientThread.run(Client.java:269)
> > > >
> > > >
> > > > however looking at the RS log, the meta region is up successfully,
> > hbase
> > > > shell works fine. I can also see it in zk cli:
> > > >
> > > > [zk: localhost:2181(CONNECTED) 2] get /hbase/meta-region-server
> > > > ? regionserver:60020??|~md"PBUF
> > > >
> > > > ihc028 ?? ?????)
> > > > cZxid = 0xc57
> > > > ctime = Tue Oct 21 16:23:41 CST 2014
> > > > mZxid = 0xc57
> > > > mtime = Tue Oct 21 16:23:41 CST 2014
> > > > pZxid = 0xc57
> > > > cversion = 0
> > > > dataVersion = 0
> > > > aclVersion = 0
> > > > ephemeralOwner = 0x0
> > > > dataLength = 71
> > > > numChildren = 0
> > > >
> > > >
> > > > is it a configure issue? anything wrong with my steps?
> > > > 1. on hbase master node, download ycsb-0.1.4.tar.gz
> > > > 2. copy all jars in hbase lib dir to hbase-binding/lib
> > > > 3. copy hbase-site.xml to hbase-binding/conf
> > > > 4. create table in hbase shell: create 'usertable','f1','f2','f3'
> > > > 5. load data via above command.
> > > >
> > > > thanks.
> > >
> >
>
--
Sean
Re: YCSB load data stucked
Posted by Qiang Tian <ti...@gmail.com>.
update:
the problem exists with below repo:
https://github.com/ramkrish86/ycsb/tree/ycsb_visibility
Andrew's repo is fine:
https://github.com/apurtell/YCSB/tree/new_hbase_client
On Wed, Oct 22, 2014 at 2:59 PM, Qiang Tian <ti...@gmail.com> wrote:
> with command "bin/ycsb load hbase -P workloads/workloada -p threadcount=10
> -p columnfamily=f1 -p recordcount=1000 -s", I just see 10 rows in hbase
> shell...I tried it in both standalone and a real 3 node cluster, the same
> result. a quick look at core/src/main/java/com/yahoo/ycsb/Client.java, the
> parameter looks to be right..
>
> anything wrong?
>
> thanks..
>
>
> On Wed, Oct 22, 2014 at 11:42 AM, Qiang Tian <ti...@gmail.com> wrote:
>
>> Thanks Ted,
>> I also got Mapkeeper error, comment the module works around it(
>> https://github.com/brianfrankcooper/YCSB/issues/152)
>>
>> On Wed, Oct 22, 2014 at 10:23 AM, Ted Yu <yu...@gmail.com> wrote:
>>
>>> Once you clone ycsb, you should build it with your choice of 0.98
>>>
>>> Here's thread where Andrew mentioned his ycsb repo:
>>>
>>>
>>> http://search-hadoop.com/m/DHED4NaxYb1/andrew+purtell+ycsb+2014&subj=Re+Performance+oddity+between+AWS+instance+sizes
>>>
>>> Cheers
>>>
>>> On Tue, Oct 21, 2014 at 7:15 PM, Qiang Tian <ti...@gmail.com> wrote:
>>>
>>> > Thanks Ted.
>>> > do you mean I should rebuild the ycsb? could you point me Andrew's
>>> repo?
>>> >
>>> >
>>> > On Tue, Oct 21, 2014 at 5:37 PM, Ted Yu <yu...@gmail.com> wrote:
>>> >
>>> > > Cycling bits:
>>> > > http://search-hadoop.com/m/DHED4N0syk1
>>> > >
>>> > > Andrew has his ycsb repo as well.
>>> > >
>>> > > Cheers
>>> > >
>>> > > On Oct 21, 2014, at 2:28 AM, Qiang Tian <ti...@gmail.com> wrote:
>>> > >
>>> > > > Hi Guys,
>>> > > > I am running YCSB 0.1.4 against hbase 0.98.5,
>>> > > >
>>> > > > "bin/ycsb load hbase -P workloads/workloada -p columnfamily=f1 -p
>>> > > > recordcount=1000 -p threadcount=4 -s | tee -a workloada.dat"
>>> stucked as
>>> > > > below:
>>> > > >
>>> > > > 10 sec: 0 operations;
>>> > > > 20 sec: 0 operations;
>>> > > > 30 sec: 0 operations;
>>> > > > 40 sec: 0 operations;
>>> > > > 50 sec: 0 operations;
>>> > > > ...
>>> > > >
>>> > > > stacktrace shows YCSB is locating meta region:
>>> > > >
>>> > > > at java/lang/Object.wait(Object.java:201)
>>> > > > at
>>> > > >
>>> > >
>>> >
>>> org/apache/hadoop/hbase/zookeeper/ZooKeeperNodeTracker.blockUntilAvailable(ZooKeeperNodeTracker.java:122)
>>> > > > (entered lock:
>>> > > >
>>> org/apache/hadoop/hbase/zookeeper/RootRegionTracker@0x00000000E01A02B0
>>> > ,
>>> > > > entry count: 1)
>>> > > > at
>>> > > >
>>> > >
>>> >
>>> org/apache/hadoop/hbase/zookeeper/RootRegionTracker.waitRootRegionLocation(RootRegionTracker.java:73)
>>> > > > at
>>> > > >
>>> > >
>>> >
>>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:786)
>>> > > > at
>>> > > >
>>> > >
>>> >
>>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
>>> > > > at
>>> > > >
>>> > >
>>> >
>>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
>>> > > > at
>>> > > >
>>> > >
>>> >
>>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:797)
>>> > > > at
>>> > > >
>>> > >
>>> >
>>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
>>> > > > at
>>> > > >
>>> > >
>>> >
>>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
>>> > > > at
>>> > > >
>>> > >
>>> >
>>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
>>> > > > at
>>> > > >
>>> > >
>>> >
>>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
>>> > > > at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:189)
>>> > > > at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:163)
>>> > > > at com/yahoo/ycsb/db/HBaseClient.getHTable(HBaseClient.java:111)
>>> > > > (entered lock: java/lang/Object@0x00000000E006AD20, entry count:
>>> 1)
>>> > > > at com/yahoo/ycsb/db/HBaseClient.update(HBaseClient.java:295)
>>> > > > at com/yahoo/ycsb/db/HBaseClient.insert(HBaseClient.java:350)
>>> > > > at com/yahoo/ycsb/DBWrapper.insert(DBWrapper.java:145)
>>> > > > at
>>> > com/yahoo/ycsb/workloads/CoreWorkload.doInsert(CoreWorkload.java:461)
>>> > > > at com/yahoo/ycsb/ClientThread.run(Client.java:269)
>>> > > >
>>> > > >
>>> > > > however looking at the RS log, the meta region is up successfully,
>>> > hbase
>>> > > > shell works fine. I can also see it in zk cli:
>>> > > >
>>> > > > [zk: localhost:2181(CONNECTED) 2] get /hbase/meta-region-server
>>> > > > ? regionserver:60020??|~md"PBUF
>>> > > >
>>> > > > ihc028 ?? ?????)
>>> > > > cZxid = 0xc57
>>> > > > ctime = Tue Oct 21 16:23:41 CST 2014
>>> > > > mZxid = 0xc57
>>> > > > mtime = Tue Oct 21 16:23:41 CST 2014
>>> > > > pZxid = 0xc57
>>> > > > cversion = 0
>>> > > > dataVersion = 0
>>> > > > aclVersion = 0
>>> > > > ephemeralOwner = 0x0
>>> > > > dataLength = 71
>>> > > > numChildren = 0
>>> > > >
>>> > > >
>>> > > > is it a configure issue? anything wrong with my steps?
>>> > > > 1. on hbase master node, download ycsb-0.1.4.tar.gz
>>> > > > 2. copy all jars in hbase lib dir to hbase-binding/lib
>>> > > > 3. copy hbase-site.xml to hbase-binding/conf
>>> > > > 4. create table in hbase shell: create 'usertable','f1','f2','f3'
>>> > > > 5. load data via above command.
>>> > > >
>>> > > > thanks.
>>> > >
>>> >
>>>
>>
>>
>
Re: YCSB load data stucked
Posted by Qiang Tian <ti...@gmail.com>.
with command "bin/ycsb load hbase -P workloads/workloada -p threadcount=10
-p columnfamily=f1 -p recordcount=1000 -s", I just see 10 rows in hbase
shell...I tried it in both standalone and a real 3 node cluster, the same
result. a quick look at core/src/main/java/com/yahoo/ycsb/Client.java, the
parameter looks to be right..
anything wrong?
thanks..
On Wed, Oct 22, 2014 at 11:42 AM, Qiang Tian <ti...@gmail.com> wrote:
> Thanks Ted,
> I also got Mapkeeper error, comment the module works around it(
> https://github.com/brianfrankcooper/YCSB/issues/152)
>
> On Wed, Oct 22, 2014 at 10:23 AM, Ted Yu <yu...@gmail.com> wrote:
>
>> Once you clone ycsb, you should build it with your choice of 0.98
>>
>> Here's thread where Andrew mentioned his ycsb repo:
>>
>>
>> http://search-hadoop.com/m/DHED4NaxYb1/andrew+purtell+ycsb+2014&subj=Re+Performance+oddity+between+AWS+instance+sizes
>>
>> Cheers
>>
>> On Tue, Oct 21, 2014 at 7:15 PM, Qiang Tian <ti...@gmail.com> wrote:
>>
>> > Thanks Ted.
>> > do you mean I should rebuild the ycsb? could you point me Andrew's repo?
>> >
>> >
>> > On Tue, Oct 21, 2014 at 5:37 PM, Ted Yu <yu...@gmail.com> wrote:
>> >
>> > > Cycling bits:
>> > > http://search-hadoop.com/m/DHED4N0syk1
>> > >
>> > > Andrew has his ycsb repo as well.
>> > >
>> > > Cheers
>> > >
>> > > On Oct 21, 2014, at 2:28 AM, Qiang Tian <ti...@gmail.com> wrote:
>> > >
>> > > > Hi Guys,
>> > > > I am running YCSB 0.1.4 against hbase 0.98.5,
>> > > >
>> > > > "bin/ycsb load hbase -P workloads/workloada -p columnfamily=f1 -p
>> > > > recordcount=1000 -p threadcount=4 -s | tee -a workloada.dat"
>> stucked as
>> > > > below:
>> > > >
>> > > > 10 sec: 0 operations;
>> > > > 20 sec: 0 operations;
>> > > > 30 sec: 0 operations;
>> > > > 40 sec: 0 operations;
>> > > > 50 sec: 0 operations;
>> > > > ...
>> > > >
>> > > > stacktrace shows YCSB is locating meta region:
>> > > >
>> > > > at java/lang/Object.wait(Object.java:201)
>> > > > at
>> > > >
>> > >
>> >
>> org/apache/hadoop/hbase/zookeeper/ZooKeeperNodeTracker.blockUntilAvailable(ZooKeeperNodeTracker.java:122)
>> > > > (entered lock:
>> > > >
>> org/apache/hadoop/hbase/zookeeper/RootRegionTracker@0x00000000E01A02B0
>> > ,
>> > > > entry count: 1)
>> > > > at
>> > > >
>> > >
>> >
>> org/apache/hadoop/hbase/zookeeper/RootRegionTracker.waitRootRegionLocation(RootRegionTracker.java:73)
>> > > > at
>> > > >
>> > >
>> >
>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:786)
>> > > > at
>> > > >
>> > >
>> >
>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
>> > > > at
>> > > >
>> > >
>> >
>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
>> > > > at
>> > > >
>> > >
>> >
>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:797)
>> > > > at
>> > > >
>> > >
>> >
>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
>> > > > at
>> > > >
>> > >
>> >
>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
>> > > > at
>> > > >
>> > >
>> >
>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
>> > > > at
>> > > >
>> > >
>> >
>> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
>> > > > at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:189)
>> > > > at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:163)
>> > > > at com/yahoo/ycsb/db/HBaseClient.getHTable(HBaseClient.java:111)
>> > > > (entered lock: java/lang/Object@0x00000000E006AD20, entry count: 1)
>> > > > at com/yahoo/ycsb/db/HBaseClient.update(HBaseClient.java:295)
>> > > > at com/yahoo/ycsb/db/HBaseClient.insert(HBaseClient.java:350)
>> > > > at com/yahoo/ycsb/DBWrapper.insert(DBWrapper.java:145)
>> > > > at
>> > com/yahoo/ycsb/workloads/CoreWorkload.doInsert(CoreWorkload.java:461)
>> > > > at com/yahoo/ycsb/ClientThread.run(Client.java:269)
>> > > >
>> > > >
>> > > > however looking at the RS log, the meta region is up successfully,
>> > hbase
>> > > > shell works fine. I can also see it in zk cli:
>> > > >
>> > > > [zk: localhost:2181(CONNECTED) 2] get /hbase/meta-region-server
>> > > > ? regionserver:60020??|~md"PBUF
>> > > >
>> > > > ihc028 ?? ?????)
>> > > > cZxid = 0xc57
>> > > > ctime = Tue Oct 21 16:23:41 CST 2014
>> > > > mZxid = 0xc57
>> > > > mtime = Tue Oct 21 16:23:41 CST 2014
>> > > > pZxid = 0xc57
>> > > > cversion = 0
>> > > > dataVersion = 0
>> > > > aclVersion = 0
>> > > > ephemeralOwner = 0x0
>> > > > dataLength = 71
>> > > > numChildren = 0
>> > > >
>> > > >
>> > > > is it a configure issue? anything wrong with my steps?
>> > > > 1. on hbase master node, download ycsb-0.1.4.tar.gz
>> > > > 2. copy all jars in hbase lib dir to hbase-binding/lib
>> > > > 3. copy hbase-site.xml to hbase-binding/conf
>> > > > 4. create table in hbase shell: create 'usertable','f1','f2','f3'
>> > > > 5. load data via above command.
>> > > >
>> > > > thanks.
>> > >
>> >
>>
>
>
Re: YCSB load data stucked
Posted by Qiang Tian <ti...@gmail.com>.
Thanks Ted,
I also got Mapkeeper error, comment the module works around it(
https://github.com/brianfrankcooper/YCSB/issues/152)
On Wed, Oct 22, 2014 at 10:23 AM, Ted Yu <yu...@gmail.com> wrote:
> Once you clone ycsb, you should build it with your choice of 0.98
>
> Here's thread where Andrew mentioned his ycsb repo:
>
>
> http://search-hadoop.com/m/DHED4NaxYb1/andrew+purtell+ycsb+2014&subj=Re+Performance+oddity+between+AWS+instance+sizes
>
> Cheers
>
> On Tue, Oct 21, 2014 at 7:15 PM, Qiang Tian <ti...@gmail.com> wrote:
>
> > Thanks Ted.
> > do you mean I should rebuild the ycsb? could you point me Andrew's repo?
> >
> >
> > On Tue, Oct 21, 2014 at 5:37 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > Cycling bits:
> > > http://search-hadoop.com/m/DHED4N0syk1
> > >
> > > Andrew has his ycsb repo as well.
> > >
> > > Cheers
> > >
> > > On Oct 21, 2014, at 2:28 AM, Qiang Tian <ti...@gmail.com> wrote:
> > >
> > > > Hi Guys,
> > > > I am running YCSB 0.1.4 against hbase 0.98.5,
> > > >
> > > > "bin/ycsb load hbase -P workloads/workloada -p columnfamily=f1 -p
> > > > recordcount=1000 -p threadcount=4 -s | tee -a workloada.dat" stucked
> as
> > > > below:
> > > >
> > > > 10 sec: 0 operations;
> > > > 20 sec: 0 operations;
> > > > 30 sec: 0 operations;
> > > > 40 sec: 0 operations;
> > > > 50 sec: 0 operations;
> > > > ...
> > > >
> > > > stacktrace shows YCSB is locating meta region:
> > > >
> > > > at java/lang/Object.wait(Object.java:201)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/zookeeper/ZooKeeperNodeTracker.blockUntilAvailable(ZooKeeperNodeTracker.java:122)
> > > > (entered lock:
> > > >
> org/apache/hadoop/hbase/zookeeper/RootRegionTracker@0x00000000E01A02B0
> > ,
> > > > entry count: 1)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/zookeeper/RootRegionTracker.waitRootRegionLocation(RootRegionTracker.java:73)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:786)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:797)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
> > > > at
> > > >
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
> > > > at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:189)
> > > > at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:163)
> > > > at com/yahoo/ycsb/db/HBaseClient.getHTable(HBaseClient.java:111)
> > > > (entered lock: java/lang/Object@0x00000000E006AD20, entry count: 1)
> > > > at com/yahoo/ycsb/db/HBaseClient.update(HBaseClient.java:295)
> > > > at com/yahoo/ycsb/db/HBaseClient.insert(HBaseClient.java:350)
> > > > at com/yahoo/ycsb/DBWrapper.insert(DBWrapper.java:145)
> > > > at
> > com/yahoo/ycsb/workloads/CoreWorkload.doInsert(CoreWorkload.java:461)
> > > > at com/yahoo/ycsb/ClientThread.run(Client.java:269)
> > > >
> > > >
> > > > however looking at the RS log, the meta region is up successfully,
> > hbase
> > > > shell works fine. I can also see it in zk cli:
> > > >
> > > > [zk: localhost:2181(CONNECTED) 2] get /hbase/meta-region-server
> > > > ? regionserver:60020??|~md"PBUF
> > > >
> > > > ihc028 ?? ?????)
> > > > cZxid = 0xc57
> > > > ctime = Tue Oct 21 16:23:41 CST 2014
> > > > mZxid = 0xc57
> > > > mtime = Tue Oct 21 16:23:41 CST 2014
> > > > pZxid = 0xc57
> > > > cversion = 0
> > > > dataVersion = 0
> > > > aclVersion = 0
> > > > ephemeralOwner = 0x0
> > > > dataLength = 71
> > > > numChildren = 0
> > > >
> > > >
> > > > is it a configure issue? anything wrong with my steps?
> > > > 1. on hbase master node, download ycsb-0.1.4.tar.gz
> > > > 2. copy all jars in hbase lib dir to hbase-binding/lib
> > > > 3. copy hbase-site.xml to hbase-binding/conf
> > > > 4. create table in hbase shell: create 'usertable','f1','f2','f3'
> > > > 5. load data via above command.
> > > >
> > > > thanks.
> > >
> >
>
Re: YCSB load data stucked
Posted by Ted Yu <yu...@gmail.com>.
Once you clone ycsb, you should build it with your choice of 0.98
Here's thread where Andrew mentioned his ycsb repo:
http://search-hadoop.com/m/DHED4NaxYb1/andrew+purtell+ycsb+2014&subj=Re+Performance+oddity+between+AWS+instance+sizes
Cheers
On Tue, Oct 21, 2014 at 7:15 PM, Qiang Tian <ti...@gmail.com> wrote:
> Thanks Ted.
> do you mean I should rebuild the ycsb? could you point me Andrew's repo?
>
>
> On Tue, Oct 21, 2014 at 5:37 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > Cycling bits:
> > http://search-hadoop.com/m/DHED4N0syk1
> >
> > Andrew has his ycsb repo as well.
> >
> > Cheers
> >
> > On Oct 21, 2014, at 2:28 AM, Qiang Tian <ti...@gmail.com> wrote:
> >
> > > Hi Guys,
> > > I am running YCSB 0.1.4 against hbase 0.98.5,
> > >
> > > "bin/ycsb load hbase -P workloads/workloada -p columnfamily=f1 -p
> > > recordcount=1000 -p threadcount=4 -s | tee -a workloada.dat" stucked as
> > > below:
> > >
> > > 10 sec: 0 operations;
> > > 20 sec: 0 operations;
> > > 30 sec: 0 operations;
> > > 40 sec: 0 operations;
> > > 50 sec: 0 operations;
> > > ...
> > >
> > > stacktrace shows YCSB is locating meta region:
> > >
> > > at java/lang/Object.wait(Object.java:201)
> > > at
> > >
> >
> org/apache/hadoop/hbase/zookeeper/ZooKeeperNodeTracker.blockUntilAvailable(ZooKeeperNodeTracker.java:122)
> > > (entered lock:
> > > org/apache/hadoop/hbase/zookeeper/RootRegionTracker@0x00000000E01A02B0
> ,
> > > entry count: 1)
> > > at
> > >
> >
> org/apache/hadoop/hbase/zookeeper/RootRegionTracker.waitRootRegionLocation(RootRegionTracker.java:73)
> > > at
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:786)
> > > at
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
> > > at
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
> > > at
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:797)
> > > at
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
> > > at
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
> > > at
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
> > > at
> > >
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
> > > at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:189)
> > > at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:163)
> > > at com/yahoo/ycsb/db/HBaseClient.getHTable(HBaseClient.java:111)
> > > (entered lock: java/lang/Object@0x00000000E006AD20, entry count: 1)
> > > at com/yahoo/ycsb/db/HBaseClient.update(HBaseClient.java:295)
> > > at com/yahoo/ycsb/db/HBaseClient.insert(HBaseClient.java:350)
> > > at com/yahoo/ycsb/DBWrapper.insert(DBWrapper.java:145)
> > > at
> com/yahoo/ycsb/workloads/CoreWorkload.doInsert(CoreWorkload.java:461)
> > > at com/yahoo/ycsb/ClientThread.run(Client.java:269)
> > >
> > >
> > > however looking at the RS log, the meta region is up successfully,
> hbase
> > > shell works fine. I can also see it in zk cli:
> > >
> > > [zk: localhost:2181(CONNECTED) 2] get /hbase/meta-region-server
> > > ? regionserver:60020??|~md"PBUF
> > >
> > > ihc028 ?? ?????)
> > > cZxid = 0xc57
> > > ctime = Tue Oct 21 16:23:41 CST 2014
> > > mZxid = 0xc57
> > > mtime = Tue Oct 21 16:23:41 CST 2014
> > > pZxid = 0xc57
> > > cversion = 0
> > > dataVersion = 0
> > > aclVersion = 0
> > > ephemeralOwner = 0x0
> > > dataLength = 71
> > > numChildren = 0
> > >
> > >
> > > is it a configure issue? anything wrong with my steps?
> > > 1. on hbase master node, download ycsb-0.1.4.tar.gz
> > > 2. copy all jars in hbase lib dir to hbase-binding/lib
> > > 3. copy hbase-site.xml to hbase-binding/conf
> > > 4. create table in hbase shell: create 'usertable','f1','f2','f3'
> > > 5. load data via above command.
> > >
> > > thanks.
> >
>
Re: YCSB load data stucked
Posted by Qiang Tian <ti...@gmail.com>.
Thanks Ted.
do you mean I should rebuild the ycsb? could you point me Andrew's repo?
On Tue, Oct 21, 2014 at 5:37 PM, Ted Yu <yu...@gmail.com> wrote:
> Cycling bits:
> http://search-hadoop.com/m/DHED4N0syk1
>
> Andrew has his ycsb repo as well.
>
> Cheers
>
> On Oct 21, 2014, at 2:28 AM, Qiang Tian <ti...@gmail.com> wrote:
>
> > Hi Guys,
> > I am running YCSB 0.1.4 against hbase 0.98.5,
> >
> > "bin/ycsb load hbase -P workloads/workloada -p columnfamily=f1 -p
> > recordcount=1000 -p threadcount=4 -s | tee -a workloada.dat" stucked as
> > below:
> >
> > 10 sec: 0 operations;
> > 20 sec: 0 operations;
> > 30 sec: 0 operations;
> > 40 sec: 0 operations;
> > 50 sec: 0 operations;
> > ...
> >
> > stacktrace shows YCSB is locating meta region:
> >
> > at java/lang/Object.wait(Object.java:201)
> > at
> >
> org/apache/hadoop/hbase/zookeeper/ZooKeeperNodeTracker.blockUntilAvailable(ZooKeeperNodeTracker.java:122)
> > (entered lock:
> > org/apache/hadoop/hbase/zookeeper/RootRegionTracker@0x00000000E01A02B0,
> > entry count: 1)
> > at
> >
> org/apache/hadoop/hbase/zookeeper/RootRegionTracker.waitRootRegionLocation(RootRegionTracker.java:73)
> > at
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:786)
> > at
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
> > at
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
> > at
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:797)
> > at
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
> > at
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
> > at
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
> > at
> >
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
> > at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:189)
> > at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:163)
> > at com/yahoo/ycsb/db/HBaseClient.getHTable(HBaseClient.java:111)
> > (entered lock: java/lang/Object@0x00000000E006AD20, entry count: 1)
> > at com/yahoo/ycsb/db/HBaseClient.update(HBaseClient.java:295)
> > at com/yahoo/ycsb/db/HBaseClient.insert(HBaseClient.java:350)
> > at com/yahoo/ycsb/DBWrapper.insert(DBWrapper.java:145)
> > at com/yahoo/ycsb/workloads/CoreWorkload.doInsert(CoreWorkload.java:461)
> > at com/yahoo/ycsb/ClientThread.run(Client.java:269)
> >
> >
> > however looking at the RS log, the meta region is up successfully, hbase
> > shell works fine. I can also see it in zk cli:
> >
> > [zk: localhost:2181(CONNECTED) 2] get /hbase/meta-region-server
> > ? regionserver:60020??|~md"PBUF
> >
> > ihc028 ?? ?????)
> > cZxid = 0xc57
> > ctime = Tue Oct 21 16:23:41 CST 2014
> > mZxid = 0xc57
> > mtime = Tue Oct 21 16:23:41 CST 2014
> > pZxid = 0xc57
> > cversion = 0
> > dataVersion = 0
> > aclVersion = 0
> > ephemeralOwner = 0x0
> > dataLength = 71
> > numChildren = 0
> >
> >
> > is it a configure issue? anything wrong with my steps?
> > 1. on hbase master node, download ycsb-0.1.4.tar.gz
> > 2. copy all jars in hbase lib dir to hbase-binding/lib
> > 3. copy hbase-site.xml to hbase-binding/conf
> > 4. create table in hbase shell: create 'usertable','f1','f2','f3'
> > 5. load data via above command.
> >
> > thanks.
>
Re: YCSB load data stucked
Posted by Ted Yu <yu...@gmail.com>.
Cycling bits:
http://search-hadoop.com/m/DHED4N0syk1
Andrew has his ycsb repo as well.
Cheers
On Oct 21, 2014, at 2:28 AM, Qiang Tian <ti...@gmail.com> wrote:
> Hi Guys,
> I am running YCSB 0.1.4 against hbase 0.98.5,
>
> "bin/ycsb load hbase -P workloads/workloada -p columnfamily=f1 -p
> recordcount=1000 -p threadcount=4 -s | tee -a workloada.dat" stucked as
> below:
>
> 10 sec: 0 operations;
> 20 sec: 0 operations;
> 30 sec: 0 operations;
> 40 sec: 0 operations;
> 50 sec: 0 operations;
> ...
>
> stacktrace shows YCSB is locating meta region:
>
> at java/lang/Object.wait(Object.java:201)
> at
> org/apache/hadoop/hbase/zookeeper/ZooKeeperNodeTracker.blockUntilAvailable(ZooKeeperNodeTracker.java:122)
> (entered lock:
> org/apache/hadoop/hbase/zookeeper/RootRegionTracker@0x00000000E01A02B0,
> entry count: 1)
> at
> org/apache/hadoop/hbase/zookeeper/RootRegionTracker.waitRootRegionLocation(RootRegionTracker.java:73)
> at
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:786)
> at
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
> at
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
> at
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:797)
> at
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
> at
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:895)
> at
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
> at
> org/apache/hadoop/hbase/client/HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
> at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:189)
> at org/apache/hadoop/hbase/client/HTable.<init>(HTable.java:163)
> at com/yahoo/ycsb/db/HBaseClient.getHTable(HBaseClient.java:111)
> (entered lock: java/lang/Object@0x00000000E006AD20, entry count: 1)
> at com/yahoo/ycsb/db/HBaseClient.update(HBaseClient.java:295)
> at com/yahoo/ycsb/db/HBaseClient.insert(HBaseClient.java:350)
> at com/yahoo/ycsb/DBWrapper.insert(DBWrapper.java:145)
> at com/yahoo/ycsb/workloads/CoreWorkload.doInsert(CoreWorkload.java:461)
> at com/yahoo/ycsb/ClientThread.run(Client.java:269)
>
>
> however looking at the RS log, the meta region is up successfully, hbase
> shell works fine. I can also see it in zk cli:
>
> [zk: localhost:2181(CONNECTED) 2] get /hbase/meta-region-server
> ?regionserver:60020??|~md"PBUF
>
> ihc028???????)
> cZxid = 0xc57
> ctime = Tue Oct 21 16:23:41 CST 2014
> mZxid = 0xc57
> mtime = Tue Oct 21 16:23:41 CST 2014
> pZxid = 0xc57
> cversion = 0
> dataVersion = 0
> aclVersion = 0
> ephemeralOwner = 0x0
> dataLength = 71
> numChildren = 0
>
>
> is it a configure issue? anything wrong with my steps?
> 1. on hbase master node, download ycsb-0.1.4.tar.gz
> 2. copy all jars in hbase lib dir to hbase-binding/lib
> 3. copy hbase-site.xml to hbase-binding/conf
> 4. create table in hbase shell: create 'usertable','f1','f2','f3'
> 5. load data via above command.
>
> thanks.