You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Paul Baclace <pa...@gmail.com> on 2013/07/29 09:48:43 UTC
ycsb and hbase 0.95.1 hadoop2
[This might look like a user list question, or a question for another
project, but people on the hbase dev list use ycsb to test hbase, and
the side effect of my progress on this benefits hbase more than any
other project... please indulge me.]
I have been unable to build ycsb for hbase 0.95.1 hadoop2, even though
I added the artifacts to my local repo like so:
mvn install:install-file
-Dfile=../hbase-0.95.1-hadoop2/lib/hadoop-mapreduce-client-core-2.0.2-alpha.jar
-DgroupId=org.apache.hadoop -DartifactId=hadoop-mapreduce-client-core
-Dversion=2.0.2-alpha -Dpackaging=jar
mvn install:install-file
-Dfile=../hbase-0.95.1-hadoop2/lib/hbase-client-0.95.1-hadoop2.jar
-DgroupId=org.apache.hbase -DartifactId=hbase-client
-Dversion=0.95.1-hadoop2 -Dpackaging=jar
I looked at the pom.xml in the jar files to determine the parameters,
modified ycsb pom.xml and hbase/pom.xml; ycsb "mvn clean package" has
errors, not finding basic stuff:
[ERROR]
/Volumes/pebproject/ThirdEye/LSI/ws/YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:[34,29]
package org.apache.hadoop.conf does not exist
[ERROR]
/Volumes/pebproject/ThirdEye/LSI/ws/YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:[35,30]
cannot find symbol
symbol : class KeyValue
location: package org.apache.hadoop.hbase
[ERROR]
/Volumes/pebproject/ThirdEye/LSI/ws/YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:[46,35]
cannot find symbol
symbol : class Bytes
location: package org.apache.hadoop.hbase.util
[ERROR]
/Volumes/pebproject/ThirdEye/LSI/ws/YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:[47,30]
cannot find symbol
symbol : class HBaseConfiguration
location: package org.apache.hadoop.hbase
[... more not shown for brevity ]
I suspect ycsb pom.xml [https://github.com/brianfrankcooper/YCSB] is not
ready for packaging changes made in hbase 0.95 OR could it be that
adding jars to a local maven repo, because they are not yet on a
centralized server, simply does not suffice because not all dependencies
are added? HBaseConfiguration is in hbase-common-0.95.1-hadoop2.jar ,
but how to add all the turtles all the way down?
What's the secret to using ycsb for the latest HBase?
Re: ycsb and hbase 0.95.1 hadoop2
Posted by Paul Baclace <pa...@gmail.com>.
I eventually determined that bin/ycsb (a python script) was causing most
of the linkage problems, so I recommend using your own bash script to
start ycsb for HBase. The problem is that the ordering of paths in the
classpath is completely uncontrolled: bin/ycsb does not put the
hbase-binding/conf first, so it can pick up a default hbase-site.xml
found in a jar, and it will recursively include any jars in any
subdirectories under hbase-binding/ (a surprise when I mv'ed some files
to a subdir like dont_use_these and lsof showed them to be in use).
I ended up using hbase 0.95.1 hadoop1 on Apache Hadoop 1.0.4 for my
benchmarking.
On 20130729 13:34 , Nicolas Liochon wrote:
> 0.95.2 is not yet finished. As Ted suggested, you can build HBase locally
> from the sources (the 0.95 branch is usually good as the patches are tested
> before being committed).
>
> When 0.95.2 will be out I will redo the integration with ycsb. Hopefully it
> will be simple to make it work with all hadoop versions on the standards
> maven repo...
>
>
> On Mon, Jul 29, 2013 at 10:14 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> You can perform the following command in 0.95 workspace:
>>
>> mvn clean install -DskipTests
>>
>> Cheers
>>
>> On Mon, Jul 29, 2013 at 1:11 PM, Paul Baclace <paul.baclace@gmail.com
>>> wrote:
>>> On 20130729 12:18 , Nicolas Liochon wrote:
>>>
>>>> No, I've got nothing in my setting.xml
>>>> I build HBase locally (mvn install -Dhadoop.profile=2.0 -DskipTests), so
>>>> yes, it goes into my .m2
>>>>
>>>>
>>>> I've just rechecked with the last 0.97 as of now, it does compile. It
>>>> worked as well 2 weeks ago with the 0.95 branch. I'm not sure about the
>>>> 95.1. I remember some nasty maven issues.
>>>>
>>> ((I rather be attacked by angry ants than nasty mavens.))
>>>
>>>
>>>> If you're using the maven repo for the 0.95.1, IIRC there is no version
>>>> available that works well with hadoop 2 with a maven-built client (it's
>> ok
>>>> if you build hbase locally with the right hadoop version).
>>>>
>>> Is there an experimental or nightly 0.95.2 maven repo I can connect to? I
>>> can probably use 0.95.2 if all the tests passed; it looks like
>>> https://builds.apache.org/job/**HBase-0.95/373<
>> https://builds.apache.org/job/HBase-0.95/373>passed.
>>>
>>>
>>>
>>>
>>>>
>>>>
>>>>
>>>> On Mon, Jul 29, 2013 at 8:50 PM, Paul Baclace <paul.baclace@gmail.com
>>>>> wrote:
>>>> Thanks for your help, Nicolas. I adjusted the params and I get a
>>>>> ycsb/db/HBaseClient.java compile error (below) without any missing
>>>>> package
>>>>> errors, so I'm wondering how it can work for you. I am building by
>> adding
>>>>> artifacts from 0.95.1 release to my .m2; and you are building a
>> snapshot
>>>>> and "installing" that to .m2 (or other repo), I assume.
>>>>>
>>>>> It appears to be missing hbase-common which I added to .m2 and the
>>>>> compiler debug shows a classpath that only contains
>>>>> hbase-server-0.95.1-hadoop2.****jar (plus ycsb and general jars). Does
>>>>> your
>>>>> mvn -X compile cmd line for ycsb/db/HBaseClient.java contain more hbase
>>>>> or
>>>>> hadoop jars? Do you have a settings.xml that makes mvn pull in more
>>>>> dependencies?
>>>>>
>>>>>
>>>>> [ERROR] YCSB/hbase/src/main/java/com/****yahoo/ycsb/db/HBaseClient.****
>>>>> java:[34,30]
>>>>> package org.apache.hadoop.conf does not exist
>>>>> [ERROR]
>> /YCSB/hbase/src/main/java/com/****yahoo/ycsb/db/HBaseClient.****
>>>>> java:[35,31]
>>>>> cannot find symbol
>>>>> symbol : class KeyValue
>>>>>
>>>>>
>>>>>
>>>>> On 20130729 9:29 , Nicolas Liochon wrote:
>>>>>
>>>>> You can use the branch master here:
>> https://github.com/nkeywal/****YCSB<https://github.com/nkeywal/**YCSB>
>>>>>> <https://github.com/**nkeywal/YCSB <https://github.com/nkeywal/YCSB>>
>>>>>> Built it with:
>>>>>> mvn clean package -DskipTests -Dhbase-96 -Dhbase.version=0.97.0-**
>>>>>> SNAPSHOT
>>>>>> -Dhadoop.profile=2.0 -Dhadoop-two.version=2.0.5
>>>>>> (obviously, change the hbase.version & hadoop-two.version as you
>> need).
>>>>>> I'm awaiting the finalized hadoop2 flags in hbase pom to update this
>> and
>>>>>> push it to the main ycsb repo.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Nicolas
>>>>>>
>>>>>>
>>>>>> On Mon, Jul 29, 2013 at 6:04 PM, Nick Dimiduk <nd...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Hi Paul,
>>>>>>
>>>>>>> I think Nicolas Liochon maintains a personal set of branches for
>> this.
>>>>>>> Maybe he can point you in the right direction.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Nick
>>>>>>>
>>>>>>> On Mon, Jul 29, 2013 at 12:48 AM, Paul Baclace <
>> paul.baclace@gmail.com
>>>>>>> wrote:
>>>>>>>> [This might look like a user list question, or a question for
>> another
>>>>>>>> project, but people on the hbase dev list use ycsb to test hbase,
>> and
>>>>>>>> the
>>>>>>>> side effect of my progress on this benefits hbase more than any
>> other
>>>>>>>> project... please indulge me.]
>>>>>>>>
>>>>>>>> I have been unable to build ycsb for hbase 0.95.1 hadoop2, even
>>>>>>>> though
>>>>>>>> I
>>>>>>>> added the artifacts to my local repo like so:
>>>>>>>>
>>>>>>>> mvn install:install-file -Dfile=../hbase-0.95.1-**
>>>>>>>> hadoop2/lib/hadoop-mapreduce-******client-core-2.0.2-alpha.jar
>>>>>>>> -DgroupId=org.apache.hadoop -DartifactId=hadoop-mapreduce-****
>>>>>>>> **client-core
>>>>>>>> -Dversion=2.0.2-alpha -Dpackaging=jar
>>>>>>>>
>>>>>>>> mvn install:install-file -Dfile=../hbase-0.95.1-**
>>>>>>>> hadoop2/lib/hbase-client-0.95.******1-hadoop2.jar
>>>>>>>> -DgroupId=org.apache.hbase
>>>>>>>> -DartifactId=hbase-client -Dversion=0.95.1-hadoop2 -Dpackaging=jar
>>>>>>>>
>>>>>>>> I looked at the pom.xml in the jar files to determine the
>> parameters,
>>>>>>>> modified ycsb pom.xml and hbase/pom.xml; ycsb "mvn clean package"
>> has
>>>>>>>> errors, not finding basic stuff:
>>>>>>>>
>>>>>>>> [ERROR] YCSB/hbase/src/main/**
>>>>>>>>
>>>>>>>> java/com/yahoo/ycsb/db/******HBaseClient.java:[34,29] package
>>>>>>>> org.apache.hadoop.conf does not exist
>>>>>>>> [ERROR] YCSB/hbase/src/main/**
>>>>>>>>
>>>>>>>> java/com/yahoo/ycsb/db/******HBaseClient.java:[35,30] cannot find
>>>>>>>> symbol
>>>>>>>> symbol : class KeyValue
>>>>>>>> location: package org.apache.hadoop.hbase
>>>>>>>> [ERROR] YCSB/hbase/src/main/**
>>>>>>>>
>>>>>>>> java/com/yahoo/ycsb/db/******HBaseClient.java:[46,35] cannot find
>>>>>>>> symbol
>>>>>>>> symbol : class Bytes
>>>>>>>> location: package org.apache.hadoop.hbase.util
>>>>>>>> [ERROR] YCSB/hbase/src/main/**
>>>>>>>>
>>>>>>>> java/com/yahoo/ycsb/db/******HBaseClient.java:[47,30] cannot find
>>>>>>>> symbol
>>>>>>>> symbol : class HBaseConfiguration
>>>>>>>> location: package org.apache.hadoop.hbase
>>>>>>>> [... more not shown for brevity ]
>>>>>>>>
>>>>>>>> I suspect ycsb pom.xml [https://github.com/******
>>>>>>>> brianfrankcooper/YCSB <https://github.com/****brianfrankcooper/YCSB
>>> <
>>>>>>>> https://**github.com/**brianfrankcooper/**YCSB<
>> https://github.com/**brianfrankcooper/YCSB>
>>>>>>>> <
>>>>>>>>
>>>>>>>> https://github.com/****brianfrankcooper/YCSB<
>> https://github.com/**brianfrankcooper/YCSB>
>>>>>>> <https://**github.com/brianfrankcooper/**YCSB<
>> https://github.com/brianfrankcooper/YCSB>
>>>>>>>> ]
>>>>>>>> is not ready for packaging changes made in hbase 0.95 OR could it be
>>>>>>>> that
>>>>>>>> adding jars to a local maven repo, because they are not yet on a
>>>>>>>> centralized server, simply does not suffice because not all
>>>>>>>> dependencies
>>>>>>>> are added? HBaseConfiguration is in
>> hbase-common-0.95.1-hadoop2.***
>>>>>>>> ***jar
>>>>>>>> ,
>>>>>>>> but how to add all the turtles all the way down?
>>>>>>>>
>>>>>>>> What's the secret to using ycsb for the latest HBase?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
Re: ycsb and hbase 0.95.1 hadoop2
Posted by Nicolas Liochon <nk...@gmail.com>.
0.95.2 is not yet finished. As Ted suggested, you can build HBase locally
from the sources (the 0.95 branch is usually good as the patches are tested
before being committed).
When 0.95.2 will be out I will redo the integration with ycsb. Hopefully it
will be simple to make it work with all hadoop versions on the standards
maven repo...
On Mon, Jul 29, 2013 at 10:14 PM, Ted Yu <yu...@gmail.com> wrote:
> You can perform the following command in 0.95 workspace:
>
> mvn clean install -DskipTests
>
> Cheers
>
> On Mon, Jul 29, 2013 at 1:11 PM, Paul Baclace <paul.baclace@gmail.com
> >wrote:
>
> > On 20130729 12:18 , Nicolas Liochon wrote:
> >
> >> No, I've got nothing in my setting.xml
> >> I build HBase locally (mvn install -Dhadoop.profile=2.0 -DskipTests), so
> >> yes, it goes into my .m2
> >>
> >>
> >> I've just rechecked with the last 0.97 as of now, it does compile. It
> >> worked as well 2 weeks ago with the 0.95 branch. I'm not sure about the
> >> 95.1. I remember some nasty maven issues.
> >>
> > ((I rather be attacked by angry ants than nasty mavens.))
> >
> >
> >> If you're using the maven repo for the 0.95.1, IIRC there is no version
> >> available that works well with hadoop 2 with a maven-built client (it's
> ok
> >> if you build hbase locally with the right hadoop version).
> >>
> > Is there an experimental or nightly 0.95.2 maven repo I can connect to? I
> > can probably use 0.95.2 if all the tests passed; it looks like
> > https://builds.apache.org/job/**HBase-0.95/373<
> https://builds.apache.org/job/HBase-0.95/373>passed.
> >
> >
> >
> >
> >
> >>
> >>
> >>
> >>
> >> On Mon, Jul 29, 2013 at 8:50 PM, Paul Baclace <paul.baclace@gmail.com
> >> >wrote:
> >>
> >> Thanks for your help, Nicolas. I adjusted the params and I get a
> >>> ycsb/db/HBaseClient.java compile error (below) without any missing
> >>> package
> >>> errors, so I'm wondering how it can work for you. I am building by
> adding
> >>> artifacts from 0.95.1 release to my .m2; and you are building a
> snapshot
> >>> and "installing" that to .m2 (or other repo), I assume.
> >>>
> >>> It appears to be missing hbase-common which I added to .m2 and the
> >>> compiler debug shows a classpath that only contains
> >>> hbase-server-0.95.1-hadoop2.****jar (plus ycsb and general jars). Does
> >>> your
> >>> mvn -X compile cmd line for ycsb/db/HBaseClient.java contain more hbase
> >>> or
> >>> hadoop jars? Do you have a settings.xml that makes mvn pull in more
> >>> dependencies?
> >>>
> >>>
> >>> [ERROR] YCSB/hbase/src/main/java/com/****yahoo/ycsb/db/HBaseClient.****
> >>> java:[34,30]
> >>> package org.apache.hadoop.conf does not exist
> >>> [ERROR]
> /YCSB/hbase/src/main/java/com/****yahoo/ycsb/db/HBaseClient.****
> >>> java:[35,31]
> >>> cannot find symbol
> >>> symbol : class KeyValue
> >>>
> >>>
> >>>
> >>> On 20130729 9:29 , Nicolas Liochon wrote:
> >>>
> >>> You can use the branch master here:
> https://github.com/nkeywal/****YCSB<https://github.com/nkeywal/**YCSB>
> >>>> <https://github.com/**nkeywal/YCSB <https://github.com/nkeywal/YCSB>>
> >>>> Built it with:
> >>>> mvn clean package -DskipTests -Dhbase-96 -Dhbase.version=0.97.0-**
> >>>> SNAPSHOT
> >>>> -Dhadoop.profile=2.0 -Dhadoop-two.version=2.0.5
> >>>> (obviously, change the hbase.version & hadoop-two.version as you
> need).
> >>>>
> >>>> I'm awaiting the finalized hadoop2 flags in hbase pom to update this
> and
> >>>> push it to the main ycsb repo.
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Nicolas
> >>>>
> >>>>
> >>>> On Mon, Jul 29, 2013 at 6:04 PM, Nick Dimiduk <nd...@gmail.com>
> >>>> wrote:
> >>>>
> >>>> Hi Paul,
> >>>>
> >>>>> I think Nicolas Liochon maintains a personal set of branches for
> this.
> >>>>> Maybe he can point you in the right direction.
> >>>>>
> >>>>> Thanks,
> >>>>> Nick
> >>>>>
> >>>>> On Mon, Jul 29, 2013 at 12:48 AM, Paul Baclace <
> paul.baclace@gmail.com
> >>>>>
> >>>>> wrote:
> >>>>>> [This might look like a user list question, or a question for
> another
> >>>>>> project, but people on the hbase dev list use ycsb to test hbase,
> and
> >>>>>> the
> >>>>>> side effect of my progress on this benefits hbase more than any
> other
> >>>>>> project... please indulge me.]
> >>>>>>
> >>>>>> I have been unable to build ycsb for hbase 0.95.1 hadoop2, even
> >>>>>> though
> >>>>>> I
> >>>>>> added the artifacts to my local repo like so:
> >>>>>>
> >>>>>> mvn install:install-file -Dfile=../hbase-0.95.1-**
> >>>>>> hadoop2/lib/hadoop-mapreduce-******client-core-2.0.2-alpha.jar
> >>>>>> -DgroupId=org.apache.hadoop -DartifactId=hadoop-mapreduce-****
> >>>>>> **client-core
> >>>>>> -Dversion=2.0.2-alpha -Dpackaging=jar
> >>>>>>
> >>>>>> mvn install:install-file -Dfile=../hbase-0.95.1-**
> >>>>>> hadoop2/lib/hbase-client-0.95.******1-hadoop2.jar
> >>>>>> -DgroupId=org.apache.hbase
> >>>>>> -DartifactId=hbase-client -Dversion=0.95.1-hadoop2 -Dpackaging=jar
> >>>>>>
> >>>>>> I looked at the pom.xml in the jar files to determine the
> parameters,
> >>>>>> modified ycsb pom.xml and hbase/pom.xml; ycsb "mvn clean package"
> has
> >>>>>> errors, not finding basic stuff:
> >>>>>>
> >>>>>> [ERROR] YCSB/hbase/src/main/**
> >>>>>>
> >>>>>> java/com/yahoo/ycsb/db/******HBaseClient.java:[34,29] package
> >>>>>> org.apache.hadoop.conf does not exist
> >>>>>> [ERROR] YCSB/hbase/src/main/**
> >>>>>>
> >>>>>> java/com/yahoo/ycsb/db/******HBaseClient.java:[35,30] cannot find
> >>>>>> symbol
> >>>>>> symbol : class KeyValue
> >>>>>> location: package org.apache.hadoop.hbase
> >>>>>> [ERROR] YCSB/hbase/src/main/**
> >>>>>>
> >>>>>> java/com/yahoo/ycsb/db/******HBaseClient.java:[46,35] cannot find
> >>>>>> symbol
> >>>>>> symbol : class Bytes
> >>>>>> location: package org.apache.hadoop.hbase.util
> >>>>>> [ERROR] YCSB/hbase/src/main/**
> >>>>>>
> >>>>>> java/com/yahoo/ycsb/db/******HBaseClient.java:[47,30] cannot find
> >>>>>> symbol
> >>>>>> symbol : class HBaseConfiguration
> >>>>>> location: package org.apache.hadoop.hbase
> >>>>>> [... more not shown for brevity ]
> >>>>>>
> >>>>>> I suspect ycsb pom.xml [https://github.com/******
> >>>>>> brianfrankcooper/YCSB <https://github.com/****brianfrankcooper/YCSB
> ><
> >>>>>> https://**github.com/**brianfrankcooper/**YCSB<
> https://github.com/**brianfrankcooper/YCSB>
> >>>>>> >
> >>>>>> <
> >>>>>>
> >>>>>> https://github.com/****brianfrankcooper/YCSB<
> https://github.com/**brianfrankcooper/YCSB>
> >>>>> <https://**github.com/brianfrankcooper/**YCSB<
> https://github.com/brianfrankcooper/YCSB>
> >>>>> >
> >>>>>
> >>>>>> ]
> >>>>>> is not ready for packaging changes made in hbase 0.95 OR could it be
> >>>>>> that
> >>>>>> adding jars to a local maven repo, because they are not yet on a
> >>>>>> centralized server, simply does not suffice because not all
> >>>>>> dependencies
> >>>>>> are added? HBaseConfiguration is in
> hbase-common-0.95.1-hadoop2.***
> >>>>>> ***jar
> >>>>>> ,
> >>>>>> but how to add all the turtles all the way down?
> >>>>>>
> >>>>>> What's the secret to using ycsb for the latest HBase?
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >
>
Re: ycsb and hbase 0.95.1 hadoop2
Posted by Ted Yu <yu...@gmail.com>.
You can perform the following command in 0.95 workspace:
mvn clean install -DskipTests
Cheers
On Mon, Jul 29, 2013 at 1:11 PM, Paul Baclace <pa...@gmail.com>wrote:
> On 20130729 12:18 , Nicolas Liochon wrote:
>
>> No, I've got nothing in my setting.xml
>> I build HBase locally (mvn install -Dhadoop.profile=2.0 -DskipTests), so
>> yes, it goes into my .m2
>>
>>
>> I've just rechecked with the last 0.97 as of now, it does compile. It
>> worked as well 2 weeks ago with the 0.95 branch. I'm not sure about the
>> 95.1. I remember some nasty maven issues.
>>
> ((I rather be attacked by angry ants than nasty mavens.))
>
>
>> If you're using the maven repo for the 0.95.1, IIRC there is no version
>> available that works well with hadoop 2 with a maven-built client (it's ok
>> if you build hbase locally with the right hadoop version).
>>
> Is there an experimental or nightly 0.95.2 maven repo I can connect to? I
> can probably use 0.95.2 if all the tests passed; it looks like
> https://builds.apache.org/job/**HBase-0.95/373<https://builds.apache.org/job/HBase-0.95/373>passed.
>
>
>
>
>
>>
>>
>>
>>
>> On Mon, Jul 29, 2013 at 8:50 PM, Paul Baclace <paul.baclace@gmail.com
>> >wrote:
>>
>> Thanks for your help, Nicolas. I adjusted the params and I get a
>>> ycsb/db/HBaseClient.java compile error (below) without any missing
>>> package
>>> errors, so I'm wondering how it can work for you. I am building by adding
>>> artifacts from 0.95.1 release to my .m2; and you are building a snapshot
>>> and "installing" that to .m2 (or other repo), I assume.
>>>
>>> It appears to be missing hbase-common which I added to .m2 and the
>>> compiler debug shows a classpath that only contains
>>> hbase-server-0.95.1-hadoop2.****jar (plus ycsb and general jars). Does
>>> your
>>> mvn -X compile cmd line for ycsb/db/HBaseClient.java contain more hbase
>>> or
>>> hadoop jars? Do you have a settings.xml that makes mvn pull in more
>>> dependencies?
>>>
>>>
>>> [ERROR] YCSB/hbase/src/main/java/com/****yahoo/ycsb/db/HBaseClient.****
>>> java:[34,30]
>>> package org.apache.hadoop.conf does not exist
>>> [ERROR] /YCSB/hbase/src/main/java/com/****yahoo/ycsb/db/HBaseClient.****
>>> java:[35,31]
>>> cannot find symbol
>>> symbol : class KeyValue
>>>
>>>
>>>
>>> On 20130729 9:29 , Nicolas Liochon wrote:
>>>
>>> You can use the branch master here: https://github.com/nkeywal/****YCSB<https://github.com/nkeywal/**YCSB>
>>>> <https://github.com/**nkeywal/YCSB <https://github.com/nkeywal/YCSB>>
>>>> Built it with:
>>>> mvn clean package -DskipTests -Dhbase-96 -Dhbase.version=0.97.0-**
>>>> SNAPSHOT
>>>> -Dhadoop.profile=2.0 -Dhadoop-two.version=2.0.5
>>>> (obviously, change the hbase.version & hadoop-two.version as you need).
>>>>
>>>> I'm awaiting the finalized hadoop2 flags in hbase pom to update this and
>>>> push it to the main ycsb repo.
>>>>
>>>> Cheers,
>>>>
>>>> Nicolas
>>>>
>>>>
>>>> On Mon, Jul 29, 2013 at 6:04 PM, Nick Dimiduk <nd...@gmail.com>
>>>> wrote:
>>>>
>>>> Hi Paul,
>>>>
>>>>> I think Nicolas Liochon maintains a personal set of branches for this.
>>>>> Maybe he can point you in the right direction.
>>>>>
>>>>> Thanks,
>>>>> Nick
>>>>>
>>>>> On Mon, Jul 29, 2013 at 12:48 AM, Paul Baclace <paul.baclace@gmail.com
>>>>>
>>>>> wrote:
>>>>>> [This might look like a user list question, or a question for another
>>>>>> project, but people on the hbase dev list use ycsb to test hbase, and
>>>>>> the
>>>>>> side effect of my progress on this benefits hbase more than any other
>>>>>> project... please indulge me.]
>>>>>>
>>>>>> I have been unable to build ycsb for hbase 0.95.1 hadoop2, even
>>>>>> though
>>>>>> I
>>>>>> added the artifacts to my local repo like so:
>>>>>>
>>>>>> mvn install:install-file -Dfile=../hbase-0.95.1-**
>>>>>> hadoop2/lib/hadoop-mapreduce-******client-core-2.0.2-alpha.jar
>>>>>> -DgroupId=org.apache.hadoop -DartifactId=hadoop-mapreduce-****
>>>>>> **client-core
>>>>>> -Dversion=2.0.2-alpha -Dpackaging=jar
>>>>>>
>>>>>> mvn install:install-file -Dfile=../hbase-0.95.1-**
>>>>>> hadoop2/lib/hbase-client-0.95.******1-hadoop2.jar
>>>>>> -DgroupId=org.apache.hbase
>>>>>> -DartifactId=hbase-client -Dversion=0.95.1-hadoop2 -Dpackaging=jar
>>>>>>
>>>>>> I looked at the pom.xml in the jar files to determine the parameters,
>>>>>> modified ycsb pom.xml and hbase/pom.xml; ycsb "mvn clean package" has
>>>>>> errors, not finding basic stuff:
>>>>>>
>>>>>> [ERROR] YCSB/hbase/src/main/**
>>>>>>
>>>>>> java/com/yahoo/ycsb/db/******HBaseClient.java:[34,29] package
>>>>>> org.apache.hadoop.conf does not exist
>>>>>> [ERROR] YCSB/hbase/src/main/**
>>>>>>
>>>>>> java/com/yahoo/ycsb/db/******HBaseClient.java:[35,30] cannot find
>>>>>> symbol
>>>>>> symbol : class KeyValue
>>>>>> location: package org.apache.hadoop.hbase
>>>>>> [ERROR] YCSB/hbase/src/main/**
>>>>>>
>>>>>> java/com/yahoo/ycsb/db/******HBaseClient.java:[46,35] cannot find
>>>>>> symbol
>>>>>> symbol : class Bytes
>>>>>> location: package org.apache.hadoop.hbase.util
>>>>>> [ERROR] YCSB/hbase/src/main/**
>>>>>>
>>>>>> java/com/yahoo/ycsb/db/******HBaseClient.java:[47,30] cannot find
>>>>>> symbol
>>>>>> symbol : class HBaseConfiguration
>>>>>> location: package org.apache.hadoop.hbase
>>>>>> [... more not shown for brevity ]
>>>>>>
>>>>>> I suspect ycsb pom.xml [https://github.com/******
>>>>>> brianfrankcooper/YCSB <https://github.com/****brianfrankcooper/YCSB><
>>>>>> https://**github.com/**brianfrankcooper/**YCSB<https://github.com/**brianfrankcooper/YCSB>
>>>>>> >
>>>>>> <
>>>>>>
>>>>>> https://github.com/****brianfrankcooper/YCSB<https://github.com/**brianfrankcooper/YCSB>
>>>>> <https://**github.com/brianfrankcooper/**YCSB<https://github.com/brianfrankcooper/YCSB>
>>>>> >
>>>>>
>>>>>> ]
>>>>>> is not ready for packaging changes made in hbase 0.95 OR could it be
>>>>>> that
>>>>>> adding jars to a local maven repo, because they are not yet on a
>>>>>> centralized server, simply does not suffice because not all
>>>>>> dependencies
>>>>>> are added? HBaseConfiguration is in hbase-common-0.95.1-hadoop2.***
>>>>>> ***jar
>>>>>> ,
>>>>>> but how to add all the turtles all the way down?
>>>>>>
>>>>>> What's the secret to using ycsb for the latest HBase?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>
Re: ycsb and hbase 0.95.1 hadoop2
Posted by Paul Baclace <pa...@gmail.com>.
On 20130729 12:18 , Nicolas Liochon wrote:
> No, I've got nothing in my setting.xml
> I build HBase locally (mvn install -Dhadoop.profile=2.0 -DskipTests), so
> yes, it goes into my .m2
>
>
> I've just rechecked with the last 0.97 as of now, it does compile. It
> worked as well 2 weeks ago with the 0.95 branch. I'm not sure about the
> 95.1. I remember some nasty maven issues.
((I rather be attacked by angry ants than nasty mavens.))
>
> If you're using the maven repo for the 0.95.1, IIRC there is no version
> available that works well with hadoop 2 with a maven-built client (it's ok
> if you build hbase locally with the right hadoop version).
Is there an experimental or nightly 0.95.2 maven repo I can connect to?
I can probably use 0.95.2 if all the tests passed; it looks like
https://builds.apache.org/job/HBase-0.95/373 passed.
>
>
>
>
>
> On Mon, Jul 29, 2013 at 8:50 PM, Paul Baclace <pa...@gmail.com>wrote:
>
>> Thanks for your help, Nicolas. I adjusted the params and I get a
>> ycsb/db/HBaseClient.java compile error (below) without any missing package
>> errors, so I'm wondering how it can work for you. I am building by adding
>> artifacts from 0.95.1 release to my .m2; and you are building a snapshot
>> and "installing" that to .m2 (or other repo), I assume.
>>
>> It appears to be missing hbase-common which I added to .m2 and the
>> compiler debug shows a classpath that only contains
>> hbase-server-0.95.1-hadoop2.**jar (plus ycsb and general jars). Does your
>> mvn -X compile cmd line for ycsb/db/HBaseClient.java contain more hbase or
>> hadoop jars? Do you have a settings.xml that makes mvn pull in more
>> dependencies?
>>
>>
>> [ERROR] YCSB/hbase/src/main/java/com/**yahoo/ycsb/db/HBaseClient.**java:[34,30]
>> package org.apache.hadoop.conf does not exist
>> [ERROR] /YCSB/hbase/src/main/java/com/**yahoo/ycsb/db/HBaseClient.**java:[35,31]
>> cannot find symbol
>> symbol : class KeyValue
>>
>>
>>
>> On 20130729 9:29 , Nicolas Liochon wrote:
>>
>>> You can use the branch master here: https://github.com/nkeywal/**YCSB<https://github.com/nkeywal/YCSB>
>>> Built it with:
>>> mvn clean package -DskipTests -Dhbase-96 -Dhbase.version=0.97.0-**
>>> SNAPSHOT
>>> -Dhadoop.profile=2.0 -Dhadoop-two.version=2.0.5
>>> (obviously, change the hbase.version & hadoop-two.version as you need).
>>>
>>> I'm awaiting the finalized hadoop2 flags in hbase pom to update this and
>>> push it to the main ycsb repo.
>>>
>>> Cheers,
>>>
>>> Nicolas
>>>
>>>
>>> On Mon, Jul 29, 2013 at 6:04 PM, Nick Dimiduk <nd...@gmail.com> wrote:
>>>
>>> Hi Paul,
>>>> I think Nicolas Liochon maintains a personal set of branches for this.
>>>> Maybe he can point you in the right direction.
>>>>
>>>> Thanks,
>>>> Nick
>>>>
>>>> On Mon, Jul 29, 2013 at 12:48 AM, Paul Baclace <paul.baclace@gmail.com
>>>>
>>>>> wrote:
>>>>> [This might look like a user list question, or a question for another
>>>>> project, but people on the hbase dev list use ycsb to test hbase, and
>>>>> the
>>>>> side effect of my progress on this benefits hbase more than any other
>>>>> project... please indulge me.]
>>>>>
>>>>> I have been unable to build ycsb for hbase 0.95.1 hadoop2, even though
>>>>> I
>>>>> added the artifacts to my local repo like so:
>>>>>
>>>>> mvn install:install-file -Dfile=../hbase-0.95.1-**
>>>>> hadoop2/lib/hadoop-mapreduce-****client-core-2.0.2-alpha.jar
>>>>> -DgroupId=org.apache.hadoop -DartifactId=hadoop-mapreduce-**
>>>>> **client-core
>>>>> -Dversion=2.0.2-alpha -Dpackaging=jar
>>>>>
>>>>> mvn install:install-file -Dfile=../hbase-0.95.1-**
>>>>> hadoop2/lib/hbase-client-0.95.****1-hadoop2.jar
>>>>> -DgroupId=org.apache.hbase
>>>>> -DartifactId=hbase-client -Dversion=0.95.1-hadoop2 -Dpackaging=jar
>>>>>
>>>>> I looked at the pom.xml in the jar files to determine the parameters,
>>>>> modified ycsb pom.xml and hbase/pom.xml; ycsb "mvn clean package" has
>>>>> errors, not finding basic stuff:
>>>>>
>>>>> [ERROR] YCSB/hbase/src/main/**
>>>>>
>>>>> java/com/yahoo/ycsb/db/****HBaseClient.java:[34,29] package
>>>>> org.apache.hadoop.conf does not exist
>>>>> [ERROR] YCSB/hbase/src/main/**
>>>>>
>>>>> java/com/yahoo/ycsb/db/****HBaseClient.java:[35,30] cannot find symbol
>>>>> symbol : class KeyValue
>>>>> location: package org.apache.hadoop.hbase
>>>>> [ERROR] YCSB/hbase/src/main/**
>>>>>
>>>>> java/com/yahoo/ycsb/db/****HBaseClient.java:[46,35] cannot find symbol
>>>>> symbol : class Bytes
>>>>> location: package org.apache.hadoop.hbase.util
>>>>> [ERROR] YCSB/hbase/src/main/**
>>>>>
>>>>> java/com/yahoo/ycsb/db/****HBaseClient.java:[47,30] cannot find symbol
>>>>> symbol : class HBaseConfiguration
>>>>> location: package org.apache.hadoop.hbase
>>>>> [... more not shown for brevity ]
>>>>>
>>>>> I suspect ycsb pom.xml [https://github.com/****brianfrankcooper/YCSB<https://github.com/**brianfrankcooper/YCSB>
>>>>> <
>>>>>
>>>> https://github.com/**brianfrankcooper/YCSB<https://github.com/brianfrankcooper/YCSB>
>>>>> ]
>>>>> is not ready for packaging changes made in hbase 0.95 OR could it be
>>>>> that
>>>>> adding jars to a local maven repo, because they are not yet on a
>>>>> centralized server, simply does not suffice because not all dependencies
>>>>> are added? HBaseConfiguration is in hbase-common-0.95.1-hadoop2.****jar
>>>>> ,
>>>>> but how to add all the turtles all the way down?
>>>>>
>>>>> What's the secret to using ycsb for the latest HBase?
>>>>>
>>>>>
>>>>>
Re: ycsb and hbase 0.95.1 hadoop2
Posted by Nicolas Liochon <nk...@gmail.com>.
No, I've got nothing in my setting.xml
I build HBase locally (mvn install -Dhadoop.profile=2.0 -DskipTests), so
yes, it goes into my .m2
I've just rechecked with the last 0.97 as of now, it does compile. It
worked as well 2 weeks ago with the 0.95 branch. I'm not sure about the
95.1. I remember some nasty maven issues.
If you're using the maven repo for the 0.95.1, IIRC there is no version
available that works well with hadoop 2 with a maven-built client (it's ok
if you build hbase locally with the right hadoop version).
On Mon, Jul 29, 2013 at 8:50 PM, Paul Baclace <pa...@gmail.com>wrote:
> Thanks for your help, Nicolas. I adjusted the params and I get a
> ycsb/db/HBaseClient.java compile error (below) without any missing package
> errors, so I'm wondering how it can work for you. I am building by adding
> artifacts from 0.95.1 release to my .m2; and you are building a snapshot
> and "installing" that to .m2 (or other repo), I assume.
>
> It appears to be missing hbase-common which I added to .m2 and the
> compiler debug shows a classpath that only contains
> hbase-server-0.95.1-hadoop2.**jar (plus ycsb and general jars). Does your
> mvn -X compile cmd line for ycsb/db/HBaseClient.java contain more hbase or
> hadoop jars? Do you have a settings.xml that makes mvn pull in more
> dependencies?
>
>
> [ERROR] YCSB/hbase/src/main/java/com/**yahoo/ycsb/db/HBaseClient.**java:[34,30]
> package org.apache.hadoop.conf does not exist
> [ERROR] /YCSB/hbase/src/main/java/com/**yahoo/ycsb/db/HBaseClient.**java:[35,31]
> cannot find symbol
> symbol : class KeyValue
>
>
>
> On 20130729 9:29 , Nicolas Liochon wrote:
>
>> You can use the branch master here: https://github.com/nkeywal/**YCSB<https://github.com/nkeywal/YCSB>
>> Built it with:
>> mvn clean package -DskipTests -Dhbase-96 -Dhbase.version=0.97.0-**
>> SNAPSHOT
>> -Dhadoop.profile=2.0 -Dhadoop-two.version=2.0.5
>> (obviously, change the hbase.version & hadoop-two.version as you need).
>>
>> I'm awaiting the finalized hadoop2 flags in hbase pom to update this and
>> push it to the main ycsb repo.
>>
>> Cheers,
>>
>> Nicolas
>>
>>
>> On Mon, Jul 29, 2013 at 6:04 PM, Nick Dimiduk <nd...@gmail.com> wrote:
>>
>> Hi Paul,
>>>
>>> I think Nicolas Liochon maintains a personal set of branches for this.
>>> Maybe he can point you in the right direction.
>>>
>>> Thanks,
>>> Nick
>>>
>>> On Mon, Jul 29, 2013 at 12:48 AM, Paul Baclace <paul.baclace@gmail.com
>>>
>>>> wrote:
>>>> [This might look like a user list question, or a question for another
>>>> project, but people on the hbase dev list use ycsb to test hbase, and
>>>> the
>>>> side effect of my progress on this benefits hbase more than any other
>>>> project... please indulge me.]
>>>>
>>>> I have been unable to build ycsb for hbase 0.95.1 hadoop2, even though
>>>> I
>>>> added the artifacts to my local repo like so:
>>>>
>>>> mvn install:install-file -Dfile=../hbase-0.95.1-**
>>>> hadoop2/lib/hadoop-mapreduce-****client-core-2.0.2-alpha.jar
>>>> -DgroupId=org.apache.hadoop -DartifactId=hadoop-mapreduce-**
>>>> **client-core
>>>> -Dversion=2.0.2-alpha -Dpackaging=jar
>>>>
>>>> mvn install:install-file -Dfile=../hbase-0.95.1-**
>>>> hadoop2/lib/hbase-client-0.95.****1-hadoop2.jar
>>>> -DgroupId=org.apache.hbase
>>>> -DartifactId=hbase-client -Dversion=0.95.1-hadoop2 -Dpackaging=jar
>>>>
>>>> I looked at the pom.xml in the jar files to determine the parameters,
>>>> modified ycsb pom.xml and hbase/pom.xml; ycsb "mvn clean package" has
>>>> errors, not finding basic stuff:
>>>>
>>>> [ERROR] YCSB/hbase/src/main/**
>>>>
>>>> java/com/yahoo/ycsb/db/****HBaseClient.java:[34,29] package
>>>> org.apache.hadoop.conf does not exist
>>>> [ERROR] YCSB/hbase/src/main/**
>>>>
>>>> java/com/yahoo/ycsb/db/****HBaseClient.java:[35,30] cannot find symbol
>>>> symbol : class KeyValue
>>>> location: package org.apache.hadoop.hbase
>>>> [ERROR] YCSB/hbase/src/main/**
>>>>
>>>> java/com/yahoo/ycsb/db/****HBaseClient.java:[46,35] cannot find symbol
>>>> symbol : class Bytes
>>>> location: package org.apache.hadoop.hbase.util
>>>> [ERROR] YCSB/hbase/src/main/**
>>>>
>>>> java/com/yahoo/ycsb/db/****HBaseClient.java:[47,30] cannot find symbol
>>>> symbol : class HBaseConfiguration
>>>> location: package org.apache.hadoop.hbase
>>>> [... more not shown for brevity ]
>>>>
>>>> I suspect ycsb pom.xml [https://github.com/****brianfrankcooper/YCSB<https://github.com/**brianfrankcooper/YCSB>
>>>> <
>>>>
>>> https://github.com/**brianfrankcooper/YCSB<https://github.com/brianfrankcooper/YCSB>
>>> >]
>>>
>>>> is not ready for packaging changes made in hbase 0.95 OR could it be
>>>> that
>>>> adding jars to a local maven repo, because they are not yet on a
>>>> centralized server, simply does not suffice because not all dependencies
>>>> are added? HBaseConfiguration is in hbase-common-0.95.1-hadoop2.****jar
>>>> ,
>>>> but how to add all the turtles all the way down?
>>>>
>>>> What's the secret to using ycsb for the latest HBase?
>>>>
>>>>
>>>>
>
Re: ycsb and hbase 0.95.1 hadoop2
Posted by Paul Baclace <pa...@gmail.com>.
Thanks for your help, Nicolas. I adjusted the params and I get a
ycsb/db/HBaseClient.java compile error (below) without any missing
package errors, so I'm wondering how it can work for you. I am building
by adding artifacts from 0.95.1 release to my .m2; and you are building
a snapshot and "installing" that to .m2 (or other repo), I assume.
It appears to be missing hbase-common which I added to .m2 and the
compiler debug shows a classpath that only contains
hbase-server-0.95.1-hadoop2.jar (plus ycsb and general jars). Does your
mvn -X compile cmd line for ycsb/db/HBaseClient.java contain more hbase
or hadoop jars? Do you have a settings.xml that makes mvn pull in more
dependencies?
[ERROR]
YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:[34,30]
package org.apache.hadoop.conf does not exist
[ERROR]
/YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:[35,31]
cannot find symbol
symbol : class KeyValue
On 20130729 9:29 , Nicolas Liochon wrote:
> You can use the branch master here: https://github.com/nkeywal/YCSB
> Built it with:
> mvn clean package -DskipTests -Dhbase-96 -Dhbase.version=0.97.0-SNAPSHOT
> -Dhadoop.profile=2.0 -Dhadoop-two.version=2.0.5
> (obviously, change the hbase.version & hadoop-two.version as you need).
>
> I'm awaiting the finalized hadoop2 flags in hbase pom to update this and
> push it to the main ycsb repo.
>
> Cheers,
>
> Nicolas
>
>
> On Mon, Jul 29, 2013 at 6:04 PM, Nick Dimiduk <nd...@gmail.com> wrote:
>
>> Hi Paul,
>>
>> I think Nicolas Liochon maintains a personal set of branches for this.
>> Maybe he can point you in the right direction.
>>
>> Thanks,
>> Nick
>>
>> On Mon, Jul 29, 2013 at 12:48 AM, Paul Baclace <paul.baclace@gmail.com
>>> wrote:
>>> [This might look like a user list question, or a question for another
>>> project, but people on the hbase dev list use ycsb to test hbase, and the
>>> side effect of my progress on this benefits hbase more than any other
>>> project... please indulge me.]
>>>
>>> I have been unable to build ycsb for hbase 0.95.1 hadoop2, even though I
>>> added the artifacts to my local repo like so:
>>>
>>> mvn install:install-file -Dfile=../hbase-0.95.1-**
>>> hadoop2/lib/hadoop-mapreduce-**client-core-2.0.2-alpha.jar
>>> -DgroupId=org.apache.hadoop -DartifactId=hadoop-mapreduce-**client-core
>>> -Dversion=2.0.2-alpha -Dpackaging=jar
>>>
>>> mvn install:install-file -Dfile=../hbase-0.95.1-**
>>> hadoop2/lib/hbase-client-0.95.**1-hadoop2.jar -DgroupId=org.apache.hbase
>>> -DartifactId=hbase-client -Dversion=0.95.1-hadoop2 -Dpackaging=jar
>>>
>>> I looked at the pom.xml in the jar files to determine the parameters,
>>> modified ycsb pom.xml and hbase/pom.xml; ycsb "mvn clean package" has
>>> errors, not finding basic stuff:
>>>
>>> [ERROR] YCSB/hbase/src/main/**
>>> java/com/yahoo/ycsb/db/**HBaseClient.java:[34,29] package
>>> org.apache.hadoop.conf does not exist
>>> [ERROR] YCSB/hbase/src/main/**
>>> java/com/yahoo/ycsb/db/**HBaseClient.java:[35,30] cannot find symbol
>>> symbol : class KeyValue
>>> location: package org.apache.hadoop.hbase
>>> [ERROR] YCSB/hbase/src/main/**
>>> java/com/yahoo/ycsb/db/**HBaseClient.java:[46,35] cannot find symbol
>>> symbol : class Bytes
>>> location: package org.apache.hadoop.hbase.util
>>> [ERROR] YCSB/hbase/src/main/**
>>> java/com/yahoo/ycsb/db/**HBaseClient.java:[47,30] cannot find symbol
>>> symbol : class HBaseConfiguration
>>> location: package org.apache.hadoop.hbase
>>> [... more not shown for brevity ]
>>>
>>> I suspect ycsb pom.xml [https://github.com/**brianfrankcooper/YCSB<
>> https://github.com/brianfrankcooper/YCSB>]
>>> is not ready for packaging changes made in hbase 0.95 OR could it be that
>>> adding jars to a local maven repo, because they are not yet on a
>>> centralized server, simply does not suffice because not all dependencies
>>> are added? HBaseConfiguration is in hbase-common-0.95.1-hadoop2.**jar ,
>>> but how to add all the turtles all the way down?
>>>
>>> What's the secret to using ycsb for the latest HBase?
>>>
>>>
Re: ycsb and hbase 0.95.1 hadoop2
Posted by Nicolas Liochon <nk...@gmail.com>.
You can use the branch master here: https://github.com/nkeywal/YCSB
Built it with:
mvn clean package -DskipTests -Dhbase-96 -Dhbase.version=0.97.0-SNAPSHOT
-Dhadoop.profile=2.0 -Dhadoop-two.version=2.0.5
(obviously, change the hbase.version & hadoop-two.version as you need).
I'm awaiting the finalized hadoop2 flags in hbase pom to update this and
push it to the main ycsb repo.
Cheers,
Nicolas
On Mon, Jul 29, 2013 at 6:04 PM, Nick Dimiduk <nd...@gmail.com> wrote:
> Hi Paul,
>
> I think Nicolas Liochon maintains a personal set of branches for this.
> Maybe he can point you in the right direction.
>
> Thanks,
> Nick
>
> On Mon, Jul 29, 2013 at 12:48 AM, Paul Baclace <paul.baclace@gmail.com
> >wrote:
>
> > [This might look like a user list question, or a question for another
> > project, but people on the hbase dev list use ycsb to test hbase, and the
> > side effect of my progress on this benefits hbase more than any other
> > project... please indulge me.]
> >
> > I have been unable to build ycsb for hbase 0.95.1 hadoop2, even though I
> > added the artifacts to my local repo like so:
> >
> > mvn install:install-file -Dfile=../hbase-0.95.1-**
> > hadoop2/lib/hadoop-mapreduce-**client-core-2.0.2-alpha.jar
> > -DgroupId=org.apache.hadoop -DartifactId=hadoop-mapreduce-**client-core
> > -Dversion=2.0.2-alpha -Dpackaging=jar
> >
> > mvn install:install-file -Dfile=../hbase-0.95.1-**
> > hadoop2/lib/hbase-client-0.95.**1-hadoop2.jar -DgroupId=org.apache.hbase
> > -DartifactId=hbase-client -Dversion=0.95.1-hadoop2 -Dpackaging=jar
> >
> > I looked at the pom.xml in the jar files to determine the parameters,
> > modified ycsb pom.xml and hbase/pom.xml; ycsb "mvn clean package" has
> > errors, not finding basic stuff:
> >
> > [ERROR] /Volumes/pebproject/ThirdEye/**LSI/ws/YCSB/hbase/src/main/**
> > java/com/yahoo/ycsb/db/**HBaseClient.java:[34,29] package
> > org.apache.hadoop.conf does not exist
> > [ERROR] /Volumes/pebproject/ThirdEye/**LSI/ws/YCSB/hbase/src/main/**
> > java/com/yahoo/ycsb/db/**HBaseClient.java:[35,30] cannot find symbol
> > symbol : class KeyValue
> > location: package org.apache.hadoop.hbase
> > [ERROR] /Volumes/pebproject/ThirdEye/**LSI/ws/YCSB/hbase/src/main/**
> > java/com/yahoo/ycsb/db/**HBaseClient.java:[46,35] cannot find symbol
> > symbol : class Bytes
> > location: package org.apache.hadoop.hbase.util
> > [ERROR] /Volumes/pebproject/ThirdEye/**LSI/ws/YCSB/hbase/src/main/**
> > java/com/yahoo/ycsb/db/**HBaseClient.java:[47,30] cannot find symbol
> > symbol : class HBaseConfiguration
> > location: package org.apache.hadoop.hbase
> > [... more not shown for brevity ]
> >
> > I suspect ycsb pom.xml [https://github.com/**brianfrankcooper/YCSB<
> https://github.com/brianfrankcooper/YCSB>]
> > is not ready for packaging changes made in hbase 0.95 OR could it be that
> > adding jars to a local maven repo, because they are not yet on a
> > centralized server, simply does not suffice because not all dependencies
> > are added? HBaseConfiguration is in hbase-common-0.95.1-hadoop2.**jar ,
> > but how to add all the turtles all the way down?
> >
> > What's the secret to using ycsb for the latest HBase?
> >
> >
>
Re: ycsb and hbase 0.95.1 hadoop2
Posted by Nick Dimiduk <nd...@gmail.com>.
Hi Paul,
I think Nicolas Liochon maintains a personal set of branches for this.
Maybe he can point you in the right direction.
Thanks,
Nick
On Mon, Jul 29, 2013 at 12:48 AM, Paul Baclace <pa...@gmail.com>wrote:
> [This might look like a user list question, or a question for another
> project, but people on the hbase dev list use ycsb to test hbase, and the
> side effect of my progress on this benefits hbase more than any other
> project... please indulge me.]
>
> I have been unable to build ycsb for hbase 0.95.1 hadoop2, even though I
> added the artifacts to my local repo like so:
>
> mvn install:install-file -Dfile=../hbase-0.95.1-**
> hadoop2/lib/hadoop-mapreduce-**client-core-2.0.2-alpha.jar
> -DgroupId=org.apache.hadoop -DartifactId=hadoop-mapreduce-**client-core
> -Dversion=2.0.2-alpha -Dpackaging=jar
>
> mvn install:install-file -Dfile=../hbase-0.95.1-**
> hadoop2/lib/hbase-client-0.95.**1-hadoop2.jar -DgroupId=org.apache.hbase
> -DartifactId=hbase-client -Dversion=0.95.1-hadoop2 -Dpackaging=jar
>
> I looked at the pom.xml in the jar files to determine the parameters,
> modified ycsb pom.xml and hbase/pom.xml; ycsb "mvn clean package" has
> errors, not finding basic stuff:
>
> [ERROR] /Volumes/pebproject/ThirdEye/**LSI/ws/YCSB/hbase/src/main/**
> java/com/yahoo/ycsb/db/**HBaseClient.java:[34,29] package
> org.apache.hadoop.conf does not exist
> [ERROR] /Volumes/pebproject/ThirdEye/**LSI/ws/YCSB/hbase/src/main/**
> java/com/yahoo/ycsb/db/**HBaseClient.java:[35,30] cannot find symbol
> symbol : class KeyValue
> location: package org.apache.hadoop.hbase
> [ERROR] /Volumes/pebproject/ThirdEye/**LSI/ws/YCSB/hbase/src/main/**
> java/com/yahoo/ycsb/db/**HBaseClient.java:[46,35] cannot find symbol
> symbol : class Bytes
> location: package org.apache.hadoop.hbase.util
> [ERROR] /Volumes/pebproject/ThirdEye/**LSI/ws/YCSB/hbase/src/main/**
> java/com/yahoo/ycsb/db/**HBaseClient.java:[47,30] cannot find symbol
> symbol : class HBaseConfiguration
> location: package org.apache.hadoop.hbase
> [... more not shown for brevity ]
>
> I suspect ycsb pom.xml [https://github.com/**brianfrankcooper/YCSB<https://github.com/brianfrankcooper/YCSB>]
> is not ready for packaging changes made in hbase 0.95 OR could it be that
> adding jars to a local maven repo, because they are not yet on a
> centralized server, simply does not suffice because not all dependencies
> are added? HBaseConfiguration is in hbase-common-0.95.1-hadoop2.**jar ,
> but how to add all the turtles all the way down?
>
> What's the secret to using ycsb for the latest HBase?
>
>