You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Andrzej <bo...@wp.pl> on 2018/06/21 20:52:13 UTC
I am searching Java examples for HBase 2.0.0
Hi, I am trying use Java with installed HBase 2.0.0.
I see, Java for HBase has much more possibilities,like creating tables
(must not using shell)
I can find many examples, but all are old (for example using HBaseAdmin)
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.ConnectionUtils;
import org.apache.hadoop.hbase.client.HBaseAdmin;
public class Example {
public static void mk() throws IOException {
Configuration configuration = HBaseConfiguration.create();
configuration.set("hbase.zookeeper.quorum", "127.0.0.1");
configuration.set("hbase.zookeeper.property.clientPort", "2181");
final TableName tableName = TableName.valueOf("testEclipse");
final HTableDescriptor desc = new HTableDescriptor(tableName);
desc.addFamily(new HColumnDescriptor("Id"));
desc.addFamily(new HColumnDescriptor("Name"));
System.out.println( "Connecting..." );
HBaseAdmin hbase_admin = new HBaseAdmin( configuration );
System.out.println( "Creating Table..." );
hbase_admin.createTable( desc );
System.out.println("Done!");
}
public static void main(String[] args) throws IOException {
mk();
}
}
HTableDescriptor and HColumnDescriptor are deprecated,
but more important, that HBaseAdmin not works with configuration
parameter, but with interface ClusterConnection.
One implementation of ClusterConnection is ConnectionImplementation,
but I can't write
HBaseAdmin hbase_admin = new HBaseAdmin(new ConnectionImplementation());
nor
org.apache.hadoop.hbase.client.ConnectionImplementation
Where can I find newer examples for version 2.0.0?
In
https://github.com/apache/hbase/tree/master/hbase-client/src/test/java/org/apache/hadoop/hbase
in master branch?
or version is differ than 2.0? Maybe on other branch will for version 2.0?
I see there is Admin admin = new HBaseAdmin(mockConnection)
Admin, not HBaseAdmin.
Thanks in advance!
@Previous post: I am going "mvn package"
Re: Eclipse and hbase-server version 2.0.1- bad imports
Posted by Andrzej <bo...@wp.pl>.
W dniu 22.06.2018 o 23:13, Andrzej pisze:
> W dniu 22.06.2018 o 22:29, Ted Yu pisze:
> Thanks,
> "mvn eclipse:eclipse" downloaded all needed jars to .m2/repository
> I have in this directory both:
> hadoop-mapreduce-client-jobclient-2.7.4.jar (small 61 KB)
> hadoop-mapreduce-client-jobclient-2.7.4-tests.jar (full 1.4 MB)
OK.
I need additional jars:
hadoop-mapreduce-client-jobclient-2.7.4-tests.jar
hadoop-minikdc-2.7.4.jar
hamcrest-core-1.3.jar
hbase-http-2.0.1-tests.jar
hbase-procedure-2.0.1-tests.jar
mockito-core-2.1.0.jar
It is builded!
U run and have:
2018-06-22 23:30:35,249 INFO [Time-limited test]
hbase.HBaseCommonTestingUtility
(HBaseTestingUtility.java:startMiniCluster(953)) - Starting up
minicluster with 1 master(s) and 8 regionserver(s) and 8 datanode(s)
2018-06-22 23:30:35,257 INFO [Time-limited test]
hbase.HBaseCommonTestingUtility
(HBaseZKTestingUtility.java:setupClusterTestDir(86)) - Created new
mini-cluster data directory:
/home/andrzej/eclipse-workspace/jhbase/target/test-data/20ff2781-1630-4a62-b2f2-023437343046/cluster_b13f5c57-7ddb-47e2-816b-3a590aafedd3,
deleteOnExit=true
2018-06-22 23:30:35,257 INFO [Time-limited test]
hbase.HBaseCommonTestingUtility
(HBaseTestingUtility.java:startMiniCluster(968)) - STARTING DFS
2018-06-22 23:30:35,258 INFO [Time-limited test]
hbase.HBaseCommonTestingUtility
(HBaseTestingUtility.java:createDirAndSetProperty(745)) - Setting
test.cache.data to
/home/andrzej/eclipse-workspace/jhbase/target/test-data/20ff2781-1630-4a62-b2f2-023437343046/cache_data
in system properties and HBase conf
etc...
Re: Eclipse and hbase-server version 2.0.1- bad imports
Posted by Andrzej <bo...@wp.pl>.
W dniu 22.06.2018 o 22:29, Ted Yu pisze:
> Better approach is to let 'mvn eclipse:eclipse' generate the dependencies
> for you.
>
> bq. I don't have in .m2 directory
>
> Have you looked under ~/.m2/repository ?
Thanks,
"mvn eclipse:eclipse" downloaded all needed jars to .m2/repository
I have in this directory both:
hadoop-mapreduce-client-jobclient-2.7.4.jar (small 61 KB)
hadoop-mapreduce-client-jobclient-2.7.4-tests.jar (full 1.4 MB)
Some problems "mvn eclipse:eclipse" at end:
[INFO] Apache HBase - External Block Cache ................ SUCCESS [
6.864 s]
[INFO] Apache HBase - Assembly ............................ FAILURE [
9.885 s]
[INFO] Apache HBase - Shaded .............................. SKIPPED
[INFO] Apache HBase - Shaded - Client ..................... SKIPPED
[INFO] Apache HBase - Shaded - MapReduce .................. SKIPPED
[INFO] Apache HBase Shaded Packaging Invariants ........... SKIPPED
[INFO] Apache HBase - Archetypes .......................... SKIPPED
[INFO] Apache HBase - Exemplar for hbase-client archetype . SKIPPED
[INFO] Apache HBase - Exemplar for hbase-shaded-client archetype SKIPPED
[INFO] Apache HBase - Archetype builder ................... SKIPPED
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
Re: Eclipse and hbase-server version 2.0.1- bad imports
Posted by Ted Yu <yu...@gmail.com>.
If you want to run against hadoop 3.1.x, you would need HBASE-20244.
You can wait for hadoop 3.1.1 to come out which is more stable than 3.1.0
You should add dependency on hadoop-mapreduce-client-jobcli
ent-3.1.0-tests.jar
bq. Similarly "test" for other I can't found classes?
I already explained which jars contain the other two classes.
Better approach is to let 'mvn eclipse:eclipse' generate the dependencies
for you.
bq. I don't have in .m2 directory
Have you looked under ~/.m2/repository ?
Cheers
On Fri, Jun 22, 2018 at 10:27 AM, Andrzej <bo...@wp.pl> wrote:
> W dniu 22.06.2018 o 18:47, Ted Yu pisze:
>
>> $ jar tvf
>> ~/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-
>> jobclient/3.0.0/hadoop-mapreduce-client-jobclient-3.0.0-tests.jar
>> | grep MiniMRCluster
>>
>
> andrzej@andrzej-VirtualBox ~/jars $ jar tvf hadoop-mapreduce-client-jobclient-3.1.0.jar
> | grep MiniMRCluster
>
> <------nothing
>
> andrzej@andrzej-VirtualBox ~/jars $ jar tvf hadoop-mapreduce-client-jobclient-3.1.0-tests.jar
> | grep MiniMRCluster
> 1863 Mon Apr 02 21:08:24 CEST 2018 org/apache/hadoop/mapred/Clust
> erMapReduceTestCase$ConfigurableMiniMRCluster.class
> 573 Mon Apr 02 21:08:24 CEST 2018 org/apache/hadoop/mapred/MiniM
> RCluster$JobTrackerRunner.class
> 576 Mon Apr 02 21:08:24 CEST 2018 org/apache/hadoop/mapred/MiniM
> RCluster$TaskTrackerRunner.class
> 9937 Mon Apr 02 21:08:24 CEST 2018 org/apache/hadoop/mapred/MiniM
> RCluster.class
>
>
> *Solution*: I must hadoop-mapreduce-client-jobclient-3.1.0-tests.jar
> instead of hadoop-mapreduce-client-jobclient-3.1.0.jar ?
> Similarly "test" for other I can't found classes?
>
> Better is 3.1.0 than 3.0.0 for v2.0.1?
> I don't have in .m2 directory, I must run mvn on hbase-server folder?
>
Re: Eclipse and hbase-server version 2.0.1- bad imports
Posted by Andrzej <bo...@wp.pl>.
W dniu 22.06.2018 o 18:47, Ted Yu pisze:
> $ jar tvf
> ~/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/3.0.0/hadoop-mapreduce-client-jobclient-3.0.0-tests.jar
> | grep MiniMRCluster
andrzej@andrzej-VirtualBox ~/jars $ jar tvf
hadoop-mapreduce-client-jobclient-3.1.0.jar | grep MiniMRCluster
<------nothing
andrzej@andrzej-VirtualBox ~/jars $ jar tvf
hadoop-mapreduce-client-jobclient-3.1.0-tests.jar | grep MiniMRCluster
1863 Mon Apr 02 21:08:24 CEST 2018
org/apache/hadoop/mapred/ClusterMapReduceTestCase$ConfigurableMiniMRCluster.class
573 Mon Apr 02 21:08:24 CEST 2018
org/apache/hadoop/mapred/MiniMRCluster$JobTrackerRunner.class
576 Mon Apr 02 21:08:24 CEST 2018
org/apache/hadoop/mapred/MiniMRCluster$TaskTrackerRunner.class
9937 Mon Apr 02 21:08:24 CEST 2018
org/apache/hadoop/mapred/MiniMRCluster.class
*Solution*: I must hadoop-mapreduce-client-jobclient-3.1.0-tests.jar
instead of hadoop-mapreduce-client-jobclient-3.1.0.jar ?
Similarly "test" for other I can't found classes?
Better is 3.1.0 than 3.0.0 for v2.0.1?
I don't have in .m2 directory, I must run mvn on hbase-server folder?
Re: Eclipse and hbase-server version 2.0.1- bad imports
Posted by Ted Yu <yu...@gmail.com>.
$ jar tvf
~/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/3.0.0/hadoop-mapreduce-client-jobclient-3.0.0-tests.jar
| grep MiniMRCluster
1863 Fri Dec 08 11:31:44 PST 2017
org/apache/hadoop/mapred/ClusterMapReduceTestCase$ConfigurableMiniMRCluster.class
9947 Fri Dec 08 11:31:44 PST 2017
org/apache/hadoop/mapred/MiniMRCluster.class
576 Fri Dec 08 11:31:44 PST 2017
org/apache/hadoop/mapred/MiniMRCluster$TaskTrackerRunner.class
573 Fri Dec 08 11:31:44 PST 2017
org/apache/hadoop/mapred/MiniMRCluster$JobTrackerRunner.class
KeyStoreTestUtil is in hbase-http module.
ProcedureTestingUtility is in hbase-procedure module.
You can run the following command prior to importing projects into Eclipse:
mvn clean package -DskipTests eclipse:eclipse
On Fri, Jun 22, 2018 at 8:13 AM, Andrzej <bo...@wp.pl> wrote:
> W dniu 22.06.2018 o 17:02, Andrzej pisze:
>
>> hadoop-mapreduce-client-jobclient-3.1.0.jar
>> import org.apache.hadoop.mapred.MiniMRCluster;
>>
>
> for example :
> fc:org.apache.hadoop.mapred.MiniMRCluster
> give me hadoop-mapreduce-client-jobclient but this jar has package
> org.apache.hadoop.mapred but not have class MiniMRCluster inside this
> package.
>
>
Re: Eclipse and hbase-server version 2.0.1- bad imports
Posted by Andrzej <bo...@wp.pl>.
W dniu 22.06.2018 o 17:02, Andrzej pisze:
> hadoop-mapreduce-client-jobclient-3.1.0.jar
> import org.apache.hadoop.mapred.MiniMRCluster;
for example :
fc:org.apache.hadoop.mapred.MiniMRCluster
give me hadoop-mapreduce-client-jobclient but this jar has package
org.apache.hadoop.mapred but not have class MiniMRCluster inside this
package.
Eclipse and hbase-server version 2.0.1- bad imports
Posted by Andrzej <bo...@wp.pl>.
I have installed
http://mirrors.advancedhosters.com/apache/hbase/2.0.1/hbase-2.0.1-bin.tar.gz
I checkout HBase repository to version 2.0.1RC0 and add to Eclipse whole
hbase-server directory.
I add *.jar’s from /usr/local/HBase/lib/ and additionally newest
versions from http://search.maven.org:
hadoop-mapreduce-client-jobclient-3.1.0.jar
java-hamcrest-2.0.0.0.jar
hadoop-minikdc-3.1.0.jar
mockito-core-2.19.0.jar
hbase-http-2.0.1.jar
But still Eclipse can’t found some classes:
import org.apache.hadoop.mapred.MiniMRCluster;
import org.apache.hadoop.hbase.http.ssl.KeyStoreTestUtil;
import org.apache.hadoop.hbase.procedure2.ProcedureTestingUtility;
...
Re: I am searching Java examples for HBase 2.0.0
Posted by Ted Yu <yu...@gmail.com>.
See code HBaseTestingUtility :
public Connection getConnection() throws IOException {
if (this.connection == null) {
this.connection = ConnectionFactory.createConnection(this.conf);
Once you have the connection, you can call:
this.hbaseAdmin = (HBaseAdmin) getConnection().getAdmin();
Code under hbase-server/src/test would be a good place to start searching.
FYI
On Thu, Jun 21, 2018 at 1:52 PM, Andrzej <bo...@wp.pl> wrote:
> Hi, I am trying use Java with installed HBase 2.0.0.
> I see, Java for HBase has much more possibilities,like creating tables
> (must not using shell)
> I can find many examples, but all are old (for example using HBaseAdmin)
>
> import java.io.IOException;
> import org.apache.hadoop.conf.Configuration;
> import org.apache.hadoop.hbase.HBaseConfiguration;
> import org.apache.hadoop.hbase.HColumnDescriptor;
> import org.apache.hadoop.hbase.HTableDescriptor;
> import org.apache.hadoop.hbase.TableName;
> import org.apache.hadoop.hbase.client.ConnectionFactory;
> import org.apache.hadoop.hbase.client.ConnectionUtils;
> import org.apache.hadoop.hbase.client.HBaseAdmin;
>
> public class Example {
> public static void mk() throws IOException {
> Configuration configuration = HBaseConfiguration.create();
> configuration.set("hbase.zookeeper.quorum", "127.0.0.1");
> configuration.set("hbase.zookeeper.property.clientPort",
> "2181");
> final TableName tableName = TableName.valueOf("testEclipse
> ");
> final HTableDescriptor desc = new
> HTableDescriptor(tableName);
> desc.addFamily(new HColumnDescriptor("Id"));
> desc.addFamily(new HColumnDescriptor("Name"));
> System.out.println( "Connecting..." );
> HBaseAdmin hbase_admin = new HBaseAdmin( configuration );
> System.out.println( "Creating Table..." );
> hbase_admin.createTable( desc );
> System.out.println("Done!");
> }
>
> public static void main(String[] args) throws IOException {
> mk();
> }
> }
>
> HTableDescriptor and HColumnDescriptor are deprecated,
> but more important, that HBaseAdmin not works with configuration
> parameter, but with interface ClusterConnection.
> One implementation of ClusterConnection is ConnectionImplementation,
> but I can't write
> HBaseAdmin hbase_admin = new HBaseAdmin(new ConnectionImplementation());
> nor
> org.apache.hadoop.hbase.client.ConnectionImplementation
>
> Where can I find newer examples for version 2.0.0?
> In
> https://github.com/apache/hbase/tree/master/hbase-client/
> src/test/java/org/apache/hadoop/hbase
> in master branch?
> or version is differ than 2.0? Maybe on other branch will for version 2.0?
> I see there is Admin admin = new HBaseAdmin(mockConnection)
> Admin, not HBaseAdmin.
>
> Thanks in advance!
>
> @Previous post: I am going "mvn package"
>