You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Adarsh Sharma <ad...@orkash.com> on 2011/01/05 10:11:47 UTC
Re-Master Not Running Exception ( Hive/Hbase Integration )
>From that wiki page:
"If you are not using hbase-0.20.3, you will need to rebuild the handler with the HBase jar matching your version, and change the --auxpath above accordingly. Failure to use matching versions will lead to misleading connection failures such as MasterNotRunningException since the HBase RPC protocol changes often."
Thanks I think this is the correct problem.
So I configure hbase-0.20.3 to be comapatible with hive-0.6.0 .
But the problem remains the same. I am able to create tables in Hive and Hbase but when i try to create Hive/Hbase integrated table , it throws the below exception. I checked the size of hive_hbase_handler.jar, hbase-0.20.3.jar , hbase-0.20.3.test.jar. It's same.
Please help.
Best Regards
Adarsh Sharma
JVS
On Dec 29, 2010, at 5:20 AM, Adarsh Sharma wrote:
> Dear all,
>
> I am following all wiki tutorial for configurin Hive with Hbase Integrated.
> For this I use :
>
> Hadoop-0.20.2
> Hive-0.6.0 ( By Default Metastore derby)
> Hbase-0.20.6
> Java 1.6_20
>
> I established successfully a Hadoop Cluster of 4 servers and run Hbase on them. I check all Web UI's ( Namenode , Jobtracker , Hbase Master, RegionServers ) etc. All is working fine.
>
> But when I issued the below command for creating Hive/Hbase created table I got this exception :
>
> hadoop@s2-ratw-1:~/project/hive-0.6.0$ bin/hive --auxpath /home/hadoop/project/hive-0.6.0/lib/hive_hbase-handler.jar,/home/hadoop/project/hive-0.6.0/lib/hbase-0.20.3.jar,/home/hadoop/project/hive-0.6.0/lib/zookeeper-3.2.2.jar -hiveconf hbase.zookeeper.quorum=192.168.1.103,192.168.1.114,192.168.1.115
> Hive history file=/tmp/hadoop/hive_job_log_hadoop_201012291831_937563117.txt
> hive> show tables;
> OK
> Time taken: 3.445 seconds
> hive> CREATE TABLE hive_hbasetable_k(key int, value string)
> > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> > WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> > TBLPROPERTIES ("hbase.table.name" = "hivehbasek");
> FAILED: Error in metadata: MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException
> at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:374)
> at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:72)
> at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:64)
> at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:159)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:275)
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:394)
> at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2126)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:166)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:633)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:506)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:384)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:302)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> )
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
> hive>
Re: Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext,
but class was expected
Posted by Sonal Goyal <so...@gmail.com>.
Which Hadoop versions are you testing and compiling against?
Thanks and Regards,
Sonal
<https://github.com/sonalgoyal/hiho>Connect Hadoop with databases,
Salesforce, FTP servers and others <https://github.com/sonalgoyal/hiho>
Nube Technologies <http://www.nubetech.co>
<http://in.linkedin.com/in/sonalgoyal>
On Wed, Jan 5, 2011 at 3:20 PM, Cavus,M.,Fa. Post Direkt <
M.Cavus@postdirekt.de> wrote:
> Hi,
> I get this, did anyone know why I get an Error?:
>
>
> 11/01/05 10:46:55 WARN conf.Configuration: fs.checkpoint.period is
> deprecated. Instead, use dfs.namenode.checkpoint.period
> 11/01/05 10:46:55 WARN conf.Configuration: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 11/01/05 10:46:55 INFO mapreduce.JobSubmitter: number of splits:1
> 11/01/05 10:46:55 INFO mapreduce.JobSubmitter: adding the following
> namenodes' delegation tokens:null
> 11/01/05 10:46:56 INFO mapreduce.Job: Running job: job_201101051016_0008
> 11/01/05 10:46:57 INFO mapreduce.Job: map 0% reduce 0%
> 11/01/05 10:47:04 INFO mapreduce.Job: map 100% reduce 0%
> 11/01/05 10:47:13 INFO mapreduce.Job: Task Id :
> attempt_201101051016_0008_r_000000_0, Status : FAILED
> Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext,
> but class was expected
> 11/01/05 10:47:23 INFO mapreduce.Job: Task Id :
> attempt_201101051016_0008_r_000000_1, Status : FAILED
> Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext,
> but class was expected
> 11/01/05 10:47:34 INFO mapreduce.Job: Task Id :
> attempt_201101051016_0008_r_000000_2, Status : FAILED
> Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext,
> but class was expected
> 11/01/05 10:47:47 INFO mapreduce.Job: Job complete:
> job_201101051016_0008
> 11/01/05 10:47:47 INFO mapreduce.Job: Counters: 19
> FileSystemCounters
> FILE_BYTES_WRITTEN=38
> HDFS_BYTES_READ=69
> Job Counters
> Data-local map tasks=1
> Total time spent by all maps waiting after reserving
> slots (ms)=0
> Total time spent by all reduces waiting after reserving
> slots (ms)=0
> Failed reduce tasks=1
> SLOTS_MILLIS_MAPS=5781
> SLOTS_MILLIS_REDUCES=6379
> Launched map tasks=1
> Launched reduce tasks=4
> Map-Reduce Framework
> Combine input records=0
> Failed Shuffles=0
> GC time elapsed (ms)=97
> Map input records=0
> Map output bytes=0
> Map output records=0
> Merged Map outputs=0
> Spilled Records=0
> SPLIT_RAW_BYTES=69
> 11/01/05 10:47:47 INFO zookeeper.ZooKeeper: Session: 0x12d555a4ed80018
> closed
>
>
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
Posted by "Cavus,M.,Fa. Post Direkt" <M....@postdirekt.de>.
Hi,
I get this, did anyone know why I get an Error?:
11/01/05 10:46:55 WARN conf.Configuration: fs.checkpoint.period is
deprecated. Instead, use dfs.namenode.checkpoint.period
11/01/05 10:46:55 WARN conf.Configuration: mapred.map.tasks is
deprecated. Instead, use mapreduce.job.maps
11/01/05 10:46:55 INFO mapreduce.JobSubmitter: number of splits:1
11/01/05 10:46:55 INFO mapreduce.JobSubmitter: adding the following
namenodes' delegation tokens:null
11/01/05 10:46:56 INFO mapreduce.Job: Running job: job_201101051016_0008
11/01/05 10:46:57 INFO mapreduce.Job: map 0% reduce 0%
11/01/05 10:47:04 INFO mapreduce.Job: map 100% reduce 0%
11/01/05 10:47:13 INFO mapreduce.Job: Task Id :
attempt_201101051016_0008_r_000000_0, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext,
but class was expected
11/01/05 10:47:23 INFO mapreduce.Job: Task Id :
attempt_201101051016_0008_r_000000_1, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext,
but class was expected
11/01/05 10:47:34 INFO mapreduce.Job: Task Id :
attempt_201101051016_0008_r_000000_2, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext,
but class was expected
11/01/05 10:47:47 INFO mapreduce.Job: Job complete:
job_201101051016_0008
11/01/05 10:47:47 INFO mapreduce.Job: Counters: 19
FileSystemCounters
FILE_BYTES_WRITTEN=38
HDFS_BYTES_READ=69
Job Counters
Data-local map tasks=1
Total time spent by all maps waiting after reserving
slots (ms)=0
Total time spent by all reduces waiting after reserving
slots (ms)=0
Failed reduce tasks=1
SLOTS_MILLIS_MAPS=5781
SLOTS_MILLIS_REDUCES=6379
Launched map tasks=1
Launched reduce tasks=4
Map-Reduce Framework
Combine input records=0
Failed Shuffles=0
GC time elapsed (ms)=97
Map input records=0
Map output bytes=0
Map output records=0
Merged Map outputs=0
Spilled Records=0
SPLIT_RAW_BYTES=69
11/01/05 10:47:47 INFO zookeeper.ZooKeeper: Session: 0x12d555a4ed80018
closed
Re: Re-Master Not Running Exception ( Hive/Hbase Integration )
Posted by Adarsh Sharma <ad...@orkash.com>.
Adarsh Sharma wrote:
> From that wiki page:
>
> "If you are not using hbase-0.20.3, you will need to rebuild the
> handler with the HBase jar matching your version, and change the
> --auxpath above accordingly. Failure to use matching versions will
> lead to misleading connection failures such as
> MasterNotRunningException since the HBase RPC protocol changes often."
>
>
> Thanks I think this is the correct problem.
>
> So I configure hbase-0.20.3 to be comapatible with hive-0.6.0 .
>
> But the problem remains the same. I am able to create tables in Hive
> and Hbase but when i try to create Hive/Hbase integrated table , it
> throws the below exception. I checked the size of
> hive_hbase_handler.jar, hbase-0.20.3.jar , hbase-0.20.3.test.jar. It's
> same.
>
hive> CREATE TABLE hive_hbasetable_k(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "hivehbasek");
FAILED: Error in metadata:
MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:374)
at
org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:72)
at
org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:64)
at
org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:159)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:275)
at
org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:394)
at
org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2126)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:166)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:633)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:506)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:384)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:302)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
)
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
> Please help.
>
>
> Best Regards
>
> Adarsh Sharma
>
>
> JVS
>
> On Dec 29, 2010, at 5:20 AM, Adarsh Sharma wrote:
>
>
>> Dear all,
>>
>> I am following all wiki tutorial for configurin Hive with Hbase
>> Integrated.
>> For this I use :
>>
>> Hadoop-0.20.2
>> Hive-0.6.0 ( By Default Metastore derby)
>> Hbase-0.20.6
>> Java 1.6_20
>>
>> I established successfully a Hadoop Cluster of 4 servers and run
>> Hbase on them. I check all Web UI's ( Namenode , Jobtracker , Hbase
>> Master, RegionServers ) etc. All is working fine.
>>
>> But when I issued the below command for creating Hive/Hbase created
>> table I got this exception :
>>
>> hadoop@s2-ratw-1:~/project/hive-0.6.0$ bin/hive --auxpath
>> /home/hadoop/project/hive-0.6.0/lib/hive_hbase-handler.jar,/home/hadoop/project/hive-0.6.0/lib/hbase-0.20.3.jar,/home/hadoop/project/hive-0.6.0/lib/zookeeper-3.2.2.jar
>> -hiveconf
>> hbase.zookeeper.quorum=192.168.1.103,192.168.1.114,192.168.1.115
>> Hive history
>> file=/tmp/hadoop/hive_job_log_hadoop_201012291831_937563117.txt
>> hive> show tables;
>> OK
>> Time taken: 3.445 seconds
>> hive> CREATE TABLE hive_hbasetable_k(key int, value string)
>> > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>> > WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
>> > TBLPROPERTIES ("hbase.table.name" = "hivehbasek");
>> FAILED: Error in metadata:
>> MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException
>> at
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:374)
>>
>> at
>> org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:72)
>> at
>> org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:64)
>>
>> at
>> org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:159)
>>
>> at
>> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:275)
>>
>> at
>> org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:394)
>> at
>> org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2126)
>> at
>> org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:166)
>> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
>> at
>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
>>
>> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:633)
>> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:506)
>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:384)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:302)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>> )
>> FAILED: Execution Error, return code 1 from
>> org.apache.hadoop.hive.ql.exec.DDLTask
>> hive>
>