You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Ted Yu <yu...@gmail.com> on 2010/03/25 17:29:49 UTC
create table syntax in CLI
Hi,
I encountered Parse Error creating table in CLI:
[root@tyu-linux hive-0.5.0-bin]# bin/hive
Hive history file=/tmp/root/hive_job_log_root_201003240832_1752130264.txt
hive> SHOW TABLES;
OK
Time taken: 8.09 seconds
hive> CREATE EXTERNAL TABLE users(key string, state string, country string,
country_id int)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>
> WITH SERDEPROPERTIES (
>
> "hbase.key.type" = "string",
>
> "hbase.columns.mapping" = "info:state,info:country,info:country_id",
>
> "hbase.table.name" = "users"
>
> );
FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table file
format specification
*Here is the tail of hive.log:*
2010-03-24 08:32:07,937 ERROR DataNucleus.Plugin
(Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
"org.eclipse.text" but it cannot be resolved.
2010-03-24 08:34:18,337 ERROR ql.Driver (SessionState.java:printError(248))
- FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
file format specification
org.apache.hadoop.hive.ql.parse.ParseException: line 2:0 cannot recognize
input 'STORED' in table file format specification
at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:357)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:267)
at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
If you know how to fix my query, please share.
Re: create table syntax in CLI
Posted by John Sichi <js...@facebook.com>.
I'll update that page once I get our test cluster set up.
In the meantime, if you figure it out, please post to this mailing list.
Try hbase.zookeeper.quorum plus hbase.zookeeper.property.clientPort.
Hbase mailing list says those are the minimum required.
JVS
On Mar 25, 2010, at 7:12 PM, "Ted Yu" <yu...@gmail.com>> wrote:
We use distributed mode because the cluster would be deployed to production.
I didn't find zookeeper commandline parameter on <http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage> http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage
Do I need to use an entry in hbase-site.xml ?
Our hbase cluster is based on 0.20.1
But I used hbase client 0.20.3 against it before and didn't see RetriesExhaustedException.
Thanks
On Thu, Mar 25, 2010 at 6:07 PM, John Sichi <<m...@facebook.com>> wrote:
Hmmm, what kind of HBase cluster are you running (standalone, pseudo-distributed, or distributed)? If distributed, you may need to pass zookeeper connection info instead of hbase.master; I haven't tested out distributed yet but will be soon.
JVS
On Mar 25, 2010, at 4:43 PM, Ted Yu wrote:
I removed hbase-site.xml
[root@tyu-linux dist]# bin/hive -hiveconf hbase.master=snv-it-lin-006:60000
Hive history file=/tmp/root/hive_job_log_root_201003251614_1493474415.txt
hive> CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat int)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
> "hbase.key.type" = "string",
> "hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
> "<http://hbase.table.name/>hbase.table.name<http://hbase.table.name>" = "ruletable"
> );
FAILED: Error in metadata: MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:374)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:72)
at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:63)
at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:320)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:1445)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:124)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:630)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:382)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
)
But <http://snv-it-lin-006:60010/master.jsp> http://snv-it-lin-006:60010/master.jsp shows everything is normal.
And hbase.master.port is 60000 in hbase-default.xml
Some suggestions ?
Thanks
On Thu, Mar 25, 2010 at 4:21 PM, John Sichi <<m...@facebook.com>> wrote:
Don't copy hbase-site.xml; you don't want to set up a new HBase cluster. Just set your hive conf to point to your existing HBase master:
hbase.master=hbase.yoyodyne.com:60000<http://hbase.yoyodyne.com:60000/>
<http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage>http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage
JVS
On Mar 25, 2010, at 4:16 PM, Ted Yu wrote:
I got pass that error when I use CLI from trunk.
I copied hbase-site.xml from hbase master machine to trunk/dist/conf.
When I issue the following command in CLI:
CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat int)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
"hbase.key.type" = "string",
"hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
"<http://hbase.table.name/>hbase.table.name<http://hbase.table.name>" = "ruletable"
);
I get:
2010-03-25 15:49:52,557 WARN client.HConnectionManager$TableServers (HConnectionManager.java:tableExists(411)) - Testing for table existence threw exception
org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 10.10.31.143:60020<http://10.10.31.143:60020/> for region .META.,,1, row '', but failed after 10 attempts.
Exceptions:
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1048)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:454)
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.tableExists(HConnectionManager.java:404)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:113)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:100)
at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)
However, I don't see any exception in hbase-hbaseadmin-regionserver.log on 10.10.31.143 within the past 4 hours.
No exception in hbase-hbaseadmin-master.log on hbase master machine either.
Please comment.
On Thu, Mar 25, 2010 at 11:22 AM, Carl Steinbach <<m...@cloudera.com>> wrote:
Hi Ted,
It looks like your copy of Hive does not have the changes that implemented support for integration with HBase. HBase support was committed to trunk on March 12th, and currently it is only available on trunk. In order to use it you need to checkout (or update) the source from the svn repository. Instructions describing how to do this are on the Hive wiki.
Carl
On Thu, Mar 25, 2010 at 9:29 AM, Ted Yu <<m...@gmail.com>> wrote:
Hi,
I encountered Parse Error creating table in CLI:
[root@tyu-linux hive-0.5.0-bin]# bin/hive
Hive history file=/tmp/root/hive_job_log_root_201003240832_1752130264.txt
hive> SHOW TABLES;
OK
Time taken: 8.09 seconds
hive> CREATE EXTERNAL TABLE users(key string, state string, country string,
country_id int)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>
> WITH SERDEPROPERTIES (
>
> "hbase.key.type" = "string",
>
> "hbase.columns.mapping" = "info:state,info:country,info:country_id",
>
> "<http://hbase.table.name/>hbase.table.name<http://hbase.table.name>" = "users"
>
> );
FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table file
format specification
Here is the tail of hive.log:
2010-03-24 08:32:07,937 ERROR DataNucleus.Plugin
(Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
"org.eclipse.text" but it cannot be resolved.
2010-03-24 08:34:18,337 ERROR ql.Driver (SessionState.java:printError(248))
- FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
file format specification
org.apache.hadoop.hive.ql.parse.ParseException: line 2:0 cannot recognize
input 'STORED' in table file format specification
at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:357)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:267)
at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
If you know how to fix my query, please share.
Re: create table syntax in CLI
Posted by John Sichi <js...@facebook.com>.
Ted, when you reply, please reply-all to the list so that the conversation remains accessible to all.
Parameters are passed implicitly via the configuration object, not explicitly by name.
The unit test only specifies the master because that is all that is needed for a standalone server.
JVS
On Mar 25, 2010, at 9:18 PM, "Ted Yu" <yu...@gmail.com>> wrote:
I searched source code tree under trunk and didn't see how hbase.master parameter is passed to HBaseStorageHandler.java
Also it looks like the unit test only specifies hostname for hbase.master:
"Z:\kindsight\hive\trunk\hbase-handler\src\test\org\apache\hadoop\hive\hbase\HBaseQTestUtil.java"(55,17): conf.set("hbase.master", "local");
BTW I saw a lot of following exceptions:
2010-03-25 13:24:27,056 WARN zookeeper.ClientCnxn (ClientCnxn.java:cleanup(1001)) - Ignoring exception during shutdown input
java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:638)
at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
Thanks
On Thu, Mar 25, 2010 at 7:12 PM, Ted Yu <<m...@gmail.com>> wrote:
We use distributed mode because the cluster would be deployed to production.
I didn't find zookeeper commandline parameter on <http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage> http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage
Do I need to use an entry in hbase-site.xml ?
Our hbase cluster is based on 0.20.1
But I used hbase client 0.20.3 against it before and didn't see RetriesExhaustedException.
Thanks
On Thu, Mar 25, 2010 at 6:07 PM, John Sichi <<m...@facebook.com>> wrote:
Hmmm, what kind of HBase cluster are you running (standalone, pseudo-distributed, or distributed)? If distributed, you may need to pass zookeeper connection info instead of hbase.master; I haven't tested out distributed yet but will be soon.
JVS
On Mar 25, 2010, at 4:43 PM, Ted Yu wrote:
I removed hbase-site.xml
[root@tyu-linux dist]# bin/hive -hiveconf hbase.master=snv-it-lin-006:60000
Hive history file=/tmp/root/hive_job_log_root_201003251614_1493474415.txt
hive> CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat int)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
> "hbase.key.type" = "string",
> "hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
> "<http://hbase.table.name/>hbase.table.name<http://hbase.table.name>" = "ruletable"
> );
FAILED: Error in metadata: MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:374)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:72)
at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:63)
at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:320)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:1445)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:124)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:630)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:382)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
)
But <http://snv-it-lin-006:60010/master.jsp> http://snv-it-lin-006:60010/master.jsp shows everything is normal.
And hbase.master.port is 60000 in hbase-default.xml
Some suggestions ?
Thanks
On Thu, Mar 25, 2010 at 4:21 PM, John Sichi <<m...@facebook.com>> wrote:
Don't copy hbase-site.xml; you don't want to set up a new HBase cluster. Just set your hive conf to point to your existing HBase master:
hbase.master=hbase.yoyodyne.com:60000<http://hbase.yoyodyne.com:60000/>
<http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage>http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage
JVS
On Mar 25, 2010, at 4:16 PM, Ted Yu wrote:
I got pass that error when I use CLI from trunk.
I copied hbase-site.xml from hbase master machine to trunk/dist/conf.
When I issue the following command in CLI:
CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat int)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
"hbase.key.type" = "string",
"hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
"<http://hbase.table.name/>hbase.table.name<http://hbase.table.name>" = "ruletable"
);
I get:
2010-03-25 15:49:52,557 WARN client.HConnectionManager$TableServers (HConnectionManager.java:tableExists(411)) - Testing for table existence threw exception
org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 10.10.31.143:60020<http://10.10.31.143:60020/> for region .META.,,1, row '', but failed after 10 attempts.
Exceptions:
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1048)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:454)
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.tableExists(HConnectionManager.java:404)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:113)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:100)
at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)
However, I don't see any exception in hbase-hbaseadmin-regionserver.log on 10.10.31.143 within the past 4 hours.
No exception in hbase-hbaseadmin-master.log on hbase master machine either.
Please comment.
On Thu, Mar 25, 2010 at 11:22 AM, Carl Steinbach <<m...@cloudera.com>> wrote:
Hi Ted,
It looks like your copy of Hive does not have the changes that implemented support for integration with HBase. HBase support was committed to trunk on March 12th, and currently it is only available on trunk. In order to use it you need to checkout (or update) the source from the svn repository. Instructions describing how to do this are on the Hive wiki.
Carl
On Thu, Mar 25, 2010 at 9:29 AM, Ted Yu <<m...@gmail.com>> wrote:
Hi,
I encountered Parse Error creating table in CLI:
[root@tyu-linux hive-0.5.0-bin]# bin/hive
Hive history file=/tmp/root/hive_job_log_root_201003240832_1752130264.txt
hive> SHOW TABLES;
OK
Time taken: 8.09 seconds
hive> CREATE EXTERNAL TABLE users(key string, state string, country string,
country_id int)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>
> WITH SERDEPROPERTIES (
>
> "hbase.key.type" = "string",
>
> "hbase.columns.mapping" = "info:state,info:country,info:country_id",
>
> "<http://hbase.table.name/>hbase.table.name<http://hbase.table.name>" = "users"
>
> );
FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table file
format specification
Here is the tail of hive.log:
2010-03-24 08:32:07,937 ERROR DataNucleus.Plugin
(Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
"org.eclipse.text" but it cannot be resolved.
2010-03-24 08:34:18,337 ERROR ql.Driver (SessionState.java:printError(248))
- FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
file format specification
org.apache.hadoop.hive.ql.parse.ParseException: line 2:0 cannot recognize
input 'STORED' in table file format specification
at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:357)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:267)
at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
If you know how to fix my query, please share.
Re: create table syntax in CLI
Posted by John Sichi <js...@facebook.com>.
Hmmm, what kind of HBase cluster are you running (standalone, pseudo-distributed, or distributed)? If distributed, you may need to pass zookeeper connection info instead of hbase.master; I haven't tested out distributed yet but will be soon.
JVS
On Mar 25, 2010, at 4:43 PM, Ted Yu wrote:
I removed hbase-site.xml
[root@tyu-linux dist]# bin/hive -hiveconf hbase.master=snv-it-lin-006:60000
Hive history file=/tmp/root/hive_job_log_root_201003251614_1493474415.txt
hive> CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat int)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
> "hbase.key.type" = "string",
> "hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
> "hbase.table.name<http://hbase.table.name/>" = "ruletable"
> );
FAILED: Error in metadata: MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:374)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:72)
at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:63)
at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:320)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:1445)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:124)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:630)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:382)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
)
But http://snv-it-lin-006:60010/master.jsp shows everything is normal.
And hbase.master.port is 60000 in hbase-default.xml
Some suggestions ?
Thanks
On Thu, Mar 25, 2010 at 4:21 PM, John Sichi <js...@facebook.com>> wrote:
Don't copy hbase-site.xml; you don't want to set up a new HBase cluster. Just set your hive conf to point to your existing HBase master:
hbase.master=hbase.yoyodyne.com:60000<http://hbase.yoyodyne.com:60000/>
http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage
JVS
On Mar 25, 2010, at 4:16 PM, Ted Yu wrote:
I got pass that error when I use CLI from trunk.
I copied hbase-site.xml from hbase master machine to trunk/dist/conf.
When I issue the following command in CLI:
CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat int)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
"hbase.key.type" = "string",
"hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
"hbase.table.name<http://hbase.table.name/>" = "ruletable"
);
I get:
2010-03-25 15:49:52,557 WARN client.HConnectionManager$TableServers (HConnectionManager.java:tableExists(411)) - Testing for table existence threw exception
org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 10.10.31.143:60020<http://10.10.31.143:60020/> for region .META.,,1, row '', but failed after 10 attempts.
Exceptions:
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1048)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:454)
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.tableExists(HConnectionManager.java:404)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:113)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:100)
at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)
However, I don't see any exception in hbase-hbaseadmin-regionserver.log on 10.10.31.143 within the past 4 hours.
No exception in hbase-hbaseadmin-master.log on hbase master machine either.
Please comment.
On Thu, Mar 25, 2010 at 11:22 AM, Carl Steinbach <ca...@cloudera.com>> wrote:
Hi Ted,
It looks like your copy of Hive does not have the changes that implemented support for integration with HBase. HBase support was committed to trunk on March 12th, and currently it is only available on trunk. In order to use it you need to checkout (or update) the source from the svn repository. Instructions describing how to do this are on the Hive wiki.
Carl
On Thu, Mar 25, 2010 at 9:29 AM, Ted Yu <yu...@gmail.com>> wrote:
Hi,
I encountered Parse Error creating table in CLI:
[root@tyu-linux hive-0.5.0-bin]# bin/hive
Hive history file=/tmp/root/hive_job_log_root_201003240832_1752130264.txt
hive> SHOW TABLES;
OK
Time taken: 8.09 seconds
hive> CREATE EXTERNAL TABLE users(key string, state string, country string,
country_id int)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>
> WITH SERDEPROPERTIES (
>
> "hbase.key.type" = "string",
>
> "hbase.columns.mapping" = "info:state,info:country,info:country_id",
>
> "hbase.table.name<http://hbase.table.name/>" = "users"
>
> );
FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table file
format specification
Here is the tail of hive.log:
2010-03-24 08:32:07,937 ERROR DataNucleus.Plugin
(Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
"org.eclipse.text" but it cannot be resolved.
2010-03-24 08:34:18,337 ERROR ql.Driver (SessionState.java:printError(248))
- FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
file format specification
org.apache.hadoop.hive.ql.parse.ParseException: line 2:0 cannot recognize
input 'STORED' in table file format specification
at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:357)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:267)
at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
If you know how to fix my query, please share.
Re: create table syntax in CLI
Posted by John Sichi <js...@facebook.com>.
Don't copy hbase-site.xml; you don't want to set up a new HBase cluster. Just set your hive conf to point to your existing HBase master:
hbase.master=hbase.yoyodyne.com:60000
http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage
JVS
On Mar 25, 2010, at 4:16 PM, Ted Yu wrote:
I got pass that error when I use CLI from trunk.
I copied hbase-site.xml from hbase master machine to trunk/dist/conf.
When I issue the following command in CLI:
CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat int)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
"hbase.key.type" = "string",
"hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
"hbase.table.name<http://hbase.table.name/>" = "ruletable"
);
I get:
2010-03-25 15:49:52,557 WARN client.HConnectionManager$TableServers (HConnectionManager.java:tableExists(411)) - Testing for table existence threw exception
org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 10.10.31.143:60020<http://10.10.31.143:60020/> for region .META.,,1, row '', but failed after 10 attempts.
Exceptions:
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1048)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:454)
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.tableExists(HConnectionManager.java:404)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:113)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:100)
at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)
However, I don't see any exception in hbase-hbaseadmin-regionserver.log on 10.10.31.143 within the past 4 hours.
No exception in hbase-hbaseadmin-master.log on hbase master machine either.
Please comment.
On Thu, Mar 25, 2010 at 11:22 AM, Carl Steinbach <ca...@cloudera.com>> wrote:
Hi Ted,
It looks like your copy of Hive does not have the changes that implemented support for integration with HBase. HBase support was committed to trunk on March 12th, and currently it is only available on trunk. In order to use it you need to checkout (or update) the source from the svn repository. Instructions describing how to do this are on the Hive wiki.
Carl
On Thu, Mar 25, 2010 at 9:29 AM, Ted Yu <yu...@gmail.com>> wrote:
Hi,
I encountered Parse Error creating table in CLI:
[root@tyu-linux hive-0.5.0-bin]# bin/hive
Hive history file=/tmp/root/hive_job_log_root_201003240832_1752130264.txt
hive> SHOW TABLES;
OK
Time taken: 8.09 seconds
hive> CREATE EXTERNAL TABLE users(key string, state string, country string,
country_id int)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>
> WITH SERDEPROPERTIES (
>
> "hbase.key.type" = "string",
>
> "hbase.columns.mapping" = "info:state,info:country,info:country_id",
>
> "hbase.table.name<http://hbase.table.name/>" = "users"
>
> );
FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table file
format specification
Here is the tail of hive.log:
2010-03-24 08:32:07,937 ERROR DataNucleus.Plugin
(Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
"org.eclipse.text" but it cannot be resolved.
2010-03-24 08:34:18,337 ERROR ql.Driver (SessionState.java:printError(248))
- FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
file format specification
org.apache.hadoop.hive.ql.parse.ParseException: line 2:0 cannot recognize
input 'STORED' in table file format specification
at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:357)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:267)
at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
If you know how to fix my query, please share.
Re: create table syntax in CLI
Posted by Ted Yu <yu...@gmail.com>.
I got pass that error when I use CLI from trunk.
I copied hbase-site.xml from hbase master machine to trunk/dist/conf.
When I issue the following command in CLI:
CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat
int)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
"hbase.key.type" = "string",
"hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
"hbase.table.name" = "ruletable"
);
I get:
2010-03-25 15:49:52,557 WARN client.HConnectionManager$TableServers
(HConnectionManager.java:tableExists(411)) - Testing for table existence
threw exception
org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact
region server 10.10.31.143:60020 for region .META.,,1, row '', but failed
after 10 attempts.
Exceptions:
java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
java.io.EOFException
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1048)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:454)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.tableExists(HConnectionManager.java:404)
at
org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:113)
at
org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:100)
at
org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)
However, I don't see any exception in hbase-hbaseadmin-regionserver.log on
10.10.31.143 within the past 4 hours.
No exception in hbase-hbaseadmin-master.log on hbase master machine either.
Please comment.
On Thu, Mar 25, 2010 at 11:22 AM, Carl Steinbach <ca...@cloudera.com> wrote:
> Hi Ted,
>
> It looks like your copy of Hive does not have the changes that implemented
> support for integration with HBase. HBase support was committed to trunk on
> March 12th, and currently it is only available on trunk. In order to use it
> you need to checkout (or update) the source from the svn repository.
> Instructions describing how to do this are on the Hive wiki.
>
> Carl
>
>
> On Thu, Mar 25, 2010 at 9:29 AM, Ted Yu <yu...@gmail.com> wrote:
>
>> Hi,
>> I encountered Parse Error creating table in CLI:
>>
>> [root@tyu-linux hive-0.5.0-bin]# bin/hive
>> Hive history file=/tmp/root/hive_job_log_root_201003240832_1752130264.txt
>> hive> SHOW TABLES;
>> OK
>> Time taken: 8.09 seconds
>> hive> CREATE EXTERNAL TABLE users(key string, state string, country
>> string,
>> country_id int)
>> > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>> >
>> > WITH SERDEPROPERTIES (
>> >
>> > "hbase.key.type" = "string",
>> >
>> > "hbase.columns.mapping" = "info:state,info:country,info:country_id",
>> >
>> > "hbase.table.name" = "users"
>> >
>> > );
>> FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
>> file
>> format specification
>>
>> *Here is the tail of hive.log:*
>> 2010-03-24 08:32:07,937 ERROR DataNucleus.Plugin
>> (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
>> "org.eclipse.text" but it cannot be resolved.
>> 2010-03-24 08:34:18,337 ERROR ql.Driver
>> (SessionState.java:printError(248))
>> - FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
>> file format specification
>>
>> org.apache.hadoop.hive.ql.parse.ParseException: line 2:0 cannot recognize
>> input 'STORED' in table file format specification
>>
>> at
>> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:357)
>> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:267)
>> at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>
>> If you know how to fix my query, please share.
>>
>
>
Re: create table syntax in CLI
Posted by Carl Steinbach <ca...@cloudera.com>.
Hi Ted,
It looks like your copy of Hive does not have the changes that implemented
support for integration with HBase. HBase support was committed to trunk on
March 12th, and currently it is only available on trunk. In order to use it
you need to checkout (or update) the source from the svn repository.
Instructions describing how to do this are on the Hive wiki.
Carl
On Thu, Mar 25, 2010 at 9:29 AM, Ted Yu <yu...@gmail.com> wrote:
> Hi,
> I encountered Parse Error creating table in CLI:
>
> [root@tyu-linux hive-0.5.0-bin]# bin/hive
> Hive history file=/tmp/root/hive_job_log_root_201003240832_1752130264.txt
> hive> SHOW TABLES;
> OK
> Time taken: 8.09 seconds
> hive> CREATE EXTERNAL TABLE users(key string, state string, country string,
> country_id int)
> > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> >
> > WITH SERDEPROPERTIES (
> >
> > "hbase.key.type" = "string",
> >
> > "hbase.columns.mapping" = "info:state,info:country,info:country_id",
> >
> > "hbase.table.name" = "users"
> >
> > );
> FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table file
> format specification
>
> *Here is the tail of hive.log:*
> 2010-03-24 08:32:07,937 ERROR DataNucleus.Plugin
> (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
> "org.eclipse.text" but it cannot be resolved.
> 2010-03-24 08:34:18,337 ERROR ql.Driver (SessionState.java:printError(248))
> - FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
> file format specification
>
> org.apache.hadoop.hive.ql.parse.ParseException: line 2:0 cannot recognize
> input 'STORED' in table file format specification
>
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:357)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:267)
> at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
> If you know how to fix my query, please share.
>