You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hive.apache.org by Bill Au <bi...@gmail.com> on 2008/12/04 22:59:29 UTC

Apache Access Log Table example in Hive user guide

I am new to Hive.  I am using Hadoop 0.19.0 and am running the version of
Hive packaged with it under contrib/hive.  I am following the apache access
log example in the Hive user guide Wiki:

http://wiki.apache.org/hadoop/Hive/UserGuide

I cut and pasted the create table statement from the Wiki but it is giving
me an error:

$ bin/hive
hive> CREATE TABLE apachelog (
    > ipaddress STRING, identd STRING, user STRING,finishtime STRING,
    > requestline string, returncode INT, size INT)
    > ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe'
    > WITH SERDEPROPERTIES (
    >
'serialization.format'='org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol',
    > 'quote.delim'='("|\\[|\\])',
    > 'field.delim'=' ',
    > 'serialization.null.format'='-')
    > STORED AS TEXTFILE;
FAILED: Parse Error: line 1:132 cannot recognize input 'ROW'

java.lang.NullPointerException
        at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:221)
        at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:112)
        at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:137)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:234)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:165)
        at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)

$

What did I miss?  Or is this example valid?

Bill

Re: Apache Access Log Table example in Hive user guide

Posted by Josh Ferguson <jo...@besquared.net>.

I received this error when I did not have a trailing space at the end  
of my lines. The parser needs to be a little more robust in this area,  
I think there is an issue up for this already.

Josh Ferguson

On Dec 4, 2008, at 1:59 PM, Bill Au wrote:

> I am new to Hive.  I am using Hadoop 0.19.0 and am running the  
> version of Hive packaged with it under contrib/hive.  I am following  
> the apache access log example in the Hive user guide Wiki:
>
> http://wiki.apache.org/hadoop/Hive/UserGuide
>
> I cut and pasted the create table statement from the Wiki but it is  
> giving me an error:
>
> $ bin/hive
> hive> CREATE TABLE apachelog (
>     > ipaddress STRING, identd STRING, user STRING,finishtime STRING,
>     > requestline string, returncode INT, size INT)
>     > ROW FORMAT SERDE  
> 'org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe'
>     > WITH SERDEPROPERTIES (
>     >  
> 'serialization 
> .format'='org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol',
>     > 'quote.delim'='("|\\[|\\])',
>     > 'field.delim'=' ',
>     > 'serialization.null.format'='-')
>     > STORED AS TEXTFILE;
> FAILED: Parse Error: line 1:132 cannot recognize input 'ROW'
>
> java.lang.NullPointerException
>         at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java: 
> 221)
>         at  
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:112)
>         at  
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:137)
>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java: 
> 234)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at  
> sun 
> .reflect 
> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at  
> sun 
> .reflect 
> .DelegatingMethodAccessorImpl 
> .invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:165)
>         at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>         at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)
>
> $
>
> What did I miss?  Or is this example valid?
>
> Bill

Re: Apache Access Log Table example in Hive user guide

Posted by Pete Wyckoff <pw...@facebook.com>.

Hi Bill,

That version of hive does not have support for Apache Log Format. You would need one of the newer versions from contrib or the new repository.

Thanks, pete


On 12/4/08 1:59 PM, "Bill Au" <bi...@gmail.com> wrote:

I am new to Hive.  I am using Hadoop 0.19.0 and am running the version of Hive packaged with it under contrib/hive.  I am following the apache access log example in the Hive user guide Wiki:

http://wiki.apache.org/hadoop/Hive/UserGuide

I cut and pasted the create table statement from the Wiki but it is giving me an error:

$ bin/hive
hive> CREATE TABLE apachelog (
    > ipaddress STRING, identd STRING, user STRING,finishtime STRING,
    > requestline string, returncode INT, size INT)
    > ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe'
    > WITH SERDEPROPERTIES (
    > 'serialization.format'='org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol',
    > 'quote.delim'='("|\\[|\\])',
    > 'field.delim'=' ',
    > 'serialization.null.format'='-')
    > STORED AS TEXTFILE;
FAILED: Parse Error: line 1:132 cannot recognize input 'ROW'

java.lang.NullPointerException
        at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:221)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:112)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:137)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:234)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:165)
        at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)

$

What did I miss?  Or is this example valid?

Bill

RE: Apache Access Log Table example in Hive user guide

Posted by Joydeep Sen Sarma <js...@facebook.com>.

Please use hive from http://svn.apache.org/repos/asf/hadoop/hive/trunk/

This should work with hadoop-0.19.

Will update UserGuide with this info ..

________________________________
From: Bill Au [mailto:bill.w.au@gmail.com]
Sent: Thursday, December 04, 2008 1:59 PM
To: hive-user@hadoop.apache.org
Subject: Apache Access Log Table example in Hive user guide

I am new to Hive.  I am using Hadoop 0.19.0 and am running the version of Hive packaged with it under contrib/hive.  I am following the apache access log example in the Hive user guide Wiki:

http://wiki.apache.org/hadoop/Hive/UserGuide

I cut and pasted the create table statement from the Wiki but it is giving me an error:

$ bin/hive
hive> CREATE TABLE apachelog (
    > ipaddress STRING, identd STRING, user STRING,finishtime STRING,
    > requestline string, returncode INT, size INT)
    > ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe'
    > WITH SERDEPROPERTIES (
    > 'serialization.format'='org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol',
    > 'quote.delim'='("|\\[|\\])',
    > 'field.delim'=' ',
    > 'serialization.null.format'='-')
    > STORED AS TEXTFILE;
FAILED: Parse Error: line 1:132 cannot recognize input 'ROW'

java.lang.NullPointerException
        at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:221)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:112)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:137)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:234)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:165)
        at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)

$

What did I miss?  Or is this example valid?

Bill