How do I make Hive use a custom scheduler and not the default scheduler?

[Ryan LeCompte <le...@gmail.com>]

Right now when we submit queries, it uses the "hadoop" scheduler. I have a
custom fair share scheduler configured as well, but I see that jobs
generated from our Hive queries never get picked up by that scheduler. Is
there something in hive-site.xml that I can configure to make all queries
use a particular scheduler?

Thanks,
Ryan

RE: Hive Instllation Problem - Pls help

["Katukuri, Jay" <jk...@ebay.com>]

Hello all,


On our cluster (using redhat enterprise 5), we have hadoop 0.18.3 installed.

When I do "show tables" on hive prompt, I get the following error

java.lang.NoSuchFieldError: type
        at org.apache.hadoop.hive.ql.parse.HiveLexer.mKW_SHOW(HiveLexer.java:1057)
        at org.apache.hadoop.hive.ql.parse.HiveLexer.mTokens(HiveLexer.java:5243)
        at org.antlr.runtime.Lexer.nextToken(Lexer.java:84)
        at org.antlr.runtime.CommonTokenStream.fillBuffer(CommonTokenStream.java:95)
        at org.antlr.runtime.CommonTokenStream.LT(CommonTokenStream.java:238)
        at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:404)
        at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:399)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:299)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:377)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
        at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)


I am getting this error for both binaries "hive_0.5.0" and "hive _0.4.1"

I have also built hive from source . I got the source code from http://svn.apache.org/repos/asf/hadoop/hive/trunk

I get the same error while using hive script from hive built from source.

I have tried different versions of "antlr_runtime jars" by placing them into the lib folder for both binaries and hive built from source. 

Any help is appreciated.

Thanks,
Jay

Re: How do I make Hive use a custom scheduler and not the default scheduler?

[Ryan LeCompte <le...@gmail.com>]

Thanks!

On Tue, Mar 23, 2010 at 12:39 PM, Zheng Shao <zs...@gmail.com> wrote:

> Hive also loads hadoop conf in HADOOP_HOME/conf. You can set it there.
>
> On 3/23/10, Ryan LeCompte <le...@gmail.com> wrote:
> > Right now when we submit queries, it uses the "hadoop" scheduler. I have
> a
> > custom fair share scheduler configured as well, but I see that jobs
> > generated from our Hive queries never get picked up by that scheduler. Is
> > there something in hive-site.xml that I can configure to make all queries
> > use a particular scheduler?
> >
> > Thanks,
> > Ryan
> >
>
> --
> Sent from my mobile device
>
> Yours,
> Zheng
>

Re: How do I make Hive use a custom scheduler and not the default scheduler?

[Zheng Shao <zs...@gmail.com>]

Hive also loads hadoop conf in HADOOP_HOME/conf. You can set it there.

On 3/23/10, Ryan LeCompte <le...@gmail.com> wrote:
> Right now when we submit queries, it uses the "hadoop" scheduler. I have a
> custom fair share scheduler configured as well, but I see that jobs
> generated from our Hive queries never get picked up by that scheduler. Is
> there something in hive-site.xml that I can configure to make all queries
> use a particular scheduler?
>
> Thanks,
> Ryan
>

-- 
Sent from my mobile device

Yours,
Zheng