Re: Not able to submit MR or Oozie jobs through knox-0.2.0

[Kevin Minder <ke...@hortonworks.com>]

Resending again now that I'm actually subscribed to user@knox with this 
account!

Hi Umesh,
I'd be very glad to work through this with you.

Let me start by asking that we continue this thread on the user@knox 
list which I've CCed.  Also I note that that since the WebHDFS 
operations (Delete, Mkdir, Put) are working we know that in general the 
gateway is setup properly.

Next, I'll need to help you gather some diagnostic information.

First, provide the output from the gateway server.  These are probably 
being written to standard output in the console where you are running 
the gateway.  By default this output is limited to INFO level messages 
and more severe.  You will probably want to set the logging level to 
DEBUG for several loggers.  This can be done by editing the 
log4j.properties file in {GATEWAY_HOME}/conf/log4j.properties.  You 
should probably set loggers log4j.logger.org.apache.hadoop.gateway and 
log4j.logger.org.apache.http.wire to DEBUG.  This is going to generate 
significant output but should provide all the information we need.

Second provide the log generated by Oozie.  This can typically be found 
at /var/log/oozie/oozie.log.

Since I don't believe that the Apache mailing lists allow attachements 
it might be a good idea to file a jira where these log files can be 
attached.

With info we should be able to get things straightened out.  If that 
proves difficult we might want you to try out the unreleased 0.3.0 as 
the diagnostics for this have been significantly improved.

Kevin.

On 9/27/13 5:14 AM, Chaudhari, Umesh Tulshiram wrote:
>
> HI kevin,
>
> I am using hive-0.11.0 in HDP-1.3.2 through knox-0.2.0 and trying to 
> run samples provided from Knox-0.2.0 samples,
>
> But its throwing me an error while submitting the job.
>
> I have checked settings for webhcat it seems fine, and all hadoop 
> ecosystem was installed using ambari only.
>
> I have also modified servers URLS to map them to my servers but it fails.
>
> Can you please help me what can be possibly going wrong,
>
> Please find the error here,
>
> bash-4.1# java -jar bin/shell.jar test_oozie.groovy
>
> Delete /tmp/test 200
>
> Mkdir /tmp/test 200
>
> Put /tmp/test/input/FILE 201
>
> Put /tmp/test/workflow.xml 201
>
> Put /tmp/test/lib/hadoop-examples.jar 201
>
> Caught: org.apache.hadoop.gateway.shell.HadoopException: 
> org.apache.hadoop.gateway.shell.ErrorResponse: HTTP/1.1 500 Server Error
>
> org.apache.hadoop.gateway.shell.HadoopException: 
> org.apache.hadoop.gateway.shell.ErrorResponse: HTTP/1.1 500 Server Error
>
> at 
> org.apache.hadoop.gateway.shell.AbstractRequest.now(AbstractRequest.java:72)
>
> at org.apache.hadoop.gateway.shell.AbstractRequest$now.call(Unknown 
> Source)
>
> at test_oozie.run(test_oozie.groovy:62)
>
> at org.apache.hadoop.gateway.shell.Shell.main(Shell.java:40)
>
> at 
> org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:64)
>
> at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:37)
>
> at org.apache.hadoop.gateway.launcher.Command.run(Command.java:100)
>
> at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:70)
>
> at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:49)
>
> Caused by: org.apache.hadoop.gateway.shell.ErrorResponse: HTTP/1.1 500 
> Server Error
>
> at org.apache.hadoop.gateway.shell.Hadoop.executeNow(Hadoop.java:107)
>
> at 
> org.apache.hadoop.gateway.shell.AbstractRequest.execute(AbstractRequest.java:47)
>
> at 
> org.apache.hadoop.gateway.shell.workflow.Submit$Request.access$500(Submit.java:38)
>
> at 
> org.apache.hadoop.gateway.shell.workflow.Submit$Request$1.call(Submit.java:77)
>
> at 
> org.apache.hadoop.gateway.shell.workflow.Submit$Request$1.call(Submit.java:64)
>
> at 
> org.apache.hadoop.gateway.shell.AbstractRequest.now(AbstractRequest.java:70)
>
> ... 8 more
>
> Thanks,
>
> Umesh
>
>
> ******************************************************
> This message and any files or attachments sent with this message 
> contain confidential information and is intended only for the 
> individual named. If you are not the named addressee, you should not 
> disseminate, distribute, copy or use any part of this email. If you 
> have received this message in error, please delete it and all copies 
> from your system and notify the sender immediately by return Email.
>
> Email transmission cannot be guaranteed to be secure or error-free as 
> information can be intercepted, corrupted, lost, destroyed, late, 
> incomplete or may contain viruses. The sender, therefore, does not 
> accept liability for any errors or omissions in the contents of this 
> message, which arise as a result of email transmission.
> ******************************************************


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.