Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [9]

[Evan Pollan <Ev...@bazaarvoice.com>]

I'm just getting started playing around with oozie, and I hit a brick wall trying to run a simple workflow with one hive action.

For context, I'm using the Cloudera CDH3U2 distribution for both the oozie server and the hadoop cluster.  I've seeded all the CDH3U2 hive libraries onto HDFS, and my oozie-libpath points to that HDFS location.  After a couple of early failures, I sorted out the absence of a my-hive-default.xml and got that in the right place.

Now, when I run the job, it goes from PREP to RUNNING.  I see the prep m-r job complete in the job tracker UI.  A little bit later, oozie reports the job as "KILLED", and the action status as "ERROR".  However, when I look at the job log in the oozie web UI, there's really nothing helpful:

2012-02-05 14:57:45,424  WARN ActionStartCommand:528 - USER[ec2-user] GROUP[users] TOKEN[] APP[hive-wf] JOB[0000005-120202204303244-oozie-oozi-W] ACTION[-] [***0000005-120202204303244-oozie-oozi-W@hive-node***]In call()....status=PREP
2012-02-05 14:57:47,818  INFO HiveActionExecutor:525 - USER[ec2-user] GROUP[users] TOKEN[] APP[hive-wf] JOB[0000005-120202204303244-oozie-oozi-W] ACTION[0000005-120202204303244-oozie-oozi-W@hive-node] checking action, external ID [job_201202042004_0003] status [RUNNING]
2012-02-05 14:57:47,820  WARN ActionStartCommand:528 - USER[ec2-user] GROUP[users] TOKEN[] APP[hive-wf] JOB[0000005-120202204303244-oozie-oozi-W] ACTION[0000005-120202204303244-oozie-oozi-W@hive-node] [***0000005-120202204303244-oozie-oozi-W@hive-node***]Action status=RUNNING
2012-02-05 14:57:47,824  WARN ActionStartCommand:528 - USER[ec2-user] GROUP[users] TOKEN[] APP[hive-wf] JOB[0000005-120202204303244-oozie-oozi-W] ACTION[0000005-120202204303244-oozie-oozi-W@hive-node] [***0000005-120202204303244-oozie-oozi-W@hive-node***]Action updated in DB!
2012-02-05 15:08:43,287  INFO HiveActionExecutor:525 - USER[ec2-user] GROUP[users] TOKEN[] APP[hive-wf] JOB[0000005-120202204303244-oozie-oozi-W] ACTION[0000005-120202204303244-oozie-oozi-W@hive-node] action completed, external ID [job_201202042004_0003]
2012-02-05 15:08:43,351  WARN HiveActionExecutor:528 - USER[ec2-user] GROUP[users] TOKEN[] APP[hive-wf] JOB[0000005-120202204303244-oozie-oozi-W] ACTION[0000005-120202204303244-oozie-oozi-W@hive-node] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [9]
2012-02-05 15:08:43,429  INFO ActionEndCommand:525 - USER[ec2-user] GROUP[users] TOKEN[] APP[hive-wf] JOB[0000005-120202204303244-oozie-oozi-W] ACTION[0000005-120202204303244-oozie-oozi-W@hive-node] ERROR is considered as FAILED for SLA
2012-02-05 15:08:43,486  INFO CoordActionUpdateCommand:525 - USER[ec2-user] GROUP[users] TOKEN[] APP[hive-wf] JOB[0000005-120202204303244-oozie-oozi-W] ACTION[0000005-120202204303244-oozie-oozi-W@hive-node] STARTED CoordActionUpdateCommand for wfId=0000005-120202204303244-oozie-oozi-W
2012-02-05 15:08:43,488  INFO CoordActionUpdateCommand:525 - USER[ec2-user] GROUP[users] TOKEN[] APP[hive-wf] JOB[0000005-120202204303244-oozie-oozi-W] ACTION[0000005-120202204303244-oozie-oozi-W@hive-node] ENDED CoordActionUpdateCommand for wfId=0000005-120202204303244-oozie-oozi-W, coord action is null

Previously, when my workflow wasn't set up properly, the job log would clearly indicate the problem (e.g. A missing my-hive-default.xml or a missing hive library).  Now, it just says " Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [9]"

I looked at the various log files in /var/log/oozie, but didn't see anything helpful.  Where can I go to find out what exactly caused the failure?

Thanks,
Evan