You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Remus Rusanu <re...@microsoft.com> on 2014/02/06 22:34:53 UTC
Any way to preserve the .q unit tests MR execution output?
When running a test that fails, I can get the output diff and the hive log, but when the failure occurs in the query execution, I don't see how I can retrieve whatever was logged by the execution.
For example, I can see in the target\qfile-results\...q.out:
PREHOOK: query: SELECT cint,
...
AND (ctinyint > cbigint)))
PREHOOK: type: QUERY
PREHOOK: Input: default@alltypesorc
PREHOOK: Output: file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-mr-10000
Execution failed with exit status: 2
Obtaining error information
Task failed!
Task ID:
Stage-1
Logs:
/Users/rremus/Development/hive/itests/qtest/target/tmp/log/hive.log
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
The hive.log is a bit more detailed:
2014-02-06 12:17:05,741 INFO ql.Driver (SessionState.java:printInfo(504)) - Launching Job 1 out of 1
2014-02-06 12:17:05,742 INFO exec.Task (SessionState.java:printInfo(504)) - Number of reduce tasks is set to 0 since there's no reduce operator
2014-02-06 12:17:05,822 INFO mr.ExecDriver (MapRedTask.java:execute(175)) - Generating plan file file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-local-10003/plan.xml
2014-02-06 12:17:05,822 INFO log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=serializePlan from=org.apache.hadoop.hive.ql.exec.Utilities>
2014-02-06 12:17:05,823 INFO exec.Utilities (Utilities.java:serializePlan(783)) - Serializing MapredWork via kryo
2014-02-06 12:17:05,835 INFO log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=serializePlan start=1391717825822 end=1391717825835 duration=13 from=org.apache.hadoop.hive.ql.exec.Utilities>
2014-02-06 12:17:05,836 INFO mr.ExecDriver (MapRedTask.java:execute(207)) - Executing: /Users/rremus/Development/hive/itests/qtest/../../testutils/hadoop jar /Users/rremus/.m2/repository/org/apache/hive/hive-exec/0.13.0-SNAPSHOT/hive-exec-0.13.0-SNAPSHOT.jar org.apache.hadoop.hive.ql.exec.mr.ExecDriver -plan file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-local-10003/plan.xml -jobconffile file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-local-10002/jobconf.xml
2014-02-06 12:17:15,431 ERROR exec.Task (SessionState.java:printError(513)) - Execution failed with exit status: 2
2014-02-06 12:17:15,432 ERROR exec.Task (SessionState.java:printError(513)) - Obtaining error information
2014-02-06 12:17:15,432 ERROR exec.Task (SessionState.java:printError(513)) -
Task failed!
Task ID:
Stage-1
Logs:
2014-02-06 12:17:15,433 ERROR exec.Task (SessionState.java:printError(513)) - /Users/rremus/Development/hive/itests/qtest/target/tmp/log/hive.log
2014-02-06 12:17:15,433 ERROR mr.ExecDriver (MapRedTask.java:execute(280)) - Execution failed with exit status: 2
Still, at the end of running the test there is nothing left in target/tmp/localscratchdir, scratchdir or in target/tmp/hadoop-tmp. In this case, I'm missing the output from the MR job that failed.
Is there a way for me to get this missing output?
Thanks,
~Remus
RE: Any way to preserve the .q unit tests MR execution output?
Posted by Remus Rusanu <re...@microsoft.com>.
Thanks again, but no luck for me.
I started debugging this and my suspicion is it that is related to the fact that I do not have any Hadoop deployed on this laptop. When testutil/hadoop is run there is $HADOP_HOME to look at, no proper hadoop-env.sh or anything alike. I think this results in log4j not being properly initialized for the "MR" task. I'll post back if I ever get to the end of it.
~Remus
________________________________________
From: Brock Noland <br...@cloudera.com>
Sent: Saturday, February 8, 2014 7:30 PM
To: dev@hive.apache.org
Cc: hive-dev@hadoop.apache.org
Subject: Re: Any way to preserve the .q unit tests MR execution output?
I just ran a test on my local machine and the MR logs went to /tmp/brock.
Brock
On Sat, Feb 8, 2014 at 6:12 AM, Remus Rusanu <re...@microsoft.com> wrote:
> Thanks Brock,
>
> I do get a MR folder structure, like
>
> test/target/tmp/hadoop-tmp//mapred/local/1098407621/taskTracker/rremus/jobcache
> qtest/target/tmp/hadoop-tmp//mapred/local/1098407621/taskTracker/rremus/jobcache/job_local1514098477_0001
> qtest/target/tmp/hadoop-tmp//mapred/local/1098407621/taskTracker/rremus/jobcache/job_local1514098477_0001/attempt_local1514098477_0001_m_000000_0
> qtest/target/tmp/hadoop-tmp//mapred/local/1098407621/taskTracker/rremus/jobcache/job_local1514098477_0001/attempt_local1514098477_0001_r_000000_0
> qtest/target/tmp/hadoop-tmp//mapred/local/1098407621/taskTracker/rremus/jobcache/job_local1514098477_0001/attempt_local1514098477_0001_r_000000_0/output
>
> but no files in them. I'll try to dig and see whether the files get deleted or not created to start with...
>
> thanks,
> ~Remus
> ________________________________________
> From: Brock Noland <br...@cloudera.com>
> Sent: Thursday, February 6, 2014 11:58 PM
> To: dev@hive.apache.org
> Cc: hive-dev@hadoop.apache.org
> Subject: Re: Any way to preserve the .q unit tests MR execution output?
>
> Not sure about saving the output of the jobs, but I believe the mr
> logs should be in target/ somewhere or in /tmp/$USER
>
> On Thu, Feb 6, 2014 at 3:34 PM, Remus Rusanu <re...@microsoft.com> wrote:
>> When running a test that fails, I can get the output diff and the hive log, but when the failure occurs in the query execution, I don't see how I can retrieve whatever was logged by the execution.
>>
>> For example, I can see in the target\qfile-results\...q.out:
>>
>>
>> PREHOOK: query: SELECT cint,
>> ...
>> AND (ctinyint > cbigint)))
>> PREHOOK: type: QUERY
>> PREHOOK: Input: default@alltypesorc
>> PREHOOK: Output: file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-mr-10000
>> Execution failed with exit status: 2
>> Obtaining error information
>>
>> Task failed!
>> Task ID:
>> Stage-1
>>
>> Logs:
>>
>> /Users/rremus/Development/hive/itests/qtest/target/tmp/log/hive.log
>> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
>>
>> The hive.log is a bit more detailed:
>> 2014-02-06 12:17:05,741 INFO ql.Driver (SessionState.java:printInfo(504)) - Launching Job 1 out of 1
>> 2014-02-06 12:17:05,742 INFO exec.Task (SessionState.java:printInfo(504)) - Number of reduce tasks is set to 0 since there's no reduce operator
>> 2014-02-06 12:17:05,822 INFO mr.ExecDriver (MapRedTask.java:execute(175)) - Generating plan file file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-local-10003/plan.xml
>> 2014-02-06 12:17:05,822 INFO log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=serializePlan from=org.apache.hadoop.hive.ql.exec.Utilities>
>> 2014-02-06 12:17:05,823 INFO exec.Utilities (Utilities.java:serializePlan(783)) - Serializing MapredWork via kryo
>> 2014-02-06 12:17:05,835 INFO log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=serializePlan start=1391717825822 end=1391717825835 duration=13 from=org.apache.hadoop.hive.ql.exec.Utilities>
>> 2014-02-06 12:17:05,836 INFO mr.ExecDriver (MapRedTask.java:execute(207)) - Executing: /Users/rremus/Development/hive/itests/qtest/../../testutils/hadoop jar /Users/rremus/.m2/repository/org/apache/hive/hive-exec/0.13.0-SNAPSHOT/hive-exec-0.13.0-SNAPSHOT.jar org.apache.hadoop.hive.ql.exec.mr.ExecDriver -plan file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-local-10003/plan.xml -jobconffile file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-local-10002/jobconf.xml
>> 2014-02-06 12:17:15,431 ERROR exec.Task (SessionState.java:printError(513)) - Execution failed with exit status: 2
>> 2014-02-06 12:17:15,432 ERROR exec.Task (SessionState.java:printError(513)) - Obtaining error information
>> 2014-02-06 12:17:15,432 ERROR exec.Task (SessionState.java:printError(513)) -
>> Task failed!
>> Task ID:
>> Stage-1
>>
>> Logs:
>>
>> 2014-02-06 12:17:15,433 ERROR exec.Task (SessionState.java:printError(513)) - /Users/rremus/Development/hive/itests/qtest/target/tmp/log/hive.log
>> 2014-02-06 12:17:15,433 ERROR mr.ExecDriver (MapRedTask.java:execute(280)) - Execution failed with exit status: 2
>>
>> Still, at the end of running the test there is nothing left in target/tmp/localscratchdir, scratchdir or in target/tmp/hadoop-tmp. In this case, I'm missing the output from the MR job that failed.
>> Is there a way for me to get this missing output?
>>
>> Thanks,
>> ~Remus
>>
>>
>>
>>
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
Re: Any way to preserve the .q unit tests MR execution output?
Posted by Brock Noland <br...@cloudera.com>.
I just ran a test on my local machine and the MR logs went to /tmp/brock.
Brock
On Sat, Feb 8, 2014 at 6:12 AM, Remus Rusanu <re...@microsoft.com> wrote:
> Thanks Brock,
>
> I do get a MR folder structure, like
>
> test/target/tmp/hadoop-tmp//mapred/local/1098407621/taskTracker/rremus/jobcache
> qtest/target/tmp/hadoop-tmp//mapred/local/1098407621/taskTracker/rremus/jobcache/job_local1514098477_0001
> qtest/target/tmp/hadoop-tmp//mapred/local/1098407621/taskTracker/rremus/jobcache/job_local1514098477_0001/attempt_local1514098477_0001_m_000000_0
> qtest/target/tmp/hadoop-tmp//mapred/local/1098407621/taskTracker/rremus/jobcache/job_local1514098477_0001/attempt_local1514098477_0001_r_000000_0
> qtest/target/tmp/hadoop-tmp//mapred/local/1098407621/taskTracker/rremus/jobcache/job_local1514098477_0001/attempt_local1514098477_0001_r_000000_0/output
>
> but no files in them. I'll try to dig and see whether the files get deleted or not created to start with...
>
> thanks,
> ~Remus
> ________________________________________
> From: Brock Noland <br...@cloudera.com>
> Sent: Thursday, February 6, 2014 11:58 PM
> To: dev@hive.apache.org
> Cc: hive-dev@hadoop.apache.org
> Subject: Re: Any way to preserve the .q unit tests MR execution output?
>
> Not sure about saving the output of the jobs, but I believe the mr
> logs should be in target/ somewhere or in /tmp/$USER
>
> On Thu, Feb 6, 2014 at 3:34 PM, Remus Rusanu <re...@microsoft.com> wrote:
>> When running a test that fails, I can get the output diff and the hive log, but when the failure occurs in the query execution, I don't see how I can retrieve whatever was logged by the execution.
>>
>> For example, I can see in the target\qfile-results\...q.out:
>>
>>
>> PREHOOK: query: SELECT cint,
>> ...
>> AND (ctinyint > cbigint)))
>> PREHOOK: type: QUERY
>> PREHOOK: Input: default@alltypesorc
>> PREHOOK: Output: file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-mr-10000
>> Execution failed with exit status: 2
>> Obtaining error information
>>
>> Task failed!
>> Task ID:
>> Stage-1
>>
>> Logs:
>>
>> /Users/rremus/Development/hive/itests/qtest/target/tmp/log/hive.log
>> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
>>
>> The hive.log is a bit more detailed:
>> 2014-02-06 12:17:05,741 INFO ql.Driver (SessionState.java:printInfo(504)) - Launching Job 1 out of 1
>> 2014-02-06 12:17:05,742 INFO exec.Task (SessionState.java:printInfo(504)) - Number of reduce tasks is set to 0 since there's no reduce operator
>> 2014-02-06 12:17:05,822 INFO mr.ExecDriver (MapRedTask.java:execute(175)) - Generating plan file file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-local-10003/plan.xml
>> 2014-02-06 12:17:05,822 INFO log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=serializePlan from=org.apache.hadoop.hive.ql.exec.Utilities>
>> 2014-02-06 12:17:05,823 INFO exec.Utilities (Utilities.java:serializePlan(783)) - Serializing MapredWork via kryo
>> 2014-02-06 12:17:05,835 INFO log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=serializePlan start=1391717825822 end=1391717825835 duration=13 from=org.apache.hadoop.hive.ql.exec.Utilities>
>> 2014-02-06 12:17:05,836 INFO mr.ExecDriver (MapRedTask.java:execute(207)) - Executing: /Users/rremus/Development/hive/itests/qtest/../../testutils/hadoop jar /Users/rremus/.m2/repository/org/apache/hive/hive-exec/0.13.0-SNAPSHOT/hive-exec-0.13.0-SNAPSHOT.jar org.apache.hadoop.hive.ql.exec.mr.ExecDriver -plan file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-local-10003/plan.xml -jobconffile file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-local-10002/jobconf.xml
>> 2014-02-06 12:17:15,431 ERROR exec.Task (SessionState.java:printError(513)) - Execution failed with exit status: 2
>> 2014-02-06 12:17:15,432 ERROR exec.Task (SessionState.java:printError(513)) - Obtaining error information
>> 2014-02-06 12:17:15,432 ERROR exec.Task (SessionState.java:printError(513)) -
>> Task failed!
>> Task ID:
>> Stage-1
>>
>> Logs:
>>
>> 2014-02-06 12:17:15,433 ERROR exec.Task (SessionState.java:printError(513)) - /Users/rremus/Development/hive/itests/qtest/target/tmp/log/hive.log
>> 2014-02-06 12:17:15,433 ERROR mr.ExecDriver (MapRedTask.java:execute(280)) - Execution failed with exit status: 2
>>
>> Still, at the end of running the test there is nothing left in target/tmp/localscratchdir, scratchdir or in target/tmp/hadoop-tmp. In this case, I'm missing the output from the MR job that failed.
>> Is there a way for me to get this missing output?
>>
>> Thanks,
>> ~Remus
>>
>>
>>
>>
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
RE: Any way to preserve the .q unit tests MR execution output?
Posted by Remus Rusanu <re...@microsoft.com>.
Thanks Brock,
I do get a MR folder structure, like
test/target/tmp/hadoop-tmp//mapred/local/1098407621/taskTracker/rremus/jobcache
qtest/target/tmp/hadoop-tmp//mapred/local/1098407621/taskTracker/rremus/jobcache/job_local1514098477_0001
qtest/target/tmp/hadoop-tmp//mapred/local/1098407621/taskTracker/rremus/jobcache/job_local1514098477_0001/attempt_local1514098477_0001_m_000000_0
qtest/target/tmp/hadoop-tmp//mapred/local/1098407621/taskTracker/rremus/jobcache/job_local1514098477_0001/attempt_local1514098477_0001_r_000000_0
qtest/target/tmp/hadoop-tmp//mapred/local/1098407621/taskTracker/rremus/jobcache/job_local1514098477_0001/attempt_local1514098477_0001_r_000000_0/output
but no files in them. I'll try to dig and see whether the files get deleted or not created to start with...
thanks,
~Remus
________________________________________
From: Brock Noland <br...@cloudera.com>
Sent: Thursday, February 6, 2014 11:58 PM
To: dev@hive.apache.org
Cc: hive-dev@hadoop.apache.org
Subject: Re: Any way to preserve the .q unit tests MR execution output?
Not sure about saving the output of the jobs, but I believe the mr
logs should be in target/ somewhere or in /tmp/$USER
On Thu, Feb 6, 2014 at 3:34 PM, Remus Rusanu <re...@microsoft.com> wrote:
> When running a test that fails, I can get the output diff and the hive log, but when the failure occurs in the query execution, I don't see how I can retrieve whatever was logged by the execution.
>
> For example, I can see in the target\qfile-results\...q.out:
>
>
> PREHOOK: query: SELECT cint,
> ...
> AND (ctinyint > cbigint)))
> PREHOOK: type: QUERY
> PREHOOK: Input: default@alltypesorc
> PREHOOK: Output: file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-mr-10000
> Execution failed with exit status: 2
> Obtaining error information
>
> Task failed!
> Task ID:
> Stage-1
>
> Logs:
>
> /Users/rremus/Development/hive/itests/qtest/target/tmp/log/hive.log
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
>
> The hive.log is a bit more detailed:
> 2014-02-06 12:17:05,741 INFO ql.Driver (SessionState.java:printInfo(504)) - Launching Job 1 out of 1
> 2014-02-06 12:17:05,742 INFO exec.Task (SessionState.java:printInfo(504)) - Number of reduce tasks is set to 0 since there's no reduce operator
> 2014-02-06 12:17:05,822 INFO mr.ExecDriver (MapRedTask.java:execute(175)) - Generating plan file file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-local-10003/plan.xml
> 2014-02-06 12:17:05,822 INFO log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=serializePlan from=org.apache.hadoop.hive.ql.exec.Utilities>
> 2014-02-06 12:17:05,823 INFO exec.Utilities (Utilities.java:serializePlan(783)) - Serializing MapredWork via kryo
> 2014-02-06 12:17:05,835 INFO log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=serializePlan start=1391717825822 end=1391717825835 duration=13 from=org.apache.hadoop.hive.ql.exec.Utilities>
> 2014-02-06 12:17:05,836 INFO mr.ExecDriver (MapRedTask.java:execute(207)) - Executing: /Users/rremus/Development/hive/itests/qtest/../../testutils/hadoop jar /Users/rremus/.m2/repository/org/apache/hive/hive-exec/0.13.0-SNAPSHOT/hive-exec-0.13.0-SNAPSHOT.jar org.apache.hadoop.hive.ql.exec.mr.ExecDriver -plan file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-local-10003/plan.xml -jobconffile file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-local-10002/jobconf.xml
> 2014-02-06 12:17:15,431 ERROR exec.Task (SessionState.java:printError(513)) - Execution failed with exit status: 2
> 2014-02-06 12:17:15,432 ERROR exec.Task (SessionState.java:printError(513)) - Obtaining error information
> 2014-02-06 12:17:15,432 ERROR exec.Task (SessionState.java:printError(513)) -
> Task failed!
> Task ID:
> Stage-1
>
> Logs:
>
> 2014-02-06 12:17:15,433 ERROR exec.Task (SessionState.java:printError(513)) - /Users/rremus/Development/hive/itests/qtest/target/tmp/log/hive.log
> 2014-02-06 12:17:15,433 ERROR mr.ExecDriver (MapRedTask.java:execute(280)) - Execution failed with exit status: 2
>
> Still, at the end of running the test there is nothing left in target/tmp/localscratchdir, scratchdir or in target/tmp/hadoop-tmp. In this case, I'm missing the output from the MR job that failed.
> Is there a way for me to get this missing output?
>
> Thanks,
> ~Remus
>
>
>
>
--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
Re: Any way to preserve the .q unit tests MR execution output?
Posted by Brock Noland <br...@cloudera.com>.
Not sure about saving the output of the jobs, but I believe the mr
logs should be in target/ somewhere or in /tmp/$USER
On Thu, Feb 6, 2014 at 3:34 PM, Remus Rusanu <re...@microsoft.com> wrote:
> When running a test that fails, I can get the output diff and the hive log, but when the failure occurs in the query execution, I don't see how I can retrieve whatever was logged by the execution.
>
> For example, I can see in the target\qfile-results\...q.out:
>
>
> PREHOOK: query: SELECT cint,
> ...
> AND (ctinyint > cbigint)))
> PREHOOK: type: QUERY
> PREHOOK: Input: default@alltypesorc
> PREHOOK: Output: file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-mr-10000
> Execution failed with exit status: 2
> Obtaining error information
>
> Task failed!
> Task ID:
> Stage-1
>
> Logs:
>
> /Users/rremus/Development/hive/itests/qtest/target/tmp/log/hive.log
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
>
> The hive.log is a bit more detailed:
> 2014-02-06 12:17:05,741 INFO ql.Driver (SessionState.java:printInfo(504)) - Launching Job 1 out of 1
> 2014-02-06 12:17:05,742 INFO exec.Task (SessionState.java:printInfo(504)) - Number of reduce tasks is set to 0 since there's no reduce operator
> 2014-02-06 12:17:05,822 INFO mr.ExecDriver (MapRedTask.java:execute(175)) - Generating plan file file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-local-10003/plan.xml
> 2014-02-06 12:17:05,822 INFO log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=serializePlan from=org.apache.hadoop.hive.ql.exec.Utilities>
> 2014-02-06 12:17:05,823 INFO exec.Utilities (Utilities.java:serializePlan(783)) - Serializing MapredWork via kryo
> 2014-02-06 12:17:05,835 INFO log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=serializePlan start=1391717825822 end=1391717825835 duration=13 from=org.apache.hadoop.hive.ql.exec.Utilities>
> 2014-02-06 12:17:05,836 INFO mr.ExecDriver (MapRedTask.java:execute(207)) - Executing: /Users/rremus/Development/hive/itests/qtest/../../testutils/hadoop jar /Users/rremus/.m2/repository/org/apache/hive/hive-exec/0.13.0-SNAPSHOT/hive-exec-0.13.0-SNAPSHOT.jar org.apache.hadoop.hive.ql.exec.mr.ExecDriver -plan file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-local-10003/plan.xml -jobconffile file:/Users/rremus/Development/hive/itests/qtest/target/tmp/localscratchdir/hive_2014-02-06_12-17-05_641_8095220136958689426-1/-local-10002/jobconf.xml
> 2014-02-06 12:17:15,431 ERROR exec.Task (SessionState.java:printError(513)) - Execution failed with exit status: 2
> 2014-02-06 12:17:15,432 ERROR exec.Task (SessionState.java:printError(513)) - Obtaining error information
> 2014-02-06 12:17:15,432 ERROR exec.Task (SessionState.java:printError(513)) -
> Task failed!
> Task ID:
> Stage-1
>
> Logs:
>
> 2014-02-06 12:17:15,433 ERROR exec.Task (SessionState.java:printError(513)) - /Users/rremus/Development/hive/itests/qtest/target/tmp/log/hive.log
> 2014-02-06 12:17:15,433 ERROR mr.ExecDriver (MapRedTask.java:execute(280)) - Execution failed with exit status: 2
>
> Still, at the end of running the test there is nothing left in target/tmp/localscratchdir, scratchdir or in target/tmp/hadoop-tmp. In this case, I'm missing the output from the MR job that failed.
> Is there a way for me to get this missing output?
>
> Thanks,
> ~Remus
>
>
>
>
--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org