You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Syed Albiz <s....@gmail.com> on 2011/08/03 02:40:14 UTC
Re: Review Request: HIVE-2156: Improve Execution Error Messages
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/777/
-----------------------------------------------------------
(Updated 2011-08-03 00:40:14.013522)
Review request for hive and John Sichi.
Changes
-------
regenerated patch against testcase output
Summary
-------
- Add local error messages to point to job logs and provide TaskIDs
- Add a timeout to the fetching of task logs and errors
This addresses bug HIVE-2156.
https://issues.apache.org/jira/browse/HIVE-2156
Diffs (updated)
-----
build-common.xml 4856c5f
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java b46976f
conf/hive-default.xml 3a4f833
ql/build.xml 449b47a
ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1c6f092
ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java e687b1a
ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java 3d5e95d
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java cec0d46
ql/src/test/results/clientnegative/dyn_part3.q.out 5f4df65
ql/src/test/results/clientnegative/index_compact_entry_limit.q.out fcb2673
ql/src/test/results/clientnegative/index_compact_size_limit.q.out fcb2673
ql/src/test/results/clientnegative/script_broken_pipe1.q.out d33d2cc
ql/src/test/results/clientnegative/script_broken_pipe2.q.out afbaa44
ql/src/test/results/clientnegative/script_broken_pipe3.q.out fe8f757
ql/src/test/results/clientnegative/script_error.q.out c72d780
ql/src/test/results/clientnegative/udf_reflect_neg.q.out f2082a3
ql/src/test/results/clientnegative/udf_test_error.q.out 5fd9a00
ql/src/test/results/clientnegative/udf_test_error_reduce.q.out ddc5e5b
ql/src/test/templates/TestNegativeCliDriver.vm ec13f79
Diff: https://reviews.apache.org/r/777/diff
Testing
-------
Tested TestNegativeCliDriver in both local and miniMR mode
Thanks,
Syed
Re: Review Request: HIVE-2156: Improve Execution Error Messages
Posted by M IS <mi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/777/#review1445
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java
<https://reviews.apache.org/r/777/#comment3345>
assert(jobId.equals(ti.getJobId())) would have been enough
ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java
<https://reviews.apache.org/r/777/#comment3346>
Please take the declaration out of the loop
ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java
<https://reviews.apache.org/r/777/#comment3347>
the condition could have been much simpler this way:
jobId.equals(ti.getJobId())
ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java
<https://reviews.apache.org/r/777/#comment3348>
invoking size() directly on the failures map would have been enough
ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java
<https://reviews.apache.org/r/777/#comment3350>
Please use java.util.Collections.max() inplace of the existing logic.
ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java
<https://reviews.apache.org/r/777/#comment3349>
Please use entry set to loop over the failures map, instead of using the keyset
- M
On 2011-08-11 01:07:35, Syed Albiz wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/777/
> -----------------------------------------------------------
>
> (Updated 2011-08-11 01:07:35)
>
>
> Review request for hive and John Sichi.
>
>
> Summary
> -------
>
> - Add local error messages to point to job logs and provide TaskIDs
> - Add a timeout to the fetching of task logs and errors
>
>
> This addresses bug HIVE-2156.
> https://issues.apache.org/jira/browse/HIVE-2156
>
>
> Diffs
> -----
>
> build-common.xml 4856c5f
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java b46976f
> conf/hive-default.xml 3a4f833
> contrib/src/test/results/clientnegative/case_with_row_sequence.q.out 4447c65
> ql/build.xml 449b47a
> ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1c6f092
> ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java e687b1a
> ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java 3d5e95d
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java cec0d46
> ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java 8a50dc6
> ql/src/test/queries/clientnegative/minimr_broken_pipe.q PRE-CREATION
> ql/src/test/results/clientnegative/dyn_part3.q.out 5f4df65
> ql/src/test/results/clientnegative/index_compact_entry_limit.q.out 06437c7
> ql/src/test/results/clientnegative/index_compact_size_limit.q.out 54f1262
> ql/src/test/results/clientnegative/minimr_broken_pipe.q.out PRE-CREATION
> ql/src/test/results/clientnegative/script_broken_pipe1.q.out d33d2cc
> ql/src/test/results/clientnegative/script_broken_pipe2.q.out afbaa44
> ql/src/test/results/clientnegative/script_broken_pipe3.q.out fe8f757
> ql/src/test/results/clientnegative/script_error.q.out c72d780
> ql/src/test/results/clientnegative/udf_reflect_neg.q.out f2082a3
> ql/src/test/results/clientnegative/udf_test_error.q.out 5fd9a00
> ql/src/test/results/clientnegative/udf_test_error_reduce.q.out ddc5e5b
> ql/src/test/results/clientpositive/auto_join25.q.out 362b2fa
> ql/src/test/results/clientpositive/mapjoin_hook.q.out acdeefa
> ql/src/test/templates/TestNegativeCliDriver.vm ec13f79
>
> Diff: https://reviews.apache.org/r/777/diff
>
>
> Testing
> -------
>
> Tested TestNegativeCliDriver in both local and miniMR mode
>
>
> Thanks,
>
> Syed
>
>
Re: Review Request: HIVE-2156: Improve Execution Error Messages
Posted by Syed Albiz <s....@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/777/
-----------------------------------------------------------
(Updated 2011-08-11 01:07:35.822886)
Review request for hive and John Sichi.
Changes
-------
The previous patch was failing as I generated it using '-Doverwrite=true', and minimr_broken_pipe.q was being written over twice. I have fixed the patch to exclude minimr_broken_pipe.q from the regular TestNegativeCliDriver test run, and also updated the diff ignores to prevent the TestNegativeMinimrCliDriver failure
Summary
-------
- Add local error messages to point to job logs and provide TaskIDs
- Add a timeout to the fetching of task logs and errors
This addresses bug HIVE-2156.
https://issues.apache.org/jira/browse/HIVE-2156
Diffs (updated)
-----
build-common.xml 4856c5f
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java b46976f
conf/hive-default.xml 3a4f833
contrib/src/test/results/clientnegative/case_with_row_sequence.q.out 4447c65
ql/build.xml 449b47a
ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1c6f092
ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java e687b1a
ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java 3d5e95d
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java cec0d46
ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java 8a50dc6
ql/src/test/queries/clientnegative/minimr_broken_pipe.q PRE-CREATION
ql/src/test/results/clientnegative/dyn_part3.q.out 5f4df65
ql/src/test/results/clientnegative/index_compact_entry_limit.q.out 06437c7
ql/src/test/results/clientnegative/index_compact_size_limit.q.out 54f1262
ql/src/test/results/clientnegative/minimr_broken_pipe.q.out PRE-CREATION
ql/src/test/results/clientnegative/script_broken_pipe1.q.out d33d2cc
ql/src/test/results/clientnegative/script_broken_pipe2.q.out afbaa44
ql/src/test/results/clientnegative/script_broken_pipe3.q.out fe8f757
ql/src/test/results/clientnegative/script_error.q.out c72d780
ql/src/test/results/clientnegative/udf_reflect_neg.q.out f2082a3
ql/src/test/results/clientnegative/udf_test_error.q.out 5fd9a00
ql/src/test/results/clientnegative/udf_test_error_reduce.q.out ddc5e5b
ql/src/test/results/clientpositive/auto_join25.q.out 362b2fa
ql/src/test/results/clientpositive/mapjoin_hook.q.out acdeefa
ql/src/test/templates/TestNegativeCliDriver.vm ec13f79
Diff: https://reviews.apache.org/r/777/diff
Testing
-------
Tested TestNegativeCliDriver in both local and miniMR mode
Thanks,
Syed
Re: Review Request: HIVE-2156: Improve Execution Error Messages
Posted by Syed Albiz <s....@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/777/
-----------------------------------------------------------
(Updated 2011-08-09 19:33:53.896288)
Review request for hive and John Sichi.
Changes
-------
update patch to avoid TestExecDriver errors and include miniMR mode testcase
Summary
-------
- Add local error messages to point to job logs and provide TaskIDs
- Add a timeout to the fetching of task logs and errors
This addresses bug HIVE-2156.
https://issues.apache.org/jira/browse/HIVE-2156
Diffs (updated)
-----
build-common.xml 4856c5f
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java b46976f
conf/hive-default.xml 3a4f833
contrib/src/test/results/clientnegative/case_with_row_sequence.q.out 4447c65
ql/build.xml 449b47a
ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1c6f092
ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java e687b1a
ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java 3d5e95d
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java cec0d46
ql/src/test/queries/clientnegative/minimr_broken_pipe.q PRE-CREATION
ql/src/test/results/clientnegative/dyn_part3.q.out 5f4df65
ql/src/test/results/clientnegative/index_compact_entry_limit.q.out 06437c7
ql/src/test/results/clientnegative/index_compact_size_limit.q.out 54f1262
ql/src/test/results/clientnegative/minimr_broken_pipe.q.out PRE-CREATION
ql/src/test/results/clientnegative/script_broken_pipe1.q.out d33d2cc
ql/src/test/results/clientnegative/script_broken_pipe2.q.out afbaa44
ql/src/test/results/clientnegative/script_broken_pipe3.q.out fe8f757
ql/src/test/results/clientnegative/script_error.q.out c72d780
ql/src/test/results/clientnegative/udf_reflect_neg.q.out f2082a3
ql/src/test/results/clientnegative/udf_test_error.q.out 5fd9a00
ql/src/test/results/clientnegative/udf_test_error_reduce.q.out ddc5e5b
ql/src/test/results/clientpositive/auto_join25.q.out 362b2fa
ql/src/test/results/clientpositive/mapjoin_hook.q.out acdeefa
ql/src/test/templates/TestNegativeCliDriver.vm ec13f79
Diff: https://reviews.apache.org/r/777/diff
Testing
-------
Tested TestNegativeCliDriver in both local and miniMR mode
Thanks,
Syed
Re: Review Request: HIVE-2156: Improve Execution Error Messages
Posted by Syed Albiz <s....@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/777/
-----------------------------------------------------------
(Updated 2011-08-03 18:35:22.814828)
Review request for hive and John Sichi.
Changes
-------
add fixes for two failing tests
Summary
-------
- Add local error messages to point to job logs and provide TaskIDs
- Add a timeout to the fetching of task logs and errors
This addresses bug HIVE-2156.
https://issues.apache.org/jira/browse/HIVE-2156
Diffs (updated)
-----
build-common.xml 4856c5f
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java b46976f
conf/hive-default.xml 3a4f833
ql/build.xml 449b47a
ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1c6f092
ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java e687b1a
ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java 3d5e95d
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java cec0d46
ql/src/test/results/clientnegative/dyn_part3.q.out 5f4df65
ql/src/test/results/clientnegative/index_compact_entry_limit.q.out fcb2673
ql/src/test/results/clientnegative/index_compact_size_limit.q.out fcb2673
ql/src/test/results/clientnegative/script_broken_pipe1.q.out d33d2cc
ql/src/test/results/clientnegative/script_broken_pipe2.q.out afbaa44
ql/src/test/results/clientnegative/script_broken_pipe3.q.out fe8f757
ql/src/test/results/clientnegative/script_error.q.out c72d780
ql/src/test/results/clientnegative/udf_reflect_neg.q.out f2082a3
ql/src/test/results/clientnegative/udf_test_error.q.out 5fd9a00
ql/src/test/results/clientnegative/udf_test_error_reduce.q.out ddc5e5b
ql/src/test/results/clientpositive/auto_join25.q.out 362b2fa
ql/src/test/results/clientpositive/mapjoin_hook.q.out acdeefa
ql/src/test/templates/TestNegativeCliDriver.vm ec13f79
Diff: https://reviews.apache.org/r/777/diff
Testing
-------
Tested TestNegativeCliDriver in both local and miniMR mode
Thanks,
Syed