You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Barna Zsombor Klara <zs...@cloudera.com> on 2017/10/03 12:29:15 UTC
Review Request 62748: HIVE-17078: Add more logs to MapredLocalTask
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62748/
-----------------------------------------------------------
Review request for hive, Peter Vary and Adam Szita.
Bugs: HIVE-17078
https://issues.apache.org/jira/browse/HIVE-17078
Repository: hive-git
Description
-------
HIVE-17078: Add more logs to MapredLocalTask
Diffs
-----
common/src/java/org/apache/hadoop/hive/common/log/LogRedirector.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 9dfefeece56d7c3127c6921b5ad880fcaa926d56
ql/src/test/results/clientpositive/auto_join25.q.out 534bdb6ff0461614deab6e6fb84409cd34422f96
ql/src/test/results/clientpositive/auto_join_without_localtask.q.out 57f00674de30087143ef579cd0cff2bc024f529f
ql/src/test/results/clientpositive/bucketsortoptimize_insert_8.q.out f0e77f00afe0c753fbaa402c27662b2559c3e4d7
ql/src/test/results/clientpositive/infer_bucket_sort_convert_join.q.out 52ebe5aa8db035434c1bb0978b7b3ac2a25e062a
ql/src/test/results/clientpositive/mapjoin_hook.q.out a9f9be3a4dca764f70303412cc07dd31f7cece1c
spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java 03e773a803f1dc7edfb9be8d795aba32535bd433
Diff: https://reviews.apache.org/r/62748/diff/1/
Testing
-------
Thanks,
Barna Zsombor Klara
Re: Review Request 62748: HIVE-17078: Add more logs to MapredLocalTask
Posted by Peter Vary <pv...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62748/#review187212
-----------------------------------------------------------
Ship it!
Ship It!
- Peter Vary
On Oct. 3, 2017, 12:29 p.m., Barna Zsombor Klara wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62748/
> -----------------------------------------------------------
>
> (Updated Oct. 3, 2017, 12:29 p.m.)
>
>
> Review request for hive, Peter Vary, Adam Szita, and Yibing Shi.
>
>
> Bugs: HIVE-17078
> https://issues.apache.org/jira/browse/HIVE-17078
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-17078: Add more logs to MapredLocalTask
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/common/log/LogRedirector.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 9dfefeece56d7c3127c6921b5ad880fcaa926d56
> ql/src/test/results/clientpositive/auto_join25.q.out 534bdb6ff0461614deab6e6fb84409cd34422f96
> ql/src/test/results/clientpositive/auto_join_without_localtask.q.out 57f00674de30087143ef579cd0cff2bc024f529f
> ql/src/test/results/clientpositive/bucketsortoptimize_insert_8.q.out f0e77f00afe0c753fbaa402c27662b2559c3e4d7
> ql/src/test/results/clientpositive/infer_bucket_sort_convert_join.q.out 52ebe5aa8db035434c1bb0978b7b3ac2a25e062a
> ql/src/test/results/clientpositive/mapjoin_hook.q.out a9f9be3a4dca764f70303412cc07dd31f7cece1c
> spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java 03e773a803f1dc7edfb9be8d795aba32535bd433
>
>
> Diff: https://reviews.apache.org/r/62748/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Barna Zsombor Klara
>
>
Re: Review Request 62748: HIVE-17078: Add more logs to MapredLocalTask
Posted by Barna Zsombor Klara <zs...@cloudera.com>.
> On Oct. 5, 2017, 11:28 a.m., Peter Vary wrote:
> > common/src/java/org/apache/hadoop/hive/common/log/LogRedirector.java
> > Lines 67 (patched)
> > <https://reviews.apache.org/r/62748/diff/1/?file=1843648#file1843648line67>
> >
> > Is it worthwile to make MAX_LINES configurable?
Don't think so. This only gathers error lines to be concatenated into an error message.
- Barna Zsombor
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62748/#review187167
-----------------------------------------------------------
On Oct. 3, 2017, 12:29 p.m., Barna Zsombor Klara wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62748/
> -----------------------------------------------------------
>
> (Updated Oct. 3, 2017, 12:29 p.m.)
>
>
> Review request for hive, Peter Vary, Adam Szita, and Yibing Shi.
>
>
> Bugs: HIVE-17078
> https://issues.apache.org/jira/browse/HIVE-17078
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-17078: Add more logs to MapredLocalTask
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/common/log/LogRedirector.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 9dfefeece56d7c3127c6921b5ad880fcaa926d56
> ql/src/test/results/clientpositive/auto_join25.q.out 534bdb6ff0461614deab6e6fb84409cd34422f96
> ql/src/test/results/clientpositive/auto_join_without_localtask.q.out 57f00674de30087143ef579cd0cff2bc024f529f
> ql/src/test/results/clientpositive/bucketsortoptimize_insert_8.q.out f0e77f00afe0c753fbaa402c27662b2559c3e4d7
> ql/src/test/results/clientpositive/infer_bucket_sort_convert_join.q.out 52ebe5aa8db035434c1bb0978b7b3ac2a25e062a
> ql/src/test/results/clientpositive/mapjoin_hook.q.out a9f9be3a4dca764f70303412cc07dd31f7cece1c
> spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java 03e773a803f1dc7edfb9be8d795aba32535bd433
>
>
> Diff: https://reviews.apache.org/r/62748/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Barna Zsombor Klara
>
>
Re: Review Request 62748: HIVE-17078: Add more logs to MapredLocalTask
Posted by Peter Vary <pv...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62748/#review187167
-----------------------------------------------------------
Thanks Zsombor!
I like your patch. Minor nits, and questions.
Peter
common/src/java/org/apache/hadoop/hive/common/log/LogRedirector.java
Lines 36 (patched)
<https://reviews.apache.org/r/62748/#comment264081>
nit: The ParentHealthCheck name made me think, that it is the healthcheck for the caller thread, but later found out this is the healthcheck of the log generator thread. Am I right here?
common/src/java/org/apache/hadoop/hive/common/log/LogRedirector.java
Lines 67 (patched)
<https://reviews.apache.org/r/62748/#comment264077>
Is it worthwile to make MAX_LINES configurable?
ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java
Lines 332-335 (patched)
<https://reviews.apache.org/r/62748/#comment264080>
nit: Do we have a nice Java8 way to do this?
- Peter Vary
On Oct. 3, 2017, 12:29 p.m., Barna Zsombor Klara wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62748/
> -----------------------------------------------------------
>
> (Updated Oct. 3, 2017, 12:29 p.m.)
>
>
> Review request for hive, Peter Vary, Adam Szita, and Yibing Shi.
>
>
> Bugs: HIVE-17078
> https://issues.apache.org/jira/browse/HIVE-17078
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-17078: Add more logs to MapredLocalTask
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/common/log/LogRedirector.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 9dfefeece56d7c3127c6921b5ad880fcaa926d56
> ql/src/test/results/clientpositive/auto_join25.q.out 534bdb6ff0461614deab6e6fb84409cd34422f96
> ql/src/test/results/clientpositive/auto_join_without_localtask.q.out 57f00674de30087143ef579cd0cff2bc024f529f
> ql/src/test/results/clientpositive/bucketsortoptimize_insert_8.q.out f0e77f00afe0c753fbaa402c27662b2559c3e4d7
> ql/src/test/results/clientpositive/infer_bucket_sort_convert_join.q.out 52ebe5aa8db035434c1bb0978b7b3ac2a25e062a
> ql/src/test/results/clientpositive/mapjoin_hook.q.out a9f9be3a4dca764f70303412cc07dd31f7cece1c
> spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java 03e773a803f1dc7edfb9be8d795aba32535bd433
>
>
> Diff: https://reviews.apache.org/r/62748/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Barna Zsombor Klara
>
>