You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "László Bodor (Jira)" <ji...@apache.org> on 2020/11/13 09:50:00 UTC
[jira] [Updated] (TEZ-4249) UnorderedOutSpiller should inherit
parent thread's name
[ https://issues.apache.org/jira/browse/TEZ-4249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
László Bodor updated TEZ-4249:
------------------------------
Description:
https://github.com/apache/tez/blob/master/tez-runtime-library/src/main/java/org/apache/tez/runtime/library/common/writers/UnorderedPartitionedKVWriter.java#L278
{code}
.setNameFormat(
"UnorderedOutSpiller {" + TezUtilsInternal.cleanVertexName(
outputContext.getDestinationVertexName()) + "} #%d")
.build()
{code}
with this name format, we can lose track of the parent thread in logs, which could be annoying in LLAP:
{code}
2020-11-12T23:18:09,965 INFO [UnorderedOutSpiller {Reducer_10} #0 ()] org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter: Writing spill 0 to /grid/1/yarn/nm/usercache/lbodor/appcache/application_1604906786259_0157/usercache/lbodor/appcache/application_1604906786259_0161/33/output/attempt_1604906786259_0161_33_03_000067_6_27009_0/file.out
{code}
it should contain some info about parent thread, something like:
{code}
2020-11-12T23:18:11,725 INFO [TezTR-786259_161_33_3_67_6 UnorderedOutSpiller {Reducer_10} #0 ()] org.apache.tez.runtime.LogicalIOProcessorRuntimeTask: Closed processor for vertex=Map 7, index=3, interruptedStatus=false
{code}
> UnorderedOutSpiller should inherit parent thread's name
> -------------------------------------------------------
>
> Key: TEZ-4249
> URL: https://issues.apache.org/jira/browse/TEZ-4249
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: László Bodor
> Priority: Major
>
> https://github.com/apache/tez/blob/master/tez-runtime-library/src/main/java/org/apache/tez/runtime/library/common/writers/UnorderedPartitionedKVWriter.java#L278
> {code}
> .setNameFormat(
> "UnorderedOutSpiller {" + TezUtilsInternal.cleanVertexName(
> outputContext.getDestinationVertexName()) + "} #%d")
> .build()
> {code}
> with this name format, we can lose track of the parent thread in logs, which could be annoying in LLAP:
> {code}
> 2020-11-12T23:18:09,965 INFO [UnorderedOutSpiller {Reducer_10} #0 ()] org.apache.tez.runtime.library.common.writers.UnorderedPartitionedKVWriter: Writing spill 0 to /grid/1/yarn/nm/usercache/lbodor/appcache/application_1604906786259_0157/usercache/lbodor/appcache/application_1604906786259_0161/33/output/attempt_1604906786259_0161_33_03_000067_6_27009_0/file.out
> {code}
> it should contain some info about parent thread, something like:
> {code}
> 2020-11-12T23:18:11,725 INFO [TezTR-786259_161_33_3_67_6 UnorderedOutSpiller {Reducer_10} #0 ()] org.apache.tez.runtime.LogicalIOProcessorRuntimeTask: Closed processor for vertex=Map 7, index=3, interruptedStatus=false
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)