[jira] [Commented] (TEZ-4527) Add generic and pluggable hooks for DAGs and task attempts

["okumin (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-4527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17798191#comment-17798191 ] 

okumin commented on TEZ-4527:
-----------------------------

[~ayushtkn] [~abstractdog] Hi, I drafted this PR to add pluggable hooks like Hive's various hooks. Could you please take a look and put any comments?

https://github.com/apache/tez/pull/324

> Add generic and pluggable hooks for DAGs and task attempts
> ----------------------------------------------------------
>
>                 Key: TEZ-4527
>                 URL: https://issues.apache.org/jira/browse/TEZ-4527
>             Project: Apache Tez
>          Issue Type: New Feature
>            Reporter: okumin
>            Assignee: okumin
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> TEZ-4344 added a capability to periodically dump thread information to Tez. I think the basic architecture, injecting an additional process into DAGAppMaster or TezChild, is potentially extensible.
>  
> For example, we maintain a very similar patch to collect stack traces.
>  * [https://speakerdeck.com/okumin/hive-distributed-profiling-system-in-treasure-data-english-version-number-tdtechtalk?slide=26]
>  * [https://api-docs.treasuredata.com/blog/hive-distributed-profiling/]
>  
> I would propose to make the injection step pluggable and maintain TezThreadDumpHelper as one of the plugins.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)