You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Kevin Wilfong <ke...@fb.com> on 2011/08/04 21:45:20 UTC
Review Request: HIVE-2347: Added Hadoop Job ID to Task class.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1296/
-----------------------------------------------------------
Review request for hive and Ning Zhang.
Summary
-------
I added a field for the Hadoop Job ID to the Task class. This will make it accessible to the Driver and hence to the hooks for logging/debugging purposes. By including it in the Task, we only need to check that the type of the task is MAPRED, before getting the job ID.
I considered adding it to several places:
as separate fields in ExecDriver and BlockMergeTask: this would require duplicating code, require conditions to determine the type of a task and casting to either ExecDriver or BlockMergeTask in order to get the JobID from them
in the MapRedWork: this would require modifying a field in MapRedWork in the execute function, and I could not find a precedent for this
This addresses bug HIVE-2347.
https://issues.apache.org/jira/browse/HIVE-2347
Diffs
-----
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 1153966
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java 1153966
trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java 1153966
Diff: https://reviews.apache.org/r/1296/diff
Testing
-------
Ran the TestCliDriver and TestNegativeCliDriver test suites and verified they passed.
Also, created a sample post exec hook which simply logged the JobID for every map reduce task, and verified it.
Thanks,
Kevin