You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Kevin Wilfong <ke...@fb.com> on 2011/08/04 21:45:20 UTC

Review Request: HIVE-2347: Added Hadoop Job ID to Task class.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1296/
-----------------------------------------------------------

Review request for hive and Ning Zhang.


Summary
-------

I added a field for the Hadoop Job ID to the Task class.  This will make it accessible to the Driver and hence to the hooks for logging/debugging purposes.  By including it in the Task, we only need to check that the type of the task is MAPRED, before getting the job ID.

I considered adding it to several places:

as separate fields in ExecDriver and BlockMergeTask: this would require duplicating code, require conditions to determine the type of a task and casting to either ExecDriver or BlockMergeTask in order to get the JobID from them

in the MapRedWork: this would require modifying a field in MapRedWork in the execute function, and I could not find a precedent for this


This addresses bug HIVE-2347.
    https://issues.apache.org/jira/browse/HIVE-2347


Diffs
-----

  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 1153966 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java 1153966 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java 1153966 

Diff: https://reviews.apache.org/r/1296/diff


Testing
-------

Ran the TestCliDriver and TestNegativeCliDriver test suites and verified they passed.

Also, created a sample post exec hook which simply logged the JobID for every map reduce task, and verified it.


Thanks,

Kevin