Re: Review Request 68027: HIVE-19986: Add logging of runtime statistics indicating when Hdfs Erasure Coding is used by MR. These stats are not avalable until the unreleased Hadoop 3.2 so the shim is used to determine if the new counter can be used.

[Andrew Sherman via Review Board <no...@reviews.apache.org>]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68027/
-----------------------------------------------------------

(Updated July 24, 2018, 9:15 p.m.)


Review request for hive and Sahil Takiar.


Bugs: HIVE-19986
    https://issues.apache.org/jira/browse/HIVE-19986


Repository: hive-git


Description
-------

HIVE-19986: Add logging of runtime statistics indicating when Hdfs Erasure Coding is used by MR. These stats are not avalable until the unreleased Hadoop 3.2 so the shim is used to determine if the new counter can be used.


Diffs (updated)
-----

  itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2ErasureCoding.java b0a0145a4ee705b0a7d8f214d2c87397f731faec 
  ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java 483c3d9fcd2f55a644321a62224b13846e421188 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java c31e22f041e18abf077f44e79188b7479abc3629 
  ql/src/java/org/apache/hadoop/hive/ql/processors/ErasureProcessor.java 46114f50dc1ee3f03305cf9047361c39131128d6 
  shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java 02490f1171ee658a2f3208e4cf2f0416d6907dc6 
  shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 2e84ca9b180e0149fd2185183238dab2268e5474 


Diff: https://reviews.apache.org/r/68027/diff/2/

Changes: https://reviews.apache.org/r/68027/diff/1-2/


Testing
-------

New test added to TestJdbcWithMiniHS2ErasureCoding.java.
I also ran this code against a hadoop 3.2 snapshot to ensure the new stat works


Thanks,

Andrew Sherman