[jira] [Created] (HBASE-8475) TableMapReduceUtils#addDependencyJars(Job) shouldn't ship duplicate entries in tmpjars

["Nick Dimiduk (JIRA)" <ji...@apache.org>]

Nick Dimiduk created HBASE-8475:
-----------------------------------

             Summary: TableMapReduceUtils#addDependencyJars(Job) shouldn't ship duplicate entries in tmpjars
                 Key: HBASE-8475
                 URL: https://issues.apache.org/jira/browse/HBASE-8475
             Project: HBase
          Issue Type: Bug
          Components: mapreduce
            Reporter: Nick Dimiduk


*note* this is a hypothesis that needs further investigation.

Users running the above method in their jobs as a convenience for  packaging up dependencies will see duplicate entries for jars providing common functionality, ie hbase.jar, hadoop.jar, user application jar. For instance, if they specify a custom output format and custom partitioner, both in the same jar, that jar is resolved via classloader and added to tmpjars twice. This imposes a fair amount of pre-job IO (particularly if you jars are fat with packed dependencies) and delay job launch accordingly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira