You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Rajat Khandelwal <ra...@gmail.com> on 2016/06/10 12:01:12 UTC
Review Request 48544: HIVE-13903: getFunctionInfo is downloading jar
on every call
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48544/
-----------------------------------------------------------
Review request for hive.
Bugs: HIVE-13903
https://issues.apache.org/jira/browse/HIVE-13903
Repository: hive-git
Description
-------
on queries using permanent udfs, the jar file of the udf is downloaded multiple times. Each call originating from Registry.getFunctionInfo. This increases time for the query, especially if that query is just an explain query. The jar should be downloaded once, and not downloaded again if the udf class is accessible in the current thread.
Diffs
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java bb0ca3acf79426df5d34d44da8595ac78edf8c76
itests/hive-unit/src/test/java/org/apache/hadoop/hive/llap/ext/TestLlapInputSplit.java 1de8aa6686a5b5db697fb4dee4b4155b9dc14ef2
llap-client/src/java/org/apache/hadoop/hive/llap/LlapInputSplit.java 02aedfd8929038e763ed413a3545f68117e08638
llap-client/src/java/org/apache/hadoop/hive/llap/SubmitWorkInfo.java 670429412923566536dc933463a64daabc5adb45
llap-client/src/java/org/apache/hadoop/hive/llap/coordinator/LlapCoordinator.java PRE-CREATION
llap-client/src/java/org/apache/hadoop/hive/llap/ext/LlapTaskUmbilicalExternalClient.java 0edb1cdd04df578228cad97cf85216186c04007d
llap-client/src/java/org/apache/hadoop/hive/llap/io/api/LlapProxy.java 6c2618bcd3ca9f74302c3260b1535b892f543d23
llap-client/src/java/org/apache/hadoop/hive/llap/security/LlapTokenLocalClient.java af889b6a5af6499f01834886c5746243ad5847c1
llap-client/src/java/org/apache/hadoop/hive/llap/security/LlapTokenLocalClientImpl.java PRE-CREATION
llap-common/src/gen/protobuf/gen-java/org/apache/hadoop/hive/llap/daemon/rpc/LlapDaemonProtocolProtos.java 16531e3445a9bfbd55a788674fa9f4132ca6b697
llap-common/src/java/org/apache/hadoop/hive/llap/NotTezEventHelper.java PRE-CREATION
llap-common/src/java/org/apache/hadoop/hive/llap/security/LlapSigner.java 478a40a4edded3f911df636eda655be2c0516ba3
llap-common/src/java/org/apache/hadoop/hive/llap/security/LlapSignerImpl.java PRE-CREATION
llap-common/src/java/org/apache/hadoop/hive/llap/security/LlapTokenLocalClient.java PRE-CREATION
llap-common/src/java/org/apache/hadoop/hive/llap/security/SecretManager.java 540f978fd6244050688c6cc767d20716186410d3
llap-common/src/java/org/apache/hadoop/hive/llap/security/SigningSecretManager.java 067a98eaab81a69ead61c45595da0f3b89936b4e
llap-common/src/java/org/apache/hadoop/hive/llap/tez/Converters.java e43b72b5e5f7e78168fc6b2b9c48cabad12c035f
llap-common/src/protobuf/LlapDaemonProtocol.proto 486ba0a19483663b2dec622e6c52c45b12aa8512
llap-common/src/test/org/apache/hadoop/hive/llap/tez/TestConverters.java 1df6df03ec7bc8655e085c3247718994334573b5
llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java 4306c222442546c9c54162f95e3550ea978be1b7
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/ContainerRunnerImpl.java 6f21d3cb430cb2a9e8a1b4d94b80f286602b4882
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/LlapTaskReporter.java dc4482eaeb875d6255a389d81e1b79d8a10a4b7a
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/TaskRunnerCallable.java f97585d16609854a50f18b584fe3e1454de88c84
llap-server/src/java/org/apache/hadoop/hive/llap/security/LlapSignerImpl.java 417459353e3fbe5d64ac95eaed4a37db58c17d1d
llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/TaskExecutorTestHelpers.java 1df5253f4654f7340f0180d36112a997fa9e51c0
llap-server/src/test/org/apache/hadoop/hive/llap/security/TestLlapSignerImpl.java 04202254965b6ff78fa19eb122710a9fa01f7a6f
llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskCommunicator.java 026df3bbfef3fe6f13d1bf9b0dc18b07e3cc817e
orc/src/java/org/apache/orc/OrcConf.java 6fcbb72aeea40dea9d9143a334b42fb81c7f4404
ql/pom.xml db03978389900ff00e725cdabdf19226403f8804
ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java 86df74d26ddd60736214c0f9acb835f4253a2c17
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveSplitGenerator.java 54fc3af94c6b45f2e7ed8edb0b8f45d4fa0dbf2b
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java d04cfa3b102e1674bb2be2d8bebceb717dabc46a
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordUpdater.java 4bf2403704f4f7dd1c83ebc28f34eb01e18a0066
ql/src/java/org/apache/hadoop/hive/ql/parse/PTFTranslator.java 018d8d0d4bf35178e23453e5c6f0917e7edefc55
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java 140dbdab5162ac865a9d6427aaf4a972cebbd827
ql/src/java/org/apache/tez/dag/api/TaskSpecBuilder.java 6d3180200ce57eec8ce69c6fa8e98d1a9b6d7a52
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcRecordUpdater.java 0a61fb884a8e5825e95b442d4c97efa247da4170
ql/src/test/queries/clientpositive/windowing_range_multiorder.q d8ca4d64620d01834c24ae729332dd8d053ea1b6
ql/src/test/results/clientpositive/tez/acid_globallimit.q.out 804e5e25a65915fb867e0ca23181aafbc791ffdc
ql/src/test/results/clientpositive/windowing_range_multiorder.q.out 9910883680f45108bf3db59041d9b5ec6e749cdb
service/src/java/org/apache/hive/service/server/HiveServer2.java d61edf50df576af582aa9e9fb7112f8e72224043
shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java 68fac17311474108389fdf2dcea2baa52504ffd9
Diff: https://reviews.apache.org/r/48544/diff/
Testing
-------
Thanks,
Rajat Khandelwal
Re: Review Request 48544: HIVE-13903: getFunctionInfo is downloading
jar on every call
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48544/
-----------------------------------------------------------
(Updated June 10, 2016, 6:36 p.m.)
Review request for hive.
Bugs: HIVE-13903
https://issues.apache.org/jira/browse/HIVE-13903
Repository: hive-git
Description
-------
on queries using permanent udfs, the jar file of the udf is downloaded multiple times. Each call originating from Registry.getFunctionInfo. This increases time for the query, especially if that query is just an explain query. The jar should be downloaded once, and not downloaded again if the udf class is accessible in the current thread.
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java 86df74d26ddd60736214c0f9acb835f4253a2c17
Diff: https://reviews.apache.org/r/48544/diff/
Testing
-------
Thanks,
Rajat Khandelwal