You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Attila Magyar <am...@hortonworks.com> on 2019/07/24 15:19:11 UTC
Review Request 71156: Tez: Use a pre-parsed TezConfiguration from
DagUtils
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71156/
-----------------------------------------------------------
Review request for hive, Laszlo Bodor, Gopal V, and Jesús Camacho Rodríguez.
Bugs: HIVE-21828
https://issues.apache.org/jira/browse/HIVE-21828
Repository: hive-git
Description
-------
The HS2 tez-site.xml does not change dynamically - the XML parsed components of the config can be obtained statically and kept across sessions.
This allows for the replacing of "new TezConfiguration()" with a HS2 local version instead.
The configuration object however has to reference the right resource file (i.e location of tez-site.xml) without reparsing it for each query.
Diffs
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 440d761f03d
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java 3278dfea061
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezConfigurationFactory.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java dd7ccd4764d
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRegExp.java 3bf3cfd3d9e
ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestTezTask.java befeb4f2dd4
ql/src/test/org/apache/hive/testutils/HiveTestEnvSetup.java f872da02a3c
ql/src/test/queries/clientpositive/mm_loaddata.q 7e5787f2a65
Diff: https://reviews.apache.org/r/71156/diff/1/
Testing
-------
unittests
Thanks,
Attila Magyar
Re: Review Request 71156: Tez: Use a pre-parsed TezConfiguration from
DagUtils
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71156/#review216863
-----------------------------------------------------------
Ship it!
Ship It!
- Jesús Camacho Rodríguez
On July 25, 2019, 8:05 a.m., Attila Magyar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71156/
> -----------------------------------------------------------
>
> (Updated July 25, 2019, 8:05 a.m.)
>
>
> Review request for hive, Laszlo Bodor, Gopal V, and Jesús Camacho Rodríguez.
>
>
> Bugs: HIVE-21828
> https://issues.apache.org/jira/browse/HIVE-21828
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> The HS2 tez-site.xml does not change dynamically - the XML parsed components of the config can be obtained statically and kept across sessions.
>
> This allows for the replacing of "new TezConfiguration()" with a HS2 local version instead.
>
> The configuration object however has to reference the right resource file (i.e location of tez-site.xml) without reparsing it for each query.
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 2b7468a1ab7
> ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java 3278dfea061
> ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezConfigurationFactory.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java dd7ccd4764d
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRegExp.java 3bf3cfd3d9e
> ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestTezTask.java befeb4f2dd4
> ql/src/test/org/apache/hive/testutils/HiveTestEnvSetup.java f872da02a3c
> ql/src/test/queries/clientpositive/mm_loaddata.q 7e5787f2a65
> ql/src/test/results/clientpositive/llap/tez_fixed_bucket_pruning.q.out eaed60c1ba7
>
>
> Diff: https://reviews.apache.org/r/71156/diff/2/
>
>
> Testing
> -------
>
> unittests
>
>
> Thanks,
>
> Attila Magyar
>
>
Re: Review Request 71156: Tez: Use a pre-parsed TezConfiguration from
DagUtils
Posted by Attila Magyar <am...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71156/
-----------------------------------------------------------
(Updated July 25, 2019, 8:05 a.m.)
Review request for hive, Laszlo Bodor, Gopal V, and Jesús Camacho Rodríguez.
Bugs: HIVE-21828
https://issues.apache.org/jira/browse/HIVE-21828
Repository: hive-git
Description
-------
The HS2 tez-site.xml does not change dynamically - the XML parsed components of the config can be obtained statically and kept across sessions.
This allows for the replacing of "new TezConfiguration()" with a HS2 local version instead.
The configuration object however has to reference the right resource file (i.e location of tez-site.xml) without reparsing it for each query.
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 2b7468a1ab7
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java 3278dfea061
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezConfigurationFactory.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java dd7ccd4764d
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRegExp.java 3bf3cfd3d9e
ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestTezTask.java befeb4f2dd4
ql/src/test/org/apache/hive/testutils/HiveTestEnvSetup.java f872da02a3c
ql/src/test/queries/clientpositive/mm_loaddata.q 7e5787f2a65
ql/src/test/results/clientpositive/llap/tez_fixed_bucket_pruning.q.out eaed60c1ba7
Diff: https://reviews.apache.org/r/71156/diff/2/
Changes: https://reviews.apache.org/r/71156/diff/1-2/
Testing
-------
unittests
Thanks,
Attila Magyar