You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2020/07/13 11:39:10 UTC
[GitHub] [hive] pvargacl opened a new pull request #1244: HIVE-23837: Configure StorageHandlers if FileSinkOperator is child of MergeJoinWork
pvargacl opened a new pull request #1244:
URL: https://github.com/apache/hive/pull/1244
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a change in pull request #1244: HIVE-23837: Configure StorageHandlers if FileSinkOperator is child of MergeJoinWork
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on a change in pull request #1244:
URL: https://github.com/apache/hive/pull/1244#discussion_r455711296
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/plan/MergeJoinWork.java
##########
@@ -66,6 +67,13 @@ public void replaceRoots(Map<Operator<?>, Operator<?>> replacementMap) {
@Override
public void configureJobConf(JobConf job) {
Review comment:
I think, it can be extracted to base. In MapWork and ReduceWork , for example, you can do same, at least for FileSinkOperator - iterate over getAllRootOperators() and call configureJobConf
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a change in pull request #1244: HIVE-23837: Configure StorageHandlers if FileSinkOperator is child of MergeJoinWork
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on a change in pull request #1244:
URL: https://github.com/apache/hive/pull/1244#discussion_r455711625
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/plan/MergeJoinWork.java
##########
@@ -66,6 +67,13 @@ public void replaceRoots(Map<Operator<?>, Operator<?>> replacementMap) {
@Override
public void configureJobConf(JobConf job) {
+ for(Operator rootOp : getAllRootOperators()) {
Review comment:
I think, it can be extracted to base. In MapWork and ReduceWork , for example, you can do same, at least for FileSinkOperator - iterate over getAllRootOperators() and call configureJobConf
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a change in pull request #1244: HIVE-23837: Configure StorageHandlers if FileSinkOperator is child of MergeJoinWork
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on a change in pull request #1244:
URL: https://github.com/apache/hive/pull/1244#discussion_r455705666
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/plan/MergeJoinWork.java
##########
@@ -66,6 +67,13 @@ public void replaceRoots(Map<Operator<?>, Operator<?>> replacementMap) {
@Override
public void configureJobConf(JobConf job) {
+ for(Operator rootOp : getAllRootOperators()) {
+ for (FileSinkOperator fs : OperatorUtils.findOperators(rootOp, FileSinkOperator.class)) {
Review comment:
same
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a change in pull request #1244: HIVE-23837: Configure StorageHandlers if FileSinkOperator is child of MergeJoinWork
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on a change in pull request #1244:
URL: https://github.com/apache/hive/pull/1244#discussion_r455705413
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/plan/MergeJoinWork.java
##########
@@ -66,6 +67,13 @@ public void replaceRoots(Map<Operator<?>, Operator<?>> replacementMap) {
@Override
public void configureJobConf(JobConf job) {
+ for(Operator rootOp : getAllRootOperators()) {
Review comment:
could we use forEach here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvargacl commented on a change in pull request #1244: HIVE-23837: Configure StorageHandlers if FileSinkOperator is child of MergeJoinWork
Posted by GitBox <gi...@apache.org>.
pvargacl commented on a change in pull request #1244:
URL: https://github.com/apache/hive/pull/1244#discussion_r455730854
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/plan/MergeJoinWork.java
##########
@@ -66,6 +67,13 @@ public void replaceRoots(Map<Operator<?>, Operator<?>> replacementMap) {
@Override
public void configureJobConf(JobConf job) {
+ for(Operator rootOp : getAllRootOperators()) {
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ merged pull request #1244: HIVE-23837: Configure StorageHandlers if FileSinkOperator is child of MergeJoinWork
Posted by GitBox <gi...@apache.org>.
deniskuzZ merged pull request #1244:
URL: https://github.com/apache/hive/pull/1244
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] deniskuzZ commented on a change in pull request #1244: HIVE-23837: Configure StorageHandlers if FileSinkOperator is child of MergeJoinWork
Posted by GitBox <gi...@apache.org>.
deniskuzZ commented on a change in pull request #1244:
URL: https://github.com/apache/hive/pull/1244#discussion_r455711296
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/plan/MergeJoinWork.java
##########
@@ -66,6 +67,13 @@ public void replaceRoots(Map<Operator<?>, Operator<?>> replacementMap) {
@Override
public void configureJobConf(JobConf job) {
Review comment:
I think it can be extracted to base. In MapWork and ReduceWork , for example, you can do same, at least for FileSinkOperator - iterate over getAllRootOperators() and call configureJobConf
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org