You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2021/09/23 11:25:11 UTC

[GitHub] [hive] klcopp opened a new pull request #2667: [wip] HIVE-25549: Need to transient init partition expressions in vectorized PTFs

klcopp opened a new pull request #2667:
URL: https://github.com/apache/hive/pull/2667


   ### How was this patch tested?
   [Tests coming soon]


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] abstractdog commented on a change in pull request #2667: HIVE-25549: Need to transient init partition expressions in vectorized PTFs

Posted by GitBox <gi...@apache.org>.
abstractdog commented on a change in pull request #2667:
URL: https://github.com/apache/hive/pull/2667#discussion_r716657850



##########
File path: ql/src/test/queries/clientpositive/vector_windowing_row_number.q
##########
@@ -6,13 +6,14 @@ set hive.fetch.task.conversion=none;
 

Review comment:
       please include at least one "EXPLAIN VECTORIZATION DETAIL" of a query that utilizes a CastLongToString
   
   explain vectorization should print partition expressions I think, in the form of:
   ```
   partitionExpressions: [ConstantVectorExpression(val 0) -> 4:int]
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] klcopp merged pull request #2667: HIVE-25549: Wrong results for window function with expression in PARTITION BY or ORDER BY clause

Posted by GitBox <gi...@apache.org>.
klcopp merged pull request #2667:
URL: https://github.com/apache/hive/pull/2667


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] abstractdog commented on a change in pull request #2667: HIVE-25549: Need to transient init partition expressions in vectorized PTFs

Posted by GitBox <gi...@apache.org>.
abstractdog commented on a change in pull request #2667:
URL: https://github.com/apache/hive/pull/2667#discussion_r716650055



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ptf/VectorPTFOperator.java
##########
@@ -318,6 +318,7 @@ protected void initializeOp(Configuration hconf) throws HiveException {
     currentPartitionIntervalDayTimes = new HiveIntervalDayTime[partitionKeyCount];
 
     VectorExpression.doTransientInit(partitionExpressions, hconf);
+    VectorExpression.doTransientInit(orderExpressions, hconf);

Review comment:
       could you please move doTransientInit calls to the beginning of the method, after super.initializeOp?
   I don't think it will change behavior but I can see every vectorized operator that calls this method, calls at the beginning of initializeOp, I would try to stay aligned with the pattern :)
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] abstractdog commented on a change in pull request #2667: HIVE-25549: Need to transient init partition expressions in vectorized PTFs

Posted by GitBox <gi...@apache.org>.
abstractdog commented on a change in pull request #2667:
URL: https://github.com/apache/hive/pull/2667#discussion_r716665653



##########
File path: ql/src/test/queries/clientpositive/vector_windowing_row_number.q
##########
@@ -6,13 +6,14 @@ set hive.fetch.task.conversion=none;
 

Review comment:
       I think you should rather use both, keep original and add a new one with VECTORIZATION DETAIL




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] abstractdog commented on a change in pull request #2667: HIVE-25549: Need to transient init partition expressions in vectorized PTFs

Posted by GitBox <gi...@apache.org>.
abstractdog commented on a change in pull request #2667:
URL: https://github.com/apache/hive/pull/2667#discussion_r716657850



##########
File path: ql/src/test/queries/clientpositive/vector_windowing_row_number.q
##########
@@ -6,13 +6,14 @@ set hive.fetch.task.conversion=none;
 

Review comment:
       please include at least "EXPLAIN VECTORIZATION DETAIL" of a query that utilizes a CastLongToString vectorization should print partition expressions as well I think, in the form of:
   ```
   partitionExpressions: [ConstantVectorExpression(val 0) -> 4:int]
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] abstractdog commented on a change in pull request #2667: HIVE-25549: Need to transient init partition expressions in vectorized PTFs

Posted by GitBox <gi...@apache.org>.
abstractdog commented on a change in pull request #2667:
URL: https://github.com/apache/hive/pull/2667#discussion_r716657850



##########
File path: ql/src/test/queries/clientpositive/vector_windowing_row_number.q
##########
@@ -6,13 +6,14 @@ set hive.fetch.task.conversion=none;
 

Review comment:
       please include at least one "EXPLAIN VECTORIZATION DETAIL" of a query that utilizes a CastLongToString vectorization should print partition expressions as well I think, in the form of:
   ```
   partitionExpressions: [ConstantVectorExpression(val 0) -> 4:int]
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] klcopp commented on a change in pull request #2667: HIVE-25549: Need to transient init partition expressions in vectorized PTFs

Posted by GitBox <gi...@apache.org>.
klcopp commented on a change in pull request #2667:
URL: https://github.com/apache/hive/pull/2667#discussion_r716653457



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ptf/VectorPTFOperator.java
##########
@@ -318,6 +318,7 @@ protected void initializeOp(Configuration hconf) throws HiveException {
     currentPartitionIntervalDayTimes = new HiveIntervalDayTime[partitionKeyCount];
 
     VectorExpression.doTransientInit(partitionExpressions, hconf);
+    VectorExpression.doTransientInit(orderExpressions, hconf);

Review comment:
       Sure




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] klcopp commented on a change in pull request #2667: HIVE-25549: Need to transient init partition expressions in vectorized PTFs

Posted by GitBox <gi...@apache.org>.
klcopp commented on a change in pull request #2667:
URL: https://github.com/apache/hive/pull/2667#discussion_r716661583



##########
File path: ql/src/test/queries/clientpositive/vector_windowing_row_number.q
##########
@@ -6,13 +6,14 @@ set hive.fetch.task.conversion=none;
 

Review comment:
       Do you mean, for each "explain select..." in this file, change to: "EXPLAIN VECTORIZATION DETAIL select..."?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org