You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2020/07/02 17:16:55 UTC
[GitHub] [hive] jcamachor commented on a change in pull request #1177: HIVE-23665 Rewrite last_value to first_value to enable streaming results
jcamachor commented on a change in pull request #1177:
URL: https://github.com/apache/hive/pull/1177#discussion_r449158491
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
##########
@@ -2439,6 +2440,9 @@ private RelNode applyPostJoinOrderingTransform(RelNode basePlan, RelMetadataProv
HiveWindowingFixRule.INSTANCE);
}
+ generatePartialProgram(program, false, HepMatchOrder.DEPTH_FIRST,
Review comment:
Can you incorporate the rule into the block above?
```
if (profilesCBO.contains(ExtendedCBOProfile.WINDOWING_POSTPROCESSING)) {
generatePartialProgram(program, false, HepMatchOrder.DEPTH_FIRST,
HiveWindowingLastValueRewrite.INSTANCE, HiveWindowingFixRule.INSTANCE);
}
```
##########
File path: ql/src/test/results/clientpositive/llap/vector_ptf_part_simple.q.out
##########
@@ -314,46 +386,46 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@vector_ptf_part_simple_orc
#### A masked pattern was here ####
p_mfgr p_name p_retailprice rn r dr fv lv c cs
-Manufacturer#2 almond aquamarine rose maroon antique 900.66 1 1 1 900.66 2031.98 8 8
-Manufacturer#2 almond aquamarine rose maroon antique 1698.66 2 1 1 900.66 2031.98 8 8
-Manufacturer#2 almond antique violet turquoise frosted 1800.7 3 1 1 900.66 2031.98 8 8
-Manufacturer#2 almond antique violet chocolate turquoise 1690.68 4 1 1 900.66 2031.98 8 8
-Manufacturer#2 almond antique violet turquoise frosted 1800.7 5 1 1 900.66 2031.98 8 8
-Manufacturer#2 almond antique violet turquoise frosted 1800.7 6 1 1 900.66 2031.98 8 8
-Manufacturer#2 almond aquamarine sandy cyan gainsboro 1000.6 7 1 1 900.66 2031.98 8 8
-Manufacturer#2 almond aquamarine midnight light salmon 2031.98 8 1 1 900.66 2031.98 8 8
Manufacturer#3 almond antique forest lavender goldenrod 1190.27 1 1 1 1190.27 1190.27 7 8
-Manufacturer#3 almond antique chartreuse khaki white 99.68 2 1 1 1190.27 1190.27 7 8
-Manufacturer#3 almond antique forest lavender goldenrod NULL 3 1 1 1190.27 1190.27 7 8
-Manufacturer#3 almond antique metallic orange dim 55.39 4 1 1 1190.27 1190.27 7 8
-Manufacturer#3 almond antique misty red olive 1922.98 5 1 1 1190.27 1190.27 7 8
-Manufacturer#3 almond antique forest lavender goldenrod 590.27 6 1 1 1190.27 1190.27 7 8
-Manufacturer#3 almond antique olive coral navajo 1337.29 7 1 1 1190.27 1190.27 7 8
Manufacturer#3 almond antique forest lavender goldenrod 1190.27 8 1 1 1190.27 1190.27 7 8
-Manufacturer#4 almond antique gainsboro frosted violet NULL 1 1 1 NULL 1290.35 4 6
-Manufacturer#4 almond aquamarine floral ivory bisque NULL 2 1 1 NULL 1290.35 4 6
-Manufacturer#4 almond antique violet mint lemon 1375.42 3 1 1 NULL 1290.35 4 6
-Manufacturer#4 almond aquamarine yellow dodger mint 1844.92 4 1 1 NULL 1290.35 4 6
-Manufacturer#4 almond aquamarine floral ivory bisque 1206.26 5 1 1 NULL 1290.35 4 6
-Manufacturer#4 almond azure aquamarine papaya violet 1290.35 6 1 1 NULL 1290.35 4 6
+Manufacturer#3 almond antique olive coral navajo 1337.29 7 1 1 1190.27 1190.27 7 8
+Manufacturer#3 almond antique forest lavender goldenrod 590.27 6 1 1 1190.27 1190.27 7 8
+Manufacturer#3 almond antique misty red olive 1922.98 5 1 1 1190.27 1190.27 7 8
+Manufacturer#3 almond antique metallic orange dim 55.39 4 1 1 1190.27 1190.27 7 8
+Manufacturer#3 almond antique forest lavender goldenrod NULL 3 1 1 1190.27 1190.27 7 8
+Manufacturer#3 almond antique chartreuse khaki white 99.68 2 1 1 1190.27 1190.27 7 8
+Manufacturer#1 almond aquamarine pink moccasin thistle 1632.66 1 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 2 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond aquamarine pink moccasin thistle 1632.66 3 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 4 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond aquamarine pink moccasin thistle 1632.66 5 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 6 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond antique salmon chartreuse burlywood 1602.59 7 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond antique burnished rose metallic 1173.15 8 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond aquamarine burnished black steel 1414.42 9 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond aquamarine pink moccasin thistle NULL 10 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 11 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond antique burnished rose metallic 1173.15 12 1 1 1632.66 1632.66 11 12
+Manufacturer#5 almond antique blue firebrick mint 1789.69 6 1 1 1611.66 1789.69 6 6
Manufacturer#5 almond antique medium spring khaki 1611.66 1 1 1 1611.66 1789.69 6 6
Manufacturer#5 almond aquamarine dodger light gainsboro 1018.1 2 1 1 1611.66 1789.69 6 6
Manufacturer#5 almond azure blanched chiffon midnight 1464.48 3 1 1 1611.66 1789.69 6 6
Manufacturer#5 almond antique medium spring khaki 1611.66 4 1 1 1611.66 1789.69 6 6
Manufacturer#5 almond antique sky peru orange 1788.73 5 1 1 1611.66 1789.69 6 6
-Manufacturer#5 almond antique blue firebrick mint 1789.69 6 1 1 1611.66 1789.69 6 6
-Manufacturer#1 almond aquamarine pink moccasin thistle 1632.66 1 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 2 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond aquamarine pink moccasin thistle 1632.66 3 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 4 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond aquamarine pink moccasin thistle 1632.66 5 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 6 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond antique salmon chartreuse burlywood 1602.59 7 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond antique burnished rose metallic 1173.15 8 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond aquamarine burnished black steel 1414.42 9 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond aquamarine pink moccasin thistle NULL 10 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 11 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond antique burnished rose metallic 1173.15 12 1 1 1632.66 1173.15 11 12
+Manufacturer#4 almond aquamarine floral ivory bisque 1206.26 5 1 1 NULL 1206.26 4 6
+Manufacturer#4 almond azure aquamarine papaya violet 1290.35 6 1 1 NULL 1206.26 4 6
+Manufacturer#4 almond antique gainsboro frosted violet NULL 1 1 1 NULL 1206.26 4 6
+Manufacturer#4 almond aquamarine floral ivory bisque NULL 2 1 1 NULL 1206.26 4 6
+Manufacturer#4 almond antique violet mint lemon 1375.42 3 1 1 NULL 1206.26 4 6
+Manufacturer#4 almond aquamarine yellow dodger mint 1844.92 4 1 1 NULL 1206.26 4 6
+Manufacturer#2 almond antique violet turquoise frosted 1800.7 3 1 1 900.66 1800.7 8 8
Review comment:
The result for last value column has changed wrt to old value, `2031.98` -> `1800.7`. Why is that the case? Is it correct?
It seems the change also happened for other rows below.
##########
File path: ql/src/test/results/clientpositive/llap/vector_ptf_part_simple.q.out
##########
@@ -6256,46 +6328,46 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@vector_ptf_part_simple_orc
#### A masked pattern was here ####
p_mfgr p_name p_retailprice rn r dr fv lv c cs
-Manufacturer#2 almond aquamarine rose maroon antique 900.66 1 1 1 900.66 2031.98 8 8
-Manufacturer#2 almond aquamarine rose maroon antique 1698.66 2 1 1 900.66 2031.98 8 8
-Manufacturer#2 almond antique violet turquoise frosted 1800.7 3 1 1 900.66 2031.98 8 8
-Manufacturer#2 almond antique violet chocolate turquoise 1690.68 4 1 1 900.66 2031.98 8 8
-Manufacturer#2 almond antique violet turquoise frosted 1800.7 5 1 1 900.66 2031.98 8 8
-Manufacturer#2 almond antique violet turquoise frosted 1800.7 6 1 1 900.66 2031.98 8 8
-Manufacturer#2 almond aquamarine sandy cyan gainsboro 1000.6 7 1 1 900.66 2031.98 8 8
-Manufacturer#2 almond aquamarine midnight light salmon 2031.98 8 1 1 900.66 2031.98 8 8
Manufacturer#3 almond antique forest lavender goldenrod 1190.27 1 1 1 1190.27 1190.27 7 8
-Manufacturer#3 almond antique chartreuse khaki white 99.68 2 1 1 1190.27 1190.27 7 8
-Manufacturer#3 almond antique forest lavender goldenrod NULL 3 1 1 1190.27 1190.27 7 8
-Manufacturer#3 almond antique metallic orange dim 55.39 4 1 1 1190.27 1190.27 7 8
-Manufacturer#3 almond antique misty red olive 1922.98 5 1 1 1190.27 1190.27 7 8
-Manufacturer#3 almond antique forest lavender goldenrod 590.27 6 1 1 1190.27 1190.27 7 8
-Manufacturer#3 almond antique olive coral navajo 1337.29 7 1 1 1190.27 1190.27 7 8
Manufacturer#3 almond antique forest lavender goldenrod 1190.27 8 1 1 1190.27 1190.27 7 8
-Manufacturer#4 almond antique gainsboro frosted violet NULL 1 1 1 NULL 1290.35 4 6
-Manufacturer#4 almond aquamarine floral ivory bisque NULL 2 1 1 NULL 1290.35 4 6
-Manufacturer#4 almond antique violet mint lemon 1375.42 3 1 1 NULL 1290.35 4 6
-Manufacturer#4 almond aquamarine yellow dodger mint 1844.92 4 1 1 NULL 1290.35 4 6
-Manufacturer#4 almond aquamarine floral ivory bisque 1206.26 5 1 1 NULL 1290.35 4 6
-Manufacturer#4 almond azure aquamarine papaya violet 1290.35 6 1 1 NULL 1290.35 4 6
+Manufacturer#3 almond antique olive coral navajo 1337.29 7 1 1 1190.27 1190.27 7 8
+Manufacturer#3 almond antique forest lavender goldenrod 590.27 6 1 1 1190.27 1190.27 7 8
+Manufacturer#3 almond antique misty red olive 1922.98 5 1 1 1190.27 1190.27 7 8
+Manufacturer#3 almond antique metallic orange dim 55.39 4 1 1 1190.27 1190.27 7 8
+Manufacturer#3 almond antique forest lavender goldenrod NULL 3 1 1 1190.27 1190.27 7 8
+Manufacturer#3 almond antique chartreuse khaki white 99.68 2 1 1 1190.27 1190.27 7 8
+Manufacturer#1 almond aquamarine pink moccasin thistle 1632.66 1 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 2 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond aquamarine pink moccasin thistle 1632.66 3 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 4 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond aquamarine pink moccasin thistle 1632.66 5 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 6 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond antique salmon chartreuse burlywood 1602.59 7 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond antique burnished rose metallic 1173.15 8 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond aquamarine burnished black steel 1414.42 9 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond aquamarine pink moccasin thistle NULL 10 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 11 1 1 1632.66 1632.66 11 12
+Manufacturer#1 almond antique burnished rose metallic 1173.15 12 1 1 1632.66 1632.66 11 12
+Manufacturer#5 almond antique blue firebrick mint 1789.69 6 1 1 1611.66 1789.69 6 6
Manufacturer#5 almond antique medium spring khaki 1611.66 1 1 1 1611.66 1789.69 6 6
Manufacturer#5 almond aquamarine dodger light gainsboro 1018.1 2 1 1 1611.66 1789.69 6 6
Manufacturer#5 almond azure blanched chiffon midnight 1464.48 3 1 1 1611.66 1789.69 6 6
Manufacturer#5 almond antique medium spring khaki 1611.66 4 1 1 1611.66 1789.69 6 6
Manufacturer#5 almond antique sky peru orange 1788.73 5 1 1 1611.66 1789.69 6 6
-Manufacturer#5 almond antique blue firebrick mint 1789.69 6 1 1 1611.66 1789.69 6 6
-Manufacturer#1 almond aquamarine pink moccasin thistle 1632.66 1 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 2 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond aquamarine pink moccasin thistle 1632.66 3 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 4 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond aquamarine pink moccasin thistle 1632.66 5 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 6 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond antique salmon chartreuse burlywood 1602.59 7 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond antique burnished rose metallic 1173.15 8 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond aquamarine burnished black steel 1414.42 9 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond aquamarine pink moccasin thistle NULL 10 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond antique chartreuse lavender yellow 1753.76 11 1 1 1632.66 1173.15 11 12
-Manufacturer#1 almond antique burnished rose metallic 1173.15 12 1 1 1632.66 1173.15 11 12
+Manufacturer#4 almond aquamarine floral ivory bisque 1206.26 5 1 1 NULL 1206.26 4 6
+Manufacturer#4 almond azure aquamarine papaya violet 1290.35 6 1 1 NULL 1206.26 4 6
+Manufacturer#4 almond antique gainsboro frosted violet NULL 1 1 1 NULL 1206.26 4 6
+Manufacturer#4 almond aquamarine floral ivory bisque NULL 2 1 1 NULL 1206.26 4 6
+Manufacturer#4 almond antique violet mint lemon 1375.42 3 1 1 NULL 1206.26 4 6
+Manufacturer#4 almond aquamarine yellow dodger mint 1844.92 4 1 1 NULL 1206.26 4 6
+Manufacturer#2 almond antique violet turquoise frosted 1800.7 3 1 1 900.66 1800.7 8 8
Review comment:
Same as with previous query, this result is different.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org