You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Hive QA (Jira)" <ji...@apache.org> on 2019/10/07 23:46:00 UTC
[jira] [Commented] (HIVE-22239) Scale data size using column value
ranges
[ https://issues.apache.org/jira/browse/HIVE-22239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946339#comment-16946339 ]
Hive QA commented on HIVE-22239:
--------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 28s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m 31s{color} | {color:blue} standalone-metastore/metastore-common in master has 37 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 34s{color} | {color:blue} common in master has 65 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m 8s{color} | {color:blue} standalone-metastore/metastore-server in master has 170 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 14s{color} | {color:blue} ql in master has 1551 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 19s{color} | {color:red} standalone-metastore/metastore-server: The patch generated 26 new + 109 unchanged - 11 fixed = 135 total (was 120) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 41s{color} | {color:red} ql: The patch generated 15 new + 259 unchanged - 2 fixed = 274 total (was 261) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 38s{color} | {color:green} metastore-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 39s{color} | {color:green} common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 19s{color} | {color:red} standalone-metastore/metastore-server generated 2 new + 170 unchanged - 0 fixed = 172 total (was 170) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 8s{color} | {color:green} ql generated 0 new + 1550 unchanged - 1 fixed = 1550 total (was 1551) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 16s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 36s{color} | {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| | Integral division result cast to double or float in org.apache.hadoop.hive.metastore.columnstats.aggr.TimestampColumnStatsAggregator.aggregate(List, List, boolean) At TimestampColumnStatsAggregator.java:double or float in org.apache.hadoop.hive.metastore.columnstats.aggr.TimestampColumnStatsAggregator.aggregate(List, List, boolean) At TimestampColumnStatsAggregator.java:[line 102] |
| | org.apache.hadoop.hive.metastore.columnstats.cache.TimestampColumnStatsDataInspector doesn't override org.apache.hadoop.hive.metastore.api.TimestampColumnStatsData.equals(Object) At TimestampColumnStatsDataInspector.java:At TimestampColumnStatsDataInspector.java:[line 1] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-18897/dev-support/hive-personality.sh |
| git revision | master / e16877c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-18897/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt |
| checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-18897/yetus/diff-checkstyle-ql.txt |
| findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-18897/yetus/new-findbugs-standalone-metastore_metastore-server.html |
| asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-18897/yetus/patch-asflicense-problems.txt |
| modules | C: standalone-metastore/metastore-common common standalone-metastore/metastore-server ql U: . |
| Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-18897/yetus.txt |
| Powered by | Apache Yetus http://yetus.apache.org |
This message was automatically generated.
> Scale data size using column value ranges
> -----------------------------------------
>
> Key: HIVE-22239
> URL: https://issues.apache.org/jira/browse/HIVE-22239
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
> Reporter: Jesus Camacho Rodriguez
> Assignee: Jesus Camacho Rodriguez
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22239.01.patch, HIVE-22239.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, min/max values for columns are only used to determine whether a certain range filter falls out of range and thus filters all rows or none at all. If it does not, we just use a heuristic that the condition will filter 1/3 of the input rows. Instead of using that heuristic, we can use another one that assumes that data will be uniformly distributed across that range, and calculate the selectivity for the condition accordingly.
> This patch also includes the propagation of min/max column values from statistics to the optimizer for timestamp type.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)