[jira] [Commented] (HIVE-12763) Use bit vector to track per partition NDV

["Hive QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082165#comment-15082165 ] 

Hive QA commented on HIVE-12763:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12780213/HIVE-12763.01.patch

{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 137 failed/errored test(s), 9980 tests executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniTezCliDriver-schema_evol_orc_nonvec_mapwork_part.q-vector_decimal_2.q-schema_evol_text_fetchwork_table.q-and-12-more - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_udf1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnstats_partlvl
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnstats_partlvl_dp
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnstats_quoting
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnstats_tbllvl
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_compustat_avro
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_compute_stats_date
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_compute_stats_decimal
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_compute_stats_double
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_compute_stats_empty_table
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_compute_stats_long
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_compute_stats_string
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_convert_enum_to_string
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_display_colstats_tbllvl
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_order2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_temp_table_display_colstats_tbllvl
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_varchar_udf1
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.org.apache.hadoop.hive.cli.TestMiniTezCliDriver
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_alter_merge_orc
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_explainuser_1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_explainuser_2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mapjoin_mapjoin
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_scriptfile1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_fsstat
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_multi_union
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union_dynamic_partition
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_transform_ppr1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_grouping_sets
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_interval_mapjoin
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_join_filters
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_mapjoin_reduce
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_reduce1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_bucketmapjoin1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_string_funcs
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_columnstats_partlvl_multiple_part_clause
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testAlterPartition
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testColumnStatistics
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testComplexTable
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testComplexTypeApi
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testConcurrentMetastores
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testDBOwner
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testDropTable
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testFunctionWithResources
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testGetTableObjects
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testListPartitionNames
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testListPartitions
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testNameMethods
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testPartitionFilter
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testRenamePartition
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testSimpleFunction
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testStatsFastTrivial
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testTableDatabase
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testAlterPartition
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testColumnStatistics
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testComplexTable
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testComplexTypeApi
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testConcurrentMetastores
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testDBOwner
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testDropTable
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testFunctionWithResources
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testGetTableObjects
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testListPartitionNames
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testListPartitions
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testNameMethods
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testPartitionFilter
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testRenamePartition
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testSimpleFunction
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testStatsFastTrivial
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testTableDatabase
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testAlterPartition
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testColumnStatistics
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testComplexTable
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testComplexTypeApi
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testConcurrentMetastores
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testDBOwner
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testDropTable
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testFunctionWithResources
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testGetTableObjects
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testListPartitionNames
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testListPartitions
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testNameMethods
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testPartitionFilter
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testRenamePartition
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testSimpleFunction
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testStatsFastTrivial
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testTableDatabase
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testAlterPartition
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testColumnStatistics
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testComplexTable
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testComplexTypeApi
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testConcurrentMetastores
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testDBOwner
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testDropTable
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testFunctionWithResources
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testGetTableObjects
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testListPartitionNames
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testListPartitions
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testNameMethods
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testPartitionFilter
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testRenamePartition
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testSimpleFunction
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testStatsFastTrivial
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testTableDatabase
org.apache.hadoop.hive.metastore.hbase.TestHBaseAggrStatsCacheIntegration.hit
org.apache.hadoop.hive.metastore.hbase.TestHBaseAggrStatsCacheIntegration.someWithStats
org.apache.hadoop.hive.metastore.hbase.TestHBaseAggregateStatsCache.someNonexistentPartitions
org.apache.hadoop.hive.metastore.hbase.TestHBaseSchemaTool.oneMondoTest
org.apache.hadoop.hive.metastore.hbase.TestHBaseStore.decimalPartitionStatistics
org.apache.hadoop.hive.metastore.hbase.TestHBaseStore.decimalTableStatistics
org.apache.hadoop.hive.metastore.hbase.TestHBaseStore.doublePartitionStatistics
org.apache.hadoop.hive.metastore.hbase.TestHBaseStore.doubleTableStatistics
org.apache.hadoop.hive.metastore.hbase.TestHBaseStore.longPartitionStatistics
org.apache.hadoop.hive.metastore.hbase.TestHBaseStore.longTableStatistics
org.apache.hadoop.hive.metastore.hbase.TestHBaseStore.stringPartitionStatistics
org.apache.hadoop.hive.metastore.hbase.TestHBaseStore.stringTableStatistics
org.apache.hadoop.hive.metastore.hbase.TestHBaseStoreIntegration.partitionStatistics
org.apache.hadoop.hive.metastore.hbase.TestHBaseStoreIntegration.tableStatistics
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testMultiSessionMultipleUse
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testSingleSessionMultipleUse
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveOperationType.checkHiveOperationTypeMatch
org.apache.hadoop.hive.thrift.TestHadoopAuthBridge23.testDelegationTokenSharedStore
org.apache.hive.jdbc.TestSSL.testSSLVersion
org.apache.hive.spark.client.TestSparkClient.testAddJarsAndFiles
org.apache.hive.spark.client.TestSparkClient.testCounters
org.apache.hive.spark.client.TestSparkClient.testErrorJob
org.apache.hive.spark.client.TestSparkClient.testJobSubmission
org.apache.hive.spark.client.TestSparkClient.testMetricsCollection
org.apache.hive.spark.client.TestSparkClient.testRemoteClient
org.apache.hive.spark.client.TestSparkClient.testSimpleSparkJob
org.apache.hive.spark.client.TestSparkClient.testSyncRpc
{noformat}

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6510/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6510/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6510/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 137 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12780213 - PreCommit-HIVE-TRUNK-Build

> Use bit vector to track per partition NDV
> -----------------------------------------
>
>                 Key: HIVE-12763
>                 URL: https://issues.apache.org/jira/browse/HIVE-12763
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Pengcheng Xiong
>            Assignee: Pengcheng Xiong
>         Attachments: HIVE-12763.01.patch
>
>
> This will improve merging of per partitions stats.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)