[jira] [Commented] (HIVE-11297) Combine op trees for partition info generating tasks [Spark branch]

["Hive QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16032558#comment-16032558 ] 

Hive QA commented on HIVE-11297:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12870495/HIVE-11297.1.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10813 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries] (batchId=228)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[insert_overwrite_local_directory_1] (batchId=237)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=232)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=232)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5495/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5495/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5495/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12870495 - PreCommit-HIVE-Build

> Combine op trees for partition info generating tasks [Spark branch]
> -------------------------------------------------------------------
>
>                 Key: HIVE-11297
>                 URL: https://issues.apache.org/jira/browse/HIVE-11297
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: spark-branch
>            Reporter: Chao Sun
>            Assignee: liyunzhang_intel
>         Attachments: HIVE-11297.1.patch
>
>
> Currently, for dynamic partition pruning in Spark, if a small table generates partition info for more than one partition columns, multiple operator trees are created, which all start from the same table scan op, but have different spark partition pruning sinks.
> As an optimization, we can combine these op trees and so don't have to do table scan multiple times.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)