[jira] [Commented] (HIVE-11675) make use of file footer PPD API in ETL strategy or separate strategy

["Hive QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-11675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15130017#comment-15130017 ] 

Hive QA commented on HIVE-11675:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12785648/HIVE-11675.04.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10033 tests executed
*Failed tests:*
{noformat}
TestSparkCliDriver-timestamp_lazy.q-bucketsortoptimize_insert_4.q-date_udf.q-and-12-more - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
org.apache.hive.service.cli.TestEmbeddedThriftBinaryCLIService.testExecuteStatementAsync
{noformat}

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6848/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6848/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6848/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12785648 - PreCommit-HIVE-TRUNK-Build

> make use of file footer PPD API in ETL strategy or separate strategy
> --------------------------------------------------------------------
>
>                 Key: HIVE-11675
>                 URL: https://issues.apache.org/jira/browse/HIVE-11675
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-11675.01.patch, HIVE-11675.02.patch, HIVE-11675.03.patch, HIVE-11675.04.patch, HIVE-11675.patch
>
>
> Need to take a look at the best flow. It won't be much different if we do filtering metastore call for each partition. So perhaps we'd need the custom sync point/batching after all.
> Or we can make it opportunistic and not fetch any footers unless it can be pushed down to metastore or fetched from local cache, that way the only slow threaded op is directory listings



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)