You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Prasanth Jayachandran (JIRA)" <ji...@apache.org> on 2017/06/28 20:12:00 UTC
[jira] [Commented] (HIVE-10567) partial scan for rcfile table
doesn't work for dynamic partition
[ https://issues.apache.org/jira/browse/HIVE-10567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16067143#comment-16067143 ]
Prasanth Jayachandran commented on HIVE-10567:
----------------------------------------------
[~ashutoshc] This is related to the feature where analyze does not need partition spec to be explicitly defined. right? If partition spec is missing, analyze will happen for all partitions. isn't it?
> partial scan for rcfile table doesn't work for dynamic partition
> ----------------------------------------------------------------
>
> Key: HIVE-10567
> URL: https://issues.apache.org/jira/browse/HIVE-10567
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.14.0, 1.0.0
> Reporter: Thomas Friedrich
> Assignee: Thomas Friedrich
> Priority: Minor
> Labels: rcfile
> Attachments: HIVE-10567.1.patch
>
>
> HIVE-3958 added support for partial scan for RCFile. This works fine for static partitions (for example: analyze table analyze_srcpart_partial_scan PARTITION(ds='2008-04-08',hr=11) compute statistics partialscan).
> For dynamic partition, the analyze files with an IOException "java.io.IOException: No input paths specified in job":
> hive> ANALYZE TABLE testtable PARTITION(col_varchar) COMPUTE STATISTICS PARTIALSCAN;
> java.io.IOException: No input paths specified in job
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.getInputPaths(HiveInputFormat.java:318)
> at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:459)
> at org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:624)
> at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:616)
> at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:492)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)