[jira] [Resolved] (PIG-3760) Predicate pushdown for columnar file formats

["Daniel Dai (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai resolved PIG-3760.
-----------------------------
    Resolution: Fixed
      Assignee: Rohini Palaniswamy

The major work has been done in 0.14.0. Close the ticket and use individual ticket to track remaining work.

> Predicate pushdown for columnar file formats
> --------------------------------------------
>
>                 Key: PIG-3760
>                 URL: https://issues.apache.org/jira/browse/PIG-3760
>             Project: Pig
>          Issue Type: New Feature
>            Reporter: Andrew Musselman
>            Assignee: Rohini Palaniswamy
>             Fix For: 0.15.0
>
>
> From the conversation on dev@pig:
> "Partition pruning for ORC is not addressed in PIG-3558. We will need
> to do partition pruning for both ORC and Parquet in a new ticket.
> Curently there is no interface to deal with this kind of pushdown
> (LoadMetadata.setPartitionFilter push the filter to loader, but remove
> the filter statement, for ORC/Parquet, filter is a hint, and we need
> to do the filter again in Pig even it is pushed to loader), we will
> need to define a new interface for that. You are welcome to initiate
> the work. I know Aniket is also interested in doing that, so be sure
> the talk with him about this work.
> Thanks,
> Daniel
> On Mon, Feb 10, 2014 at 11:42 AM, Andrew Musselman
> <an...@gmail.com> wrote:
> > I had a chat with a couple people last week about a feature request for
> > Pig:  in a "where" or "filter" clause, when loading an ORC file, to skip
> > directly to the right offset instead of scanning the whole file."



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)