You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by Ryan Blue <bl...@apache.org> on 2016/06/13 23:55:08 UTC
Re: Apache Pig and Parquet Predicate Pushdown
+Parquet Dev List
Predicate push-down in the Pig loader is in the current master, but not in
a released 1.8 version. It should be out in 1.9. Otherwise, your code looks
correct.
rb
On Mon, Jun 13, 2016 at 4:32 PM, Rana Faisal Munir <fm...@essi.upc.edu>
wrote:
> Dear Sir,
>
>
>
> I am working on Apache Pig and Parquet. I am testing predicate pushdown
> but somehow it is not working fine. I am using Parquet 1.8 and Pig 0.16
> versions. I am using this simple code to apply predicate pushdown which is
> working fine in ORC. Could you please help me to identify the problem?
>
>
>
>
>
> lineitempart = load '/intermediate/Lineitem-Part/' USING
> org.apache.parquet.pig.ParquetLoader();
>
>
>
> SET parquet.pig.predicate.pushdown.enable true
>
>
>
> lineitem = FILTER lineitempart BY l_orderkey <= 27;
>
>
>
>
>
> STORE lineitem INTO '/output/query/s20/' USING PigStorage('|');
>
>
>
>
>
> Regards
>
> Faisal
>
--
Ryan Blue
RE: Apache Pig and Parquet Predicate Pushdown
Posted by Rana Faisal Munir <fm...@essi.upc.edu>.
Thank you very much. It means I have to download current master if I want to work with predicate pushdown of Parquet in Pig.
From: Ryan Blue [mailto:blue@apache.org]
Sent: Tuesday, June 14, 2016 1:55 AM
To: Rana Faisal Munir; Parquet Dev
Cc: Ryan Blue
Subject: Re: Apache Pig and Parquet Predicate Pushdown
+Parquet Dev List
Predicate push-down in the Pig loader is in the current master, but not in a released 1.8 version. It should be out in 1.9. Otherwise, your code looks correct.
rb
On Mon, Jun 13, 2016 at 4:32 PM, Rana Faisal Munir <fm...@essi.upc.edu> wrote:
Dear Sir,
I am working on Apache Pig and Parquet. I am testing predicate pushdown but somehow it is not working fine. I am using Parquet 1.8 and Pig 0.16 versions. I am using this simple code to apply predicate pushdown which is working fine in ORC. Could you please help me to identify the problem?
lineitempart = load '/intermediate/Lineitem-Part/' USING org.apache.parquet.pig.ParquetLoader();
SET parquet.pig.predicate.pushdown.enable true
lineitem = FILTER lineitempart BY l_orderkey <= 27;
STORE lineitem INTO '/output/query/s20/' USING PigStorage('|');
Regards
Faisal
--
Ryan Blue