You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2015/11/09 03:25:10 UTC
[jira] [Comment Edited] (SPARK-9557) Refactor ParquetFilterSuite
and remove old ParquetFilters code
[ https://issues.apache.org/jira/browse/SPARK-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14995922#comment-14995922 ]
Hyukjin Kwon edited comment on SPARK-9557 at 11/9/15 2:24 AM:
--------------------------------------------------------------
But I would like to just add a just little bit of corrections.
I updated the test for this by not reducing just collecting filters as an array; however, this actually creates filters after reducing in the {{DataSourceStrategy}}, which the original test codes followed.
was (Author: hyukjin.kwon):
But I would like to just add a just little corrections.
I update the test for this by not reducing just collecting filters as an array; however, this actually create filters after reducing in the {{DataSourceStrategy}}, which the original test codes followed this.
> Refactor ParquetFilterSuite and remove old ParquetFilters code
> --------------------------------------------------------------
>
> Key: SPARK-9557
> URL: https://issues.apache.org/jira/browse/SPARK-9557
> Project: Spark
> Issue Type: Test
> Components: SQL
> Affects Versions: 1.5.0
> Reporter: Cheng Lian
> Assignee: Cheng Lian
>
> Currently there are two Parquet filter conversion code path, one for converting data sources {{Filter}}, the other for converting Catalyst predicate {{Expression}}, which is used by the removed old Parquet code. We should remove the latter, but {{ParquetFilterSuite}} uses it to test Parquet filter push-down.
> Need to refactor {{ParquetFilterSuite}} to make it test the data source version and then remove the old Parquet filter conversion code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org