You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Terry Siu <Te...@smartfocus.com> on 2014/11/03 20:04:53 UTC

ParquetFilters and StringType support for GT, GTE, LT, LTE

Is there any reason why StringType is not a supported type the GT, GTE, LT, LTE operations? I was able to previously have a predicate where my column type was a string and execute a filter with one of the above operators in SparkSQL w/o any problems. However, I synced up to the latest code this morning and now the same query will give me a MatchError for this column of string type.

Thanks,
-Terry

Re: ParquetFilters and StringType support for GT, GTE, LT, LTE

Posted by Terry Siu <Te...@smartfocus.com>.
Done.

https://issues.apache.org/jira/browse/SPARK-4213

Thanks,
-Terry

From: Michael Armbrust <mi...@databricks.com>>
Date: Monday, November 3, 2014 at 1:37 PM
To: Terry Siu <te...@smartfocus.com>>
Cc: "user@spark.apache.org<ma...@spark.apache.org>" <us...@spark.apache.org>>
Subject: Re: ParquetFilters and StringType support for GT, GTE, LT, LTE

That sounds like a regression.  Could you open a JIRA with steps to reproduce (https://issues.apache.org/jira/browse/SPARK)?  We'll want to fix this before the 1.2 release.

On Mon, Nov 3, 2014 at 11:04 AM, Terry Siu <Te...@smartfocus.com>> wrote:
Is there any reason why StringType is not a supported type the GT, GTE, LT, LTE operations? I was able to previously have a predicate where my column type was a string and execute a filter with one of the above operators in SparkSQL w/o any problems. However, I synced up to the latest code this morning and now the same query will give me a MatchError for this column of string type.

Thanks,
-Terry

Re: ParquetFilters and StringType support for GT, GTE, LT, LTE

Posted by Michael Armbrust <mi...@databricks.com>.
That sounds like a regression.  Could you open a JIRA with steps to
reproduce (https://issues.apache.org/jira/browse/SPARK)?  We'll want to fix
this before the 1.2 release.

On Mon, Nov 3, 2014 at 11:04 AM, Terry Siu <Te...@smartfocus.com> wrote:

>  Is there any reason why StringType is not a supported type the GT, GTE,
> LT, LTE operations? I was able to previously have a predicate where my
> column type was a string and execute a filter with one of the above
> operators in SparkSQL w/o any problems. However, I synced up to the latest
> code this morning and now the same query will give me a MatchError for this
> column of string type.
>
>  Thanks,
> -Terry
>