[jira] [Assigned] (SPARK-43357) Spark AWS Glue date partition push down broken

["Wenchen Fan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-43357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wenchen Fan reassigned SPARK-43357:
-----------------------------------

    Assignee: Stijn De Haes

> Spark AWS Glue date partition push down broken
> ----------------------------------------------
>
>                 Key: SPARK-43357
>                 URL: https://issues.apache.org/jira/browse/SPARK-43357
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.1.0, 3.1.1, 3.1.2, 3.2.0, 3.1.3, 3.2.1, 3.3.0, 3.2.2, 3.3.1, 3.2.3, 3.2.4, 3.3.2
>            Reporter: Stijn De Haes
>            Assignee: Stijn De Haes
>            Priority: Major
>
> When using the following project: [https://github.com/awslabs/aws-glue-data-catalog-client-for-apache-hive-metastore]
> To have glue supported as as a hive metastore for spark there is an issue when reading a date-partitioned data set. Writing is fine.
> You get the following error: 
> {quote}org.apache.hadoop.hive.metastore.api.InvalidObjectException: Unsupported expression '2023 - 05 - 03' (Service: AWSGlue; Status Code: 400; Error Code: InvalidInputException; Request ID: beed68c6-b228-442e-8783-52c25b9d2243; Proxy: null)
> {quote}
>  
> A fix for this is making sure the date passed to glue is quoted



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org