[jira] [Closed] (HUDI-4326) Hudi spark datasource error after migrate from 0.8 to 0.11

["sivabalan narayanan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

sivabalan narayanan closed HUDI-4326.
-------------------------------------
    Resolution: Fixed

> Hudi spark datasource error after migrate from 0.8 to 0.11
> ----------------------------------------------------------
>
>                 Key: HUDI-4326
>                 URL: https://issues.apache.org/jira/browse/HUDI-4326
>             Project: Apache Hudi
>          Issue Type: Bug
>          Components: spark
>            Reporter: Kyle Zhike Chen
>            Assignee: Kyle Zhike Chen
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 0.12.1
>
>
> After updated hudi to 0.11 from 0.8, using {{spark.table(fullTableName)}} to read a hudi table is not working, the table has been sync to hive metastore and spark is connected to the metastore. the error is
> org.sparkproject.guava.util.concurrent.UncheckedExecutionException: org.apache.hudi.exception.HoodieException: 'path' or 'Key: 'hoodie.datasource.read.paths' , default: null description: Comma separated list of file paths to read within a Hudi table. since version: version is not defined deprecated after: version is not defined)' or both must be specified.
> at org.sparkproject.guava.cache.LocalCache$Segment.get(LocalCache.java:2263)
> at org.sparkproject.guava.cache.LocalCache.get(LocalCache.java:4000)
> at org.sparkproject.guava.cache.LocalCache$LocalManualCache.get(LocalCache.java:4789)
> at org.apache.spark.sql.catalyst.catalog.SessionCatalog.
> ...
> Caused by: org.apache.hudi.exception.HoodieException: 'path' or 'Key: 'hoodie.datasource.read.paths' , default: null description: Comma separated list of file paths to read within a Hudi table. since version: version is not defined deprecated after: version is not defined)' or both must be specified.
> at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:78)
> at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:353)
> at org.apache.spark.sql.execution.datasources.FindDataSourceTable.$anonfun$readDataSourceTable$1(DataSourceStrategy.scala:261)
> at org.sparkproject.guava.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4792)
> at org.sparkproject.guava.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599)
> at org.sparkproject.guava.cache.LocalCache$Segment.loadSync(LocalCache.java:2379)
> at org.sparkproject.guava.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342)
> at org.sparkproject.guava.cache.LocalCache$Segment.get(LocalCache.java:2257) 
> After changing the table to the spark data source table, the table SerDeInfo is missing. I created a pull request.
>  
> related GH issue:
> https://github.com/apache/hudi/issues/5861



--
This message was sent by Atlassian Jira
(v8.20.10#820010)