[jira] [Updated] (HUDI-3108) Fix Purge Drop MOR Table Cause error

["sivabalan narayanan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-3108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

sivabalan narayanan updated HUDI-3108:
--------------------------------------
    Fix Version/s: 0.11.0
                   0.10.1

>  Fix Purge Drop MOR Table Cause error
> -------------------------------------
>
>                 Key: HUDI-3108
>                 URL: https://issues.apache.org/jira/browse/HUDI-3108
>             Project: Apache Hudi
>          Issue Type: Bug
>          Components: Spark Integration
>            Reporter: Forward Xu
>            Assignee: Forward Xu
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 0.11.0, 0.10.1
>
>         Attachments: image-2021-12-29-09-52-30-999.png, image-2021-12-29-10-04-31-025.png
>
>
> 1.Spark creates three tables, such as `hudi_01`, `hudi_01_ro`, `hudi_01_rt`, when creating mor table. They share the same table directory.
> 2.purge drop hudi_01_ro will delete the table directory.
> Then operation hudi_01_rt table will report an error.
> drop table test_hudi_table_ro purge;
> select * from test_hudi_table_rt;
> {code:java}
> 	at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:381)
> 	at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.$anonfun$processLine$1(SparkSQLCLIDriver.scala:500)
> 	at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.$anonfun$processLine$1$adapted(SparkSQLCLIDriver.scala:494)
> 	at scala.collection.Iterator.foreach(Iterator.scala:941)
> 	at scala.collection.Iterator.foreach$(Iterator.scala:941)
> 	at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
> 	at scala.collection.IterableLike.foreach(IterableLike.scala:74)
> 	at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
> 	at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
> 	at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processLine(SparkSQLCLIDriver.scala:494)
> 	at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:284)
> 	at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> 	at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951)
> 	at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
> 	at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
> 	at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
> 	at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1039)
> 	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1048)
> 	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> Caused by: java.io.FileNotFoundException: File file:/opt/sourcecode/data-lake/warehouse/hudi/test_hudi_table does not exist
> 	at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:611)
> 	at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:824)
> 	at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:601)
> 	at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:428)
> 	at org.apache.hudi.common.util.TablePathUtils.getTablePath(TablePathUtils.java:50)
> 	at org.apache.hudi.DataSourceUtils.getTablePath(DataSourceUtils.java:76)
> 	at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:103)
> 	at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:353)
> 	at org.apache.spark.sql.execution.datasources.FindDataSourceTable.$anonfun$readDataSourceTable$1(DataSourceStrategy.scala:261)
> 	at org.sparkproject.guava.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4792)
> 	at org.sparkproject.guava.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599)
> 	at org.sparkproject.guava.cache.LocalCache$Segment.loadSync(LocalCache.java:2379) {code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)