[GitHub] [hudi] nsivabalan commented on a change in pull request #2903: [HUDI-1850] Fixing read of a empty table but with failed write

[GitBox <gi...@apache.org>]

nsivabalan commented on a change in pull request #2903:
URL: https://github.com/apache/hudi/pull/2903#discussion_r656545767



##########
File path: hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/HoodieSparkSqlWriterSuite.scala
##########
@@ -229,7 +231,42 @@ class HoodieSparkSqlWriterSuite extends FunSuite with Matchers {
     }
   }
 
-  test("test bulk insert dataset with datasource impl multiple rounds") {
+  test("test read of a table with one failed write") {
+    initSparkContext("test_read_table_with_one_failed_write")
+    val path = java.nio.file.Files.createTempDirectory("hoodie_test_path")
+    try {
+      val hoodieFooTableName = "hoodie_foo_tbl"
+      val fooTableModifier = Map("path" -> path.toAbsolutePath.toString,
+        HoodieWriteConfig.TABLE_NAME -> hoodieFooTableName,
+        DataSourceWriteOptions.RECORDKEY_FIELD_OPT_KEY -> "_row_key",
+        DataSourceWriteOptions.PARTITIONPATH_FIELD_OPT_KEY -> "partition")
+
+      val fooTableParams = HoodieWriterUtils.parametersWithWriteDefaults(fooTableModifier)
+      val props = new Properties()
+      fooTableParams.foreach(entry => props.setProperty(entry._1, entry._2))
+      val metaClient = HoodieTableMetaClient.initTableAndGetMetaClient(spark.sparkContext.hadoopConfiguration, path.toAbsolutePath.toString, props)
+
+      val partitionAndFileId = new util.HashMap[String, String]()
+      partitionAndFileId.put(HoodieTestDataGenerator.DEFAULT_FIRST_PARTITION_PATH, "file-1")
+
+      HoodieTestTable.of(metaClient).withPartitionMetaFiles(HoodieTestDataGenerator.DEFAULT_FIRST_PARTITION_PATH)
+        .addInflightCommit("001")
+        .withBaseFilesInPartitions(partitionAndFileId)
+
+      val snapshotDF1 = spark.read.format("org.apache.hudi")
+        .load(path.toAbsolutePath.toString + "/*/*/*/*")
+      snapshotDF1.count()
+      assertFalse(true)
+    }  catch {
+      case e: InvalidTableException =>
+        assertTrue(e.getMessage.contains("Invalid Hoodie Table"))
+    } finally {
+      spark.stop()

Review comment:
       yes, we initialize at the beginning of each test. I know we have to fix this entire class for reuse in general. @hmit is working on this refactoring. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org