You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "hackergin (via GitHub)" <gi...@apache.org> on 2024/09/10 03:23:24 UTC

[PR] [FLINK-36242][table] Fix unstable test in MaterializedTableITCase [flink]

hackergin opened a new pull request, #25299:
URL: https://github.com/apache/flink/pull/25299

   ## Fix unstable test in MaterializedTableITCase
   
   Problem:
   
   As shown in the error message above, in the test method afterEach, we will list all materialized tables and then drop them to prevent any remaining refresh tasks.
   However, since the Drop Materialized Table has already been deleted, it causes the error mentioned above.
   
   * Why does listing tables show they exist when dropping tables results in an error stating they do not exist?
   1. In the test dropMaterializedTableWithDeletedRefreshWorkflowInFullMode, we manually dropped the Materialized Table.
   2. Despite manual deletion, background refresh tasks are still being submitted. This leads to data continuing to be written into the corresponding table data directory.
   3. TestFileSystemCatalog lists all directories for existing tables during listTable. If a directory exists, it returns true. However, during dropTable it checks if both table and schema files exist simultaneously. This inconsistency caused the mentioned issue.
   Solution:
   
   1. Fix TestFileSystemCatalog logic for listTable by checking not only directories but also schema file existence.
   2. To further avoid this problem, change all tables in MaterializedTableITCase to be manually dropped instead.
   
   
   ## Brief change log
   
     - *Fix inconsistent logic between listTable and getTable in TestFileSystemCatalog*
     - *Drop all materialized tables manually instead of automatically dropping them in the after method*
   
   
   ## Verifying this change
   
   1. The existing UT of MaterializedTableITCase can cover these modifications.
   2. Modify some logic in TestFileSystemCatalogTest to cover the new changes.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): (no)
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
     - The serializers: (no)
     - The runtime per-record code paths (performance sensitive): (no)
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
     - The S3 file system connector: (no)
   
   ## Documentation
   
     - Does this pull request introduce a new feature? (no)
     - If yes, how is the feature documented? (not documented)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-36242][table] Fix unstable test in MaterializedTableITCase [flink]

Posted by "lsyldliu (via GitHub)" <gi...@apache.org>.
lsyldliu commented on code in PR #25299:
URL: https://github.com/apache/flink/pull/25299#discussion_r1756678516


##########
flink-test-utils-parent/flink-table-filesystem-test-utils/src/test/java/org/apache/flink/table/file/testutils/catalog/TestFileSystemCatalogTest.java:
##########
@@ -305,12 +306,22 @@ public void testListTable() throws Exception {
 
         // test list table
         List<String> tables = catalog.listTables(TEST_DEFAULT_DATABASE);
-        assertThat(tables.contains(tablePath1.getObjectName())).isTrue();
-        assertThat(tables.contains(tablePath2.getObjectName())).isTrue();
+        assertThat(tables).contains(tablePath1.getObjectName());
+        assertThat(tables).contains(tablePath2.getObjectName());
 
         // test list non-exist database table
         assertThrows(
                 DatabaseNotExistException.class, () -> catalog.listTables(NONE_EXIST_DATABASE));
+
+        // test list table ignore the empty path
+        String nonExistTablePath =
+                String.format(
+                        "%s/%s/%s",
+                        tempFile.getAbsolutePath(), TEST_DEFAULT_DATABASE, NONE_EXIST_TABLE);
+        File file = new File(nonExistTablePath);

Review Comment:
   It would be better: Files.createDirectories(Paths.get(nonExistTablePath)) ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-36242][table] Fix unstable test in MaterializedTableITCase [flink]

Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #25299:
URL: https://github.com/apache/flink/pull/25299#issuecomment-2339536566

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "55cc34dbbde201b618473603cfbda0c48e5ef14c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "55cc34dbbde201b618473603cfbda0c48e5ef14c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 55cc34dbbde201b618473603cfbda0c48e5ef14c UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-36242][table] Fix unstable test in MaterializedTableITCase [flink]

Posted by "lsyldliu (via GitHub)" <gi...@apache.org>.
lsyldliu merged PR #25299:
URL: https://github.com/apache/flink/pull/25299


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org