You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2025/03/09 09:24:00 UTC
[jira] [Updated] (FLINK-37439) Missing file creation with overwrite semantic in ForSt wrapped file system
[ https://issues.apache.org/jira/browse/FLINK-37439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated FLINK-37439:
-----------------------------------
Labels: pull-request-available (was: )
> Missing file creation with overwrite semantic in ForSt wrapped file system
> --------------------------------------------------------------------------
>
> Key: FLINK-37439
> URL: https://issues.apache.org/jira/browse/FLINK-37439
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 2.0.0
> Reporter: Zakelly Lan
> Assignee: Zakelly Lan
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 2.0.0
>
>
> ForSt may create LOG files twice with overwrite, however, the ForSt wrapped file system does not support overwriting files. Exception thrown in this case:
> {code:java}
> Caused by: java.lang.IllegalStateException: Try to add a file that is already in mappingTable, but with inconsistent entry. Key: oss://xxxxxxxxxxxxxxxxxx/ba3bcc9061a3858257e3119f0a6493ab/shared/op_AsyncKeyedProcessOperator_7261c9d38047b376863833deae26476f__4_8__attempt_0/db/LOG, source: FileBackedSource{filePath=/tmp/ba3bcc9061a3858257e3119f0a6493ab/op_AsyncKeyedProcessOperator_7261c9d38047b376863833deae26476f__4_8__attempt_0/db/1b72a4ee-9d4f-4d37-b1d4-781774c4da78}, fileOwnership: PRIVATE_OWNED_BY_DB. Entry in table: MappingEntry{source=FileBackedSource{filePath=/tmp/ba3bcc9061a3858257e3119f0a6493ab/op_AsyncKeyedProcessOperator_7261c9d38047b376863833deae26476f__4_8__attempt_0/db/f60883d6-355e-491b-82d9-1875e5cb151e}, fileOwnership=PRIVATE_OWNED_BY_DB, isDirectory= false}
> at org.apache.flink.util.Preconditions.checkState(Preconditions.java:193) ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at org.apache.flink.state.forst.fs.filemapping.FileMappingManager.getExistingMappingEntry(FileMappingManager.java:107) ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at org.apache.flink.state.forst.fs.filemapping.FileMappingManager.addFileToMappingTable(FileMappingManager.java:97) ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at org.apache.flink.state.forst.fs.filemapping.FileMappingManager.createNewFile(FileMappingManager.java:70) ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at org.apache.flink.state.forst.fs.ForStFlinkFileSystem.create(ForStFlinkFileSystem.java:189) ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at org.apache.flink.state.forst.fs.ForStFlinkFileSystem.create(ForStFlinkFileSystem.java:182) ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at org.apache.flink.state.forst.fs.StringifiedForStFileSystem.create(StringifiedForStFileSystem.java:74) ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at org.forstdb.RocksDB.write0(Native Method) ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at org.forstdb.RocksDB.write(RocksDB.java:1843) ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at org.apache.flink.state.forst.ForStDBWriteBatchWrapper.flush(ForStDBWriteBatchWrapper.java:136) ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> at org.apache.flink.state.forst.ForStWriteBatchOperation.lambda$process$0(ForStWriteBatchOperation.java:72) ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> ... 29 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)