You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Jackeyzhe (via GitHub)" <gi...@apache.org> on 2026/09/04 04:22:16 UTC

[PR] [FLINK-25453] [core] Ignore files deleted concurrently in LocalFileSystem#listStatus [flink]

Jackeyzhe opened a new pull request, #29099:
URL: https://github.com/apache/flink/pull/29099

## What is the purpose of the change

This pull request fixes [FLINK-25453](https://issues.apache.org/jira/browse/FLINK-25453).

`LocalFileSystem.listStatus()` first obtains the names of the entries in a
directory and then retrieves the status of each entry. If an entry is deleted
between these operations, the status lookup throws `FileNotFoundException` and
the complete directory listing fails.

This change ignores an entry that disappears during its status lookup while
preserving the successfully retrieved entries. Other I/O failures continue to
be propagated.

## Brief change log

- Ignore `FileNotFoundException` for individual entries that disappear while
`LocalFileSystem.listStatus()` is collecting their statuses.
- Return a compact result array containing only the entries whose statuses were
retrieved successfully.
- Add a deterministic regression test that removes an entry between directory
listing and status retrieval.

## Verifying this change

This change added tests and can be verified as follows:

- `./mvnw -pl flink-core -DskipITs -Dtest=LocalFileSystemTest test`
- `./mvnw -pl flink-core -DskipITs -Dtest=LocalFileSystemTest,LocalFileSystemBehaviorTest test`

The second command passed locally with 32 tests, 0 failures, 0 errors, and 1
skipped test. Checkstyle and Spotless also passed.

## Does this pull request potentially affect one of the following parts:

- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no

## Documentation

- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable

---

##### Was generative AI tooling used to co-author this PR?

- [X] Yes (please specify the tool below)

Generated-by: OpenAI Codex (GPT-5)
reviewed by human, and I'm responsible for the quality.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-25453] [core] Ignore files deleted concurrently in LocalFileSystem#listStatus [flink]

Posted by "flinkbot (via GitHub)" <gi...@apache.org>.

flinkbot commented on PR #29099:
URL: https://github.com/apache/flink/pull/29099#issuecomment-5535676383

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "3351bd066f926499a9fa1688b9f5661470605bdc",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3351bd066f926499a9fa1688b9f5661470605bdc",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 3351bd066f926499a9fa1688b9f5661470605bdc UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-25453] [core] Ignore files deleted concurrently in LocalFileSystem#listStatus [flink]

Posted by "Jackeyzhe (via GitHub)" <gi...@apache.org>.

Jackeyzhe commented on PR #29099:
URL: https://github.com/apache/flink/pull/29099#issuecomment-5883088053

   Hi, could someone take a look at this PR when convenient?
   
   The current head is mergeable and Azure is green. The two-file change only skips entries that disappear during LocalFileSystem#listStatus, while preserving other I/O failures. It includes a deterministic regression test; the local filesystem test run reported 32 tests with no failures or errors and one skip.
   
   Please let me know if the approach needs adjustment or if a different reviewer would be more appropriate. Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org