[I] [client] Concurrent remote log downloads can cause Unexpected EOF [fluss]

["fxbing (via GitHub)" <gi...@apache.org>]

fxbing opened a new issue, #3815:
URL: https://github.com/apache/fluss/issues/3815

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
   
   
   ### Fluss version
   
   main (development)
   
   ### Please describe the bug 🐞
   
   When multiple scanners read the same remote log segment, duplicate downloads may target the same local file.
   
   `LogFetcher` may reuse a `fetchableBuckets()` snapshot captured before an asynchronous metadata update, causing an already buffered bucket to be fetched again. `RemoteFileDownloader` then writes directly to the final path, truncating the file while another scanner is reading it.
   
   The reader may fail with `CorruptMessageException: Unexpected EOF`.
   
   Expected: an open reader can finish reading the downloaded segment.
   
   Actual: a duplicate download may truncate the segment being read.
   
   #3263 is related to remote log downloading, but it does not address this concurrency issue.
   
   ### Solution
   
   - Recheck `fetchableBuckets()` under the `LogFetcher` lock before creating fetch requests.
   - Download to a unique temporary file and atomically move the completed file to the final path.
   - Add regression tests for both races.
   
   This does not change any public API or storage format.
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [client] Concurrent remote log downloads can cause Unexpected EOF [fluss]

["fresh-borzoni (via GitHub)" <gi...@apache.org>]

fresh-borzoni closed issue #3815: [client] Concurrent remote log downloads can cause Unexpected EOF
URL: https://github.com/apache/fluss/issues/3815


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org