You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Yi Hu (Jira)" <ji...@apache.org> on 2022/04/15 20:09:00 UTC

[jira] [Created] (BEAM-14315) Update fileio.MatchContinuously to allow reading already read files with a new timestamp

Yi Hu created BEAM-14315:
----------------------------

             Summary: Update fileio.MatchContinuously to allow reading already read files with a new timestamp
                 Key: BEAM-14315
                 URL: https://issues.apache.org/jira/browse/BEAM-14315
             Project: Beam
          Issue Type: New Feature
          Components: io-py-common
            Reporter: Yi Hu


This will be the Python counterpart of BEAM-14267.

For fileio.MatchContinuously, we want to add an option to choose to consider a file new if it has a different timestamp from an existing file, even if the file itself has the same name.

See the following design doc for more detail:

https://docs.google.com/document/d/1xnacyLGNh6rbPGgTAh5D1gZVR8rHUBsMMRV3YkvlL08/edit?usp=sharing&resourcekey=0-be0uF-DdmwAz6Vg4Li9FNw



--
This message was sent by Atlassian Jira
(v8.20.1#820001)