You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Joao Boto (JIRA)" <ji...@apache.org> on 2019/08/07 08:12:00 UTC

[jira] [Created] (FLINK-13609) StreamingFileSink - reset part counter on bucket change

Joao Boto created FLINK-13609:
---------------------------------

             Summary: StreamingFileSink - reset part counter on bucket change
                 Key: FLINK-13609
                 URL: https://issues.apache.org/jira/browse/FLINK-13609
             Project: Flink
          Issue Type: Improvement
          Components: Connectors / FileSystem
            Reporter: Joao Boto


When writing to files using StreamingFileSink on bucket change we expect that partcounter will reset its counter to 0

as a example
 * using DateTimeBucketAssigner using ({color:#6a8759}yyyy/MM/dd/HH{color}) 
 * and ten files hour (for simplicity)

this will create the:
 * bucket 2019/08/07/00 with files partfile-0-0 to partfile-0-9
 * bucket 2019/08/07/01 with files partfile-0-10 to partfile-0-19
 * bucket 2019/08/07/02 with files partfile-0-20 to partfile-0-29

and we expect this:
 * bucket 2019/08/07/00 with files partfile-0-0 to partfile-0-9
 * bucket 2019/08/07/01 with files partfile-0-0 to partfile-0-9
 * bucket 2019/08/07/02 with files partfile-0-0 to partfile-0-9

 

[~kkl0u] i don't know if it's the expected behavior  (or this can be configured)



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)