[GitHub] [flink] yxu-valleytider edited a comment on issue #9581: [FLINK-13864][streaming]: Modify the StreamingFileSink Builder interface to allow for easier subclassing of StreamingFileSink

[GitBox <gi...@apache.org>]

yxu-valleytider edited a comment on issue #9581: [FLINK-13864][streaming]: Modify the StreamingFileSink Builder interface to allow for easier subclassing of StreamingFileSink 
URL: https://github.com/apache/flink/pull/9581#issuecomment-528763841
 
 
   Hi @kl0u thanks a lot for the suggestions. 
   
   Did more investigations. I suspect there could be potential type clashing if `bucketFactory` and `bucketAssigner` have different `BucketID` type.  For example:
   
   ```
   TestBucketFactory<IN, String> bucketFactory;
   TestBucketAssigner<IN, Integer> bucketAssigner;
   
   StreamingFileSink.forBulkFormat(path, writer)
   			.withBucketFactory(bucketFactory)
   			.withBucketAssigner(bucketAssigner)
   			.build();
   ``` 
   
   Hence in the [new commit](https://github.com/apache/flink/pull/9581/commits/df82b55379806ac8502ef92999a1abc7f9a0056b) I'm putting back the old methods, which construct a new builder when altering the `BucketID` type.  Changed the function names to reflect that explicitly.  If a subclass also wants this capability, it has to override these methods (to return the correct subclass type).  
   
   I can put up a few more tests for the new functions. Do you think they can be in a separate PR ? Thanks.
   
   
   > I will have a look first thing in the morning.
   > 
   > In general changes look good, but I would like to have another pass. Also I think it is worth adding a test that verifies that we can change the type of the `BucketID` without problems. Essentially testing the `withBucketAssigner()` and the corresponding `withBucketAssignerAndPolicy()` for the `rowFormat` builder. This is mainly to guarantee that nobody will break it in the future. Such a test should have been added earlier when the initial `StreamingFileSink` was written :( .
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services