You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by xiaowenhao <xi...@corp.netease.com> on 2022/08/03 06:18:31 UTC

Flink 1.15 FileSink cannot merge compressed small files

Hey Community,

  

have you ever used the 1.15 FileSink DataStream api's function of merging
small files? I wrote files  and compressed them in gz format, but found that
the merged files could not be read, and I used ConcatFileCompactor  to merged
files.

  

Below is my relevant code:

![](cid:E76384F3-0C81-4858-B662-98A47EB016DA@mailmaster)  

  

When the files are merged, I find that the merged file is not readable:

![](cid:259B6069-744E-45C7-B0F8-3976781AC835@mailmaster)  

  

Does 1.15 File sink not support merging of compressed files? Or is there
something I haven't noticed?

  

Thank you!

[![](https://mail-online.nosdn.127.net/qiyelogo/defaultAvatar.png)|
xiaowenhao| ![](http://nos.netease.com/mail-
online/3d676aa833f8a1f6d55fb2fceb3cd239.jpg)  
---|---|---  
xiaowenhao@corp.netease.com  
](https://maas.mail.163.com/dashi-web-
extend/html/proSignature.html?ftlId=1&name=xiaowenhao&uid=xiaowenhao%40corp.netease.com&iconUrl=https%3A%2F%2Fmail-
online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22xiaowenhao%40corp.netease.com%22%5D&logoUrl=http%3A%2F%2Fnos.netease.com%2Fmail-
online%2F9add9fb5ce630a98453b83487dc4a138.jpg)