You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Site Register <si...@ymail.com.INVALID> on 2020/11/23 18:20:14 UTC

OutOfMemoryError for unzipping huge file

Hi Camel,
Is there a way to unzip huge file by using streaming? I got OutOfMemoryError if the uncompressed file size (4GB). I tried to use custom unzip processor with streaming and it works fine. 
Very appreciated if you have a solution. 
/* Unzip .gz file route */
from("file:...")
.unmarshal().gzipDeflater()

.to("file:...");


/* Unzip .gz file using custom processor */
from("file:...")
.processor("unzipProcessor");


/* unzipProcessor -> 9 seconds to unzip for uncompressed file (4GB) */
byte[] buffer = new byte[102400];

int length;

FileInputStream fis = new FileInputStream(fileName);
GZIPInputStream gzis = new GZIPInputStream(fis);
FileOutputStream fos = new FileOutputStream("d:\\temp\\test.json");
while ((length = gzis.read(buffer)) > 0) {
 fos.write(buffer, 0, length);
}