You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pdfbox.apache.org by Or...@beeriprint.co.il on 2016/08/28 08:53:38 UTC
"OutOfMemoryError: Java heap space" even when using
MemoryUsageSetting.setupTempFileOnly()
hi
when try to load large pdf files (70000 pages , 70MB size file), I get the "OutOfMemoryError: Java heap space"
tryed this :
MemoryUsageSetting mus = MemoryUsageSetting.setupTempFileOnly();
PDDocument document = PDDocument.load( file ,mus);
also tryed this :
MemoryUsageSetting mus = MemoryUsageSetting.setupTempFileOnly();
RandomAccessBufferedFileInputStream ra = new RandomAccessBufferedFileInputStream(file);
document = PDDocument.load( ra ,mus);
but in both cases I get this error :
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at org.apache.pdfbox.io.ScratchFileBuffer.addPage(ScratchFileBuffer.java:132)
at org.apache.pdfbox.io.ScratchFileBuffer.<init>(ScratchFileBuffer.java:84)
at org.apache.pdfbox.io.ScratchFile.createBuffer(ScratchFile.java:403)
at org.apache.pdfbox.cos.COSStream.createRawOutputStream(COSStream.java:253)
at org.apache.pdfbox.pdfparser.COSParser.parseCOSStream(COSParser.java:946)
at org.apache.pdfbox.pdfparser.COSParser.parseFileObject(COSParser.java:760)
at org.apache.pdfbox.pdfparser.COSParser.parseObjectDynamically(COSParser.java:721)
at org.apache.pdfbox.pdfparser.COSParser.parseObjectDynamically(COSParser.java:652)
at org.apache.pdfbox.pdfparser.COSParser.parseDictObjects(COSParser.java:612)
at org.apache.pdfbox.pdfparser.PDFParser.initialParse(PDFParser.java:215)
at org.apache.pdfbox.pdfparser.PDFParser.parse(PDFParser.java:249)
at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:966)
at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:885)
at pdftoimage_pdfbox.PrintBookmarks.main(PrintBookmarks.java:87)
Java HotSpot(TM) Client VM warning: MaxNewSize (512000k) is equal to or greater than the entire heap (512000k). A new max generation size of 511936k will be used.
C:\Users\orlev.MANAGEMENT\Documents\NetBeansProjects\PdfToImage_PdfBox\nbproject\build-impl.xml:1039: The following error occurred while executing this line:
looks like MemoryUsageSetting.setupTempFileOnly() has no affect ?
do I use it incorrectlly ?
** I can overcome this problem with -Xmx1500m but had to increase my physical memory.. , and.., I have much bigger files (250000 pages)…
thanks!
or.
Thanks,
Or Lev
Maintenance
Beeri print
Israel
orlev@beeriprint.co.il<ma...@beeriprint.co.il>
Re: "OutOfMemoryError: Java heap space" even when using
MemoryUsageSetting.setupTempFileOnly()
Posted by Tilman Hausherr <TH...@t-online.de>.
Try setting a higher -Xmx value; set a lower dpi, 78 is too big if you
just need a thumbnail, it may be possible to calculate the dpi based on
the page cropbox. Or look at your PDFs and make a guess based on what
images you get. I'd expect something around 20-30 for A4 pages.
Tilman
Am 20.09.2018 um 08:55 schrieb paolodirollo@gmail.com:
> Hi!
>
> I have the same issue with some PDF and this is my code.
>
> @Async(value = "asyncExecutor")
> public void createThumbnailForPDF(File filePDF, String jobNumber)
> {
> try
> {
> if (filePDF != null && jobNumber != null && !jobNumber.isEmpty())
> {
>
> jobNumber = jobNumber.trim();
> long fileSizeInBytes = filePDF.length();
> long fileSizeInKB = fileSizeInBytes / 1024;
> long fileSizeInMB = fileSizeInKB / 1024;
> if (fileSizeInMB > PDF_MAX_SIZE)
> {
> log.error("File too large. PDF_MAX_SIZE: " + PDF_MAX_SIZE + ". Thumbnail not created. Jobnumber: " + jobNumber);
> return;
> }
>
> try
> {
> String fileName = filePDF.getName();
> String extension = fileName.substring(fileName.lastIndexOf("."));
> if (!StringUtils.isEmpty(extension) && extension.equalsIgnoreCase(".pdf"))
> {
> PDDocument documentPDF = null;
> PDFRenderer pdfRenderer = null;
> BufferedImage bim = null;
> try
> {
> documentPDF = PDDocument.load(filePDF);
> if (documentPDF != null)
> {
>
> try
> {
> log.info("PDF document loaded. Jobnumber: " + jobNumber + ". PDF Title: " + documentPDF.getDocumentInformation().getTitle());
> } catch (Exception e)
> {
> log.error("Error reading document title property.", e.getMessage());
> }
>
> pdfRenderer = new PDFRenderer(documentPDF);
> if (pdfRenderer != null)
> {
> bim = pdfRenderer.renderImageWithDPI(0, 78f, ImageType.RGB);
> if (bim != null)
> {
> // thumbnail 200
> int newHeight200 = (int) getNewHeight(bim.getWidth(), bim.getHeight(), 200);
> BufferedImage buff200 = resizeImage(bim, 200, newHeight200);
> if (buff200 != null) ImageIO.write(buff200, "jpg", new File(applicationProperties.getDocumentWithSubfolderRootPath() + File.separator + jobNumber + File.separator + jobNumber + "_200.jpg"));
>
> // thumbnail 100
> int newHeight100 = (int) getNewHeight(bim.getWidth(), bim.getHeight(), 100);
> BufferedImage buff100 = resizeImage(bim, 100, newHeight100);
> if (buff100 != null) ImageIO.write(buff100, "jpg", new File(applicationProperties.getDocumentWithSubfolderRootPath() + File.separator + jobNumber + File.separator + jobNumber + "_100.jpg"));
>
> }
> else
> {
> log.error("Thumbnail creation, Bitmap is null. Jobnumber: " + jobNumber);
> }
>
> }
>
> }
> else
> log.error("Thumbnail creation, failed to load PDF. Jobnumber: " + jobNumber);
>
> } catch (IOException io)
> {
> log.error("Thumbnail creation, IoException while loading PDF. Jobnumber: " + jobNumber, io.getMessage());
> } catch (Exception e)
> {
> log.error("Failded to create thumbnails. Jobnumber: " + jobNumber, e.getMessage());
> } finally
> {
> try
> {
> if (documentPDF != null) documentPDF.close();
> if (bim != null)
> {
> bim.flush();
> bim = null;
> }
> pdfRenderer = null;
>
>
>
> } catch (IOException e)
> {
> log.error("Error while empty the variables after thumbnails creation." + e.getMessage());
> }
> }
> }
> else
> {
> log.info("Current file is not a PDF");
> }
>
> } catch (Exception e)
> {
> log.error("Failded to create thumbnails. Jobnumber: " + jobNumber, e);
> }
> }
>
> } catch (Exception e)
> {
> log.error("createThumbnailForPDF something went wrong during thumbnails creation.", e.getMessage());
> }
> }
>
> I think that the code is pretty much correct. So the only solution is to increase the memory of the JVM?
>
> Regards,
> Paolo
>
>
> On 2016/08/28 11:30:43, Tilman Hausherr <TH...@t-online.de> wrote:
>> Am 28.08.2016 um 10:53 schrieb Orlev@beeriprint.co.il:
>>> looks like MemoryUsageSetting.setupTempFileOnly() has no affect ?
>>> do I use it incorrectlly ?
>> You're using it correctly. The solution is below:
>>
>>> ** I can overcome this problem with -Xmx1500m but had to increase my physical memory.. , and.., I have much bigger files (250000 pages)
>>
>> Additionally, if you're doing rendering, take care not to keep the
>> result images in memory.
>>
>>
>> Tilman
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
>> For additional commands, e-mail: users-help@pdfbox.apache.org
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: users-help@pdfbox.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
Re: "OutOfMemoryError: Java heap space" even when using MemoryUsageSetting.setupTempFileOnly()
Posted by .
Hi!
I have the same issue with some PDF and this is my code.
@Async(value = "asyncExecutor")
public void createThumbnailForPDF(File filePDF, String jobNumber)
{
try
{
if (filePDF != null && jobNumber != null && !jobNumber.isEmpty())
{
jobNumber = jobNumber.trim();
long fileSizeInBytes = filePDF.length();
long fileSizeInKB = fileSizeInBytes / 1024;
long fileSizeInMB = fileSizeInKB / 1024;
if (fileSizeInMB > PDF_MAX_SIZE)
{
log.error("File too large. PDF_MAX_SIZE: " + PDF_MAX_SIZE + ". Thumbnail not created. Jobnumber: " + jobNumber);
return;
}
try
{
String fileName = filePDF.getName();
String extension = fileName.substring(fileName.lastIndexOf("."));
if (!StringUtils.isEmpty(extension) && extension.equalsIgnoreCase(".pdf"))
{
PDDocument documentPDF = null;
PDFRenderer pdfRenderer = null;
BufferedImage bim = null;
try
{
documentPDF = PDDocument.load(filePDF);
if (documentPDF != null)
{
try
{
log.info("PDF document loaded. Jobnumber: " + jobNumber + ". PDF Title: " + documentPDF.getDocumentInformation().getTitle());
} catch (Exception e)
{
log.error("Error reading document title property.", e.getMessage());
}
pdfRenderer = new PDFRenderer(documentPDF);
if (pdfRenderer != null)
{
bim = pdfRenderer.renderImageWithDPI(0, 78f, ImageType.RGB);
if (bim != null)
{
// thumbnail 200
int newHeight200 = (int) getNewHeight(bim.getWidth(), bim.getHeight(), 200);
BufferedImage buff200 = resizeImage(bim, 200, newHeight200);
if (buff200 != null) ImageIO.write(buff200, "jpg", new File(applicationProperties.getDocumentWithSubfolderRootPath() + File.separator + jobNumber + File.separator + jobNumber + "_200.jpg"));
// thumbnail 100
int newHeight100 = (int) getNewHeight(bim.getWidth(), bim.getHeight(), 100);
BufferedImage buff100 = resizeImage(bim, 100, newHeight100);
if (buff100 != null) ImageIO.write(buff100, "jpg", new File(applicationProperties.getDocumentWithSubfolderRootPath() + File.separator + jobNumber + File.separator + jobNumber + "_100.jpg"));
}
else
{
log.error("Thumbnail creation, Bitmap is null. Jobnumber: " + jobNumber);
}
}
}
else
log.error("Thumbnail creation, failed to load PDF. Jobnumber: " + jobNumber);
} catch (IOException io)
{
log.error("Thumbnail creation, IoException while loading PDF. Jobnumber: " + jobNumber, io.getMessage());
} catch (Exception e)
{
log.error("Failded to create thumbnails. Jobnumber: " + jobNumber, e.getMessage());
} finally
{
try
{
if (documentPDF != null) documentPDF.close();
if (bim != null)
{
bim.flush();
bim = null;
}
pdfRenderer = null;
} catch (IOException e)
{
log.error("Error while empty the variables after thumbnails creation." + e.getMessage());
}
}
}
else
{
log.info("Current file is not a PDF");
}
} catch (Exception e)
{
log.error("Failded to create thumbnails. Jobnumber: " + jobNumber, e);
}
}
} catch (Exception e)
{
log.error("createThumbnailForPDF something went wrong during thumbnails creation.", e.getMessage());
}
}
I think that the code is pretty much correct. So the only solution is to increase the memory of the JVM?
Regards,
Paolo
On 2016/08/28 11:30:43, Tilman Hausherr <TH...@t-online.de> wrote:
> Am 28.08.2016 um 10:53 schrieb Orlev@beeriprint.co.il:
> > looks like MemoryUsageSetting.setupTempFileOnly() has no affect ?
> > do I use it incorrectlly ?
>
> You're using it correctly. The solution is below:
>
> >
> > ** I can overcome this problem with -Xmx1500m but had to increase my physical memory.. , and.., I have much bigger files (250000 pages)
>
>
> Additionally, if you're doing rendering, take care not to keep the
> result images in memory.
>
>
> Tilman
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: users-help@pdfbox.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
Re: "OutOfMemoryError: Java heap space" even when using MemoryUsageSetting.setupTempFileOnly()
Posted by John Hewson <jo...@jahewson.com>.
> On 28 Aug 2016, at 04:30, Tilman Hausherr <TH...@t-online.de> wrote:
>
> Am 28.08.2016 um 10:53 schrieb Orlev@beeriprint.co.il:
>> looks like MemoryUsageSetting.setupTempFileOnly() has no affect ?
>> do I use it incorrectlly ?
>
> You're using it correctly. The solution is below:
>
>>
>> ** I can overcome this problem with -Xmx1500m but had to increase my physical memory.. , and.., I have much bigger files (250000 pages)…
>
>
> Additionally, if you're doing rendering, take care not to keep the result images in memory.
Also make sure you’re not creating a List<PDPage> or anything similar.
— John
> Tilman
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: users-help@pdfbox.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
Re: "OutOfMemoryError: Java heap space" even when using
MemoryUsageSetting.setupTempFileOnly()
Posted by Tilman Hausherr <TH...@t-online.de>.
Am 28.08.2016 um 10:53 schrieb Orlev@beeriprint.co.il:
> looks like MemoryUsageSetting.setupTempFileOnly() has no affect ?
> do I use it incorrectlly ?
You're using it correctly. The solution is below:
>
> ** I can overcome this problem with -Xmx1500m but had to increase my physical memory.. , and.., I have much bigger files (250000 pages)
Additionally, if you're doing rendering, take care not to keep the
result images in memory.
Tilman
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org