You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pdfbox.apache.org by Or...@beeriprint.co.il on 2016/08/28 08:53:38 UTC

"OutOfMemoryError: Java heap space" even when using MemoryUsageSetting.setupTempFileOnly()

hi
when try to load large pdf files (70000 pages , 70MB size file), I get the "OutOfMemoryError: Java heap space"
tryed this :

MemoryUsageSetting mus = MemoryUsageSetting.setupTempFileOnly();
PDDocument document = PDDocument.load( file ,mus);

also tryed this :
MemoryUsageSetting mus = MemoryUsageSetting.setupTempFileOnly();
RandomAccessBufferedFileInputStream ra = new RandomAccessBufferedFileInputStream(file);
document = PDDocument.load( ra ,mus);

but in both cases I get this error :

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
                at org.apache.pdfbox.io.ScratchFileBuffer.addPage(ScratchFileBuffer.java:132)
                at org.apache.pdfbox.io.ScratchFileBuffer.<init>(ScratchFileBuffer.java:84)
                at org.apache.pdfbox.io.ScratchFile.createBuffer(ScratchFile.java:403)
                at org.apache.pdfbox.cos.COSStream.createRawOutputStream(COSStream.java:253)
                at org.apache.pdfbox.pdfparser.COSParser.parseCOSStream(COSParser.java:946)
                at org.apache.pdfbox.pdfparser.COSParser.parseFileObject(COSParser.java:760)
                at org.apache.pdfbox.pdfparser.COSParser.parseObjectDynamically(COSParser.java:721)
                at org.apache.pdfbox.pdfparser.COSParser.parseObjectDynamically(COSParser.java:652)
                at org.apache.pdfbox.pdfparser.COSParser.parseDictObjects(COSParser.java:612)
                at org.apache.pdfbox.pdfparser.PDFParser.initialParse(PDFParser.java:215)
                at org.apache.pdfbox.pdfparser.PDFParser.parse(PDFParser.java:249)
                at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:966)
                at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:885)
                at pdftoimage_pdfbox.PrintBookmarks.main(PrintBookmarks.java:87)
Java HotSpot(TM) Client VM warning: MaxNewSize (512000k) is equal to or greater than the entire heap (512000k).  A new max generation size of 511936k will be used.
C:\Users\orlev.MANAGEMENT\Documents\NetBeansProjects\PdfToImage_PdfBox\nbproject\build-impl.xml:1039: The following error occurred while executing this line:


looks like MemoryUsageSetting.setupTempFileOnly() has no affect ?
do I use it incorrectlly ?

** I can overcome this problem with -Xmx1500m but had to increase my physical memory.. , and.., I have much bigger files (250000 pages)…


thanks!
or.

Thanks,
Or Lev
Maintenance
Beeri print
Israel
orlev@beeriprint.co.il<ma...@beeriprint.co.il>




Re: "OutOfMemoryError: Java heap space" even when using MemoryUsageSetting.setupTempFileOnly()

Posted by Tilman Hausherr <TH...@t-online.de>.
Try setting a higher -Xmx value; set a lower dpi, 78 is too big if you 
just need a thumbnail, it may be possible to calculate the dpi based on 
the page cropbox. Or look at your PDFs and make a guess based on what 
images you get. I'd expect something around 20-30 for A4 pages.

Tilman

Am 20.09.2018 um 08:55 schrieb paolodirollo@gmail.com:
> Hi!
>
> I have the same issue with some PDF and this is my code.
>
> @Async(value = "asyncExecutor")
> 	public void createThumbnailForPDF(File filePDF, String jobNumber)
> 	{
> 		try
> 		{
> 			if (filePDF != null && jobNumber != null && !jobNumber.isEmpty())
> 			{
>
> 				jobNumber = jobNumber.trim();
> 				long fileSizeInBytes = filePDF.length();
> 				long fileSizeInKB = fileSizeInBytes / 1024;
> 				long fileSizeInMB = fileSizeInKB / 1024;
> 				if (fileSizeInMB > PDF_MAX_SIZE)
> 				{
> 					log.error("File too large. PDF_MAX_SIZE: " + PDF_MAX_SIZE + ". Thumbnail not created. Jobnumber: " + jobNumber);
> 					return;
> 				}
>
> 				try
> 				{
> 					String fileName = filePDF.getName();
> 					String extension = fileName.substring(fileName.lastIndexOf("."));
> 					if (!StringUtils.isEmpty(extension) && extension.equalsIgnoreCase(".pdf"))
> 					{
> 						PDDocument documentPDF = null;
> 						PDFRenderer pdfRenderer = null;
> 						BufferedImage bim = null;
> 						try
> 						{
> 							documentPDF = PDDocument.load(filePDF);
> 							if (documentPDF != null)
> 							{
>
> 								try
> 								{
> 									log.info("PDF document loaded. Jobnumber: " + jobNumber + ". PDF Title: " + documentPDF.getDocumentInformation().getTitle());
> 								} catch (Exception e)
> 								{
> 									log.error("Error reading document title property.", e.getMessage());
> 								}
>
> 								pdfRenderer = new PDFRenderer(documentPDF);
> 								if (pdfRenderer != null)
> 								{
> 									bim = pdfRenderer.renderImageWithDPI(0, 78f, ImageType.RGB);
> 									if (bim != null)
> 									{
> 										// thumbnail 200
> 										int newHeight200 = (int) getNewHeight(bim.getWidth(), bim.getHeight(), 200);
> 										BufferedImage buff200 = resizeImage(bim, 200, newHeight200);
> 										if (buff200 != null) ImageIO.write(buff200, "jpg", new File(applicationProperties.getDocumentWithSubfolderRootPath() + File.separator + jobNumber + File.separator + jobNumber + "_200.jpg"));
>
> 										// thumbnail 100
> 										int newHeight100 = (int) getNewHeight(bim.getWidth(), bim.getHeight(), 100);
> 										BufferedImage buff100 = resizeImage(bim, 100, newHeight100);
> 										if (buff100 != null) ImageIO.write(buff100, "jpg", new File(applicationProperties.getDocumentWithSubfolderRootPath() + File.separator + jobNumber + File.separator + jobNumber + "_100.jpg"));
>
> 									}
> 									else
> 									{
> 										log.error("Thumbnail creation, Bitmap is null. Jobnumber: " + jobNumber);
> 									}
>
> 								}
>
> 							}
> 							else
> 								log.error("Thumbnail creation, failed to load PDF. Jobnumber: " + jobNumber);
>
> 						} catch (IOException io)
> 						{
> 							log.error("Thumbnail creation, IoException while loading PDF. Jobnumber: " + jobNumber, io.getMessage());
> 						} catch (Exception e)
> 						{
> 							log.error("Failded to create thumbnails. Jobnumber: " + jobNumber, e.getMessage());
> 						} finally
> 						{
> 							try
> 							{
> 								if (documentPDF != null) documentPDF.close();
> 								if (bim != null)
> 								{
> 									bim.flush();
> 									bim = null;
> 								}
> 								pdfRenderer = null;
>
> 								
>
> 							} catch (IOException e)
> 							{
> 								log.error("Error while empty the variables after thumbnails creation." + e.getMessage());
> 							}
> 						}
> 					}
> 					else
> 					{
> 						log.info("Current file is not a PDF");
> 					}
>
> 				} catch (Exception e)
> 				{
> 					log.error("Failded to create thumbnails. Jobnumber: " + jobNumber, e);
> 				}
> 			}
>
> 		} catch (Exception e)
> 		{
> 			log.error("createThumbnailForPDF something went wrong during thumbnails creation.", e.getMessage());
> 		}
> 	}
>
> I think that the code is pretty much correct. So the only solution is to increase the memory of the JVM?
>
> Regards,
> Paolo
>
>
> On 2016/08/28 11:30:43, Tilman Hausherr <TH...@t-online.de> wrote:
>> Am 28.08.2016 um 10:53 schrieb Orlev@beeriprint.co.il:
>>> looks like MemoryUsageSetting.setupTempFileOnly() has no affect ?
>>> do I use it incorrectlly ?
>> You're using it correctly. The solution is below:
>>
>>> ** I can overcome this problem with -Xmx1500m but had to increase my physical memory.. , and.., I have much bigger files (250000 pages)
>>
>> Additionally, if you're doing rendering, take care not to keep the
>> result images in memory.
>>
>>
>> Tilman
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
>> For additional commands, e-mail: users-help@pdfbox.apache.org
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: users-help@pdfbox.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org


Re: "OutOfMemoryError: Java heap space" even when using MemoryUsageSetting.setupTempFileOnly()

Posted by .
Hi! 

I have the same issue with some PDF and this is my code.

@Async(value = "asyncExecutor")
	public void createThumbnailForPDF(File filePDF, String jobNumber)
	{
		try
		{
			if (filePDF != null && jobNumber != null && !jobNumber.isEmpty())
			{

				jobNumber = jobNumber.trim();
				long fileSizeInBytes = filePDF.length();
				long fileSizeInKB = fileSizeInBytes / 1024;
				long fileSizeInMB = fileSizeInKB / 1024;
				if (fileSizeInMB > PDF_MAX_SIZE)
				{
					log.error("File too large. PDF_MAX_SIZE: " + PDF_MAX_SIZE + ". Thumbnail not created. Jobnumber: " + jobNumber);
					return;
				}

				try
				{
					String fileName = filePDF.getName();
					String extension = fileName.substring(fileName.lastIndexOf("."));
					if (!StringUtils.isEmpty(extension) && extension.equalsIgnoreCase(".pdf"))
					{
						PDDocument documentPDF = null;
						PDFRenderer pdfRenderer = null;
						BufferedImage bim = null;
						try
						{
							documentPDF = PDDocument.load(filePDF);
							if (documentPDF != null)
							{

								try
								{
									log.info("PDF document loaded. Jobnumber: " + jobNumber + ". PDF Title: " + documentPDF.getDocumentInformation().getTitle());
								} catch (Exception e)
								{
									log.error("Error reading document title property.", e.getMessage());
								}

								pdfRenderer = new PDFRenderer(documentPDF);
								if (pdfRenderer != null)
								{
									bim = pdfRenderer.renderImageWithDPI(0, 78f, ImageType.RGB);
									if (bim != null)
									{
										// thumbnail 200
										int newHeight200 = (int) getNewHeight(bim.getWidth(), bim.getHeight(), 200);
										BufferedImage buff200 = resizeImage(bim, 200, newHeight200);
										if (buff200 != null) ImageIO.write(buff200, "jpg", new File(applicationProperties.getDocumentWithSubfolderRootPath() + File.separator + jobNumber + File.separator + jobNumber + "_200.jpg"));

										// thumbnail 100
										int newHeight100 = (int) getNewHeight(bim.getWidth(), bim.getHeight(), 100);
										BufferedImage buff100 = resizeImage(bim, 100, newHeight100);
										if (buff100 != null) ImageIO.write(buff100, "jpg", new File(applicationProperties.getDocumentWithSubfolderRootPath() + File.separator + jobNumber + File.separator + jobNumber + "_100.jpg"));

									}
									else
									{
										log.error("Thumbnail creation, Bitmap is null. Jobnumber: " + jobNumber);
									}

								}

							}
							else
								log.error("Thumbnail creation, failed to load PDF. Jobnumber: " + jobNumber);

						} catch (IOException io)
						{
							log.error("Thumbnail creation, IoException while loading PDF. Jobnumber: " + jobNumber, io.getMessage());
						} catch (Exception e)
						{
							log.error("Failded to create thumbnails. Jobnumber: " + jobNumber, e.getMessage());
						} finally
						{
							try
							{
								if (documentPDF != null) documentPDF.close();
								if (bim != null)
								{
									bim.flush();
									bim = null;
								}
								pdfRenderer = null;

								

							} catch (IOException e)
							{
								log.error("Error while empty the variables after thumbnails creation." + e.getMessage());
							}
						}
					}
					else
					{
						log.info("Current file is not a PDF");
					}

				} catch (Exception e)
				{
					log.error("Failded to create thumbnails. Jobnumber: " + jobNumber, e);
				}
			}

		} catch (Exception e)
		{
			log.error("createThumbnailForPDF something went wrong during thumbnails creation.", e.getMessage());
		}
	}

I think that the code is pretty much correct. So the only solution is to increase the memory of the JVM?

Regards,
Paolo


On 2016/08/28 11:30:43, Tilman Hausherr <TH...@t-online.de> wrote: 
> Am 28.08.2016 um 10:53 schrieb Orlev@beeriprint.co.il:
> > looks like MemoryUsageSetting.setupTempFileOnly() has no affect ?
> > do I use it incorrectlly ?
> 
> You're using it correctly. The solution is below:
> 
> >
> > ** I can overcome this problem with -Xmx1500m but had to increase my physical memory.. , and.., I have much bigger files (250000 pages)
> 
> 
> Additionally, if you're doing rendering, take care not to keep the 
> result images in memory.
> 
> 
> Tilman
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: users-help@pdfbox.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org


Re: "OutOfMemoryError: Java heap space" even when using MemoryUsageSetting.setupTempFileOnly()

Posted by John Hewson <jo...@jahewson.com>.
> On 28 Aug 2016, at 04:30, Tilman Hausherr <TH...@t-online.de> wrote:
> 
> Am 28.08.2016 um 10:53 schrieb Orlev@beeriprint.co.il:
>> looks like MemoryUsageSetting.setupTempFileOnly() has no affect ?
>> do I use it incorrectlly ?
> 
> You're using it correctly. The solution is below:
> 
>> 
>> ** I can overcome this problem with -Xmx1500m but had to increase my physical memory.. , and.., I have much bigger files (250000 pages)…
> 
> 
> Additionally, if you're doing rendering, take care not to keep the result images in memory.

Also make sure you’re not creating a List<PDPage> or anything similar.

— John

> Tilman
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: users-help@pdfbox.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org


Re: "OutOfMemoryError: Java heap space" even when using MemoryUsageSetting.setupTempFileOnly()

Posted by Tilman Hausherr <TH...@t-online.de>.
Am 28.08.2016 um 10:53 schrieb Orlev@beeriprint.co.il:
> looks like MemoryUsageSetting.setupTempFileOnly() has no affect ?
> do I use it incorrectlly ?

You're using it correctly. The solution is below:

>
> ** I can overcome this problem with -Xmx1500m but had to increase my physical memory.. , and.., I have much bigger files (250000 pages)


Additionally, if you're doing rendering, take care not to keep the 
result images in memory.


Tilman


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org