You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pdfbox.apache.org by Gabriel <ga...@bry.com.br> on 2016/09/01 18:39:13 UTC
Trouble loading large files with PDFBox 2.0.2
Hello there.
We are in the process of migrating our application between versions 1.8.12
to 2.0.2 and bumped into a somewhat serious problem for our application.
In version 1.8.12 we were using the method load(InputStream, RandomAccess)
for our digital signature process.
With 2.0.2 we began to use the load(InputStream, MemoryUsageSetting) with
setupTempFileOnly configuration.
For small files we obtained a similar performance, but when we moved the
tests to 1MB and 10MB files, it dropped dramatically.
Analyzing via JProfiler was possible to detect the "load" method as the
bottleneck.
We have isolated the method in a test to get some performance metrics, which
I provide in the link in addition to the source code used:
https://drive.google.com/open?id=0B2hR5R90BDFQU1VwOXNlUmJoTnc
Unfortunately I cannot provide the document used, mut it is a 10MB PDF.
Is there something we're doing wrong, or we're not doing that could help
improve the performance?
Thank you very much for you time.
Atenciosamente,
Gabriel Pessoa
Analista
BRy Tecnologia
Rua Lauro Linhares, 2123 Torre B - 3 andar
88036-002 - Florianópolis - SC - Brasil
+55 (48) 3234 6696
Re: RES: RES: Trouble loading large files with PDFBox 2.0.2
Posted by Tilman Hausherr <TH...@t-online.de>.
More huge files here:
http://ims.er.usgs.gov/images/USTopo/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
RES: RES: Trouble loading large files with PDFBox 2.0.2
Posted by Gabriel <ga...@bry.com.br>.
Thank you very much, I'll do it and come back with a reply.
-----Mensagem original-----
De: Tilman Hausherr [mailto:THausherr@t-online.de]
Enviada em: sábado, 3 de setembro de 2016 12:28
Para: users@pdfbox.apache.org
Assunto: Re: RES: Trouble loading large files with PDFBox 2.0.2
Am 02.09.2016 um 17:15 schrieb Tilman Hausherr:
>
> Maybe compare with loadNonSeq in the 1.8 version. The 2.0 parser is
> different than the old parser, and maybe slower because it has to go
> back and forth.
>
> You said your file is confidential. What you could do, if you have
> some time to kill, go here:
> http://digitalcorpora.org/corp/files/govdocs1/zipfiles/
>
> try to find some PDF file that has this effect.
Here are some large ffiles from there, all are larger than 15MB. To get one, download the file from the link above that starts with the first 3 numbers. I.e. to get 475419.pdf, you need to download
http://digitalcorpora.org/corp/files/govdocs1/zipfiles/475.zip
Then do your test and if you think it is much slower, mention the file here.
475419.pdf
620038.pdf
302439.pdf
209086.pdf
755045.pdf
503657.pdf
767115.pdf
942416.pdf
364591.pdf
240242.pdf
574442.pdf
560466.pdf
134823.pdf
234570.pdf
071300.pdf
884613.pdf
022391.pdf
160655.pdf
898927.pdf
509787.pdf
125112.pdf
486395.pdf
510488.pdf
586504.pdf
510944.pdf
861355.pdf
765184.pdf
659153.pdf
173325.pdf
577932.pdf
990065.pdf
660170.pdf
660166.pdf
584460.pdf
979241.pdf
483173.pdf
078656.pdf
296771.pdf
434841.pdf
922401.pdf
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
Re: RES: Trouble loading large files with PDFBox 2.0.2
Posted by Tilman Hausherr <TH...@t-online.de>.
Am 02.09.2016 um 17:15 schrieb Tilman Hausherr:
>
> Maybe compare with loadNonSeq in the 1.8 version. The 2.0 parser is
> different than the old parser, and maybe slower because it has to go
> back and forth.
>
> You said your file is confidential. What you could do, if you have
> some time to kill, go here:
> http://digitalcorpora.org/corp/files/govdocs1/zipfiles/
>
> try to find some PDF file that has this effect.
Here are some large ffiles from there, all are larger than 15MB. To get
one, download the file from the link above that starts with the first 3
numbers. I.e. to get 475419.pdf, you need to download
http://digitalcorpora.org/corp/files/govdocs1/zipfiles/475.zip
Then do your test and if you think it is much slower, mention the file here.
475419.pdf
620038.pdf
302439.pdf
209086.pdf
755045.pdf
503657.pdf
767115.pdf
942416.pdf
364591.pdf
240242.pdf
574442.pdf
560466.pdf
134823.pdf
234570.pdf
071300.pdf
884613.pdf
022391.pdf
160655.pdf
898927.pdf
509787.pdf
125112.pdf
486395.pdf
510488.pdf
586504.pdf
510944.pdf
861355.pdf
765184.pdf
659153.pdf
173325.pdf
577932.pdf
990065.pdf
660170.pdf
660166.pdf
584460.pdf
979241.pdf
483173.pdf
078656.pdf
296771.pdf
434841.pdf
922401.pdf
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
Re: RES: Trouble loading large files with PDFBox 2.0.2
Posted by Tilman Hausherr <TH...@t-online.de>.
Am 02.09.2016 um 13:30 schrieb Gabriel:
> Just tried with both, setupMainMemoryOnly (which is the same as just load(File)) and setupTempFileOnly, and it got a few seconds better with temp files.
>
> load(File, MemoryUsageSetting.setupMainMemoryOnly()): 165641ms
> load(File, MemoryUsageSetting.setupTempFileOnly()): 192022ms (opposed to 210154ms)
>
> But that still is 6 times slower than 1.8.12.
Maybe compare with loadNonSeq in the 1.8 version. The 2.0 parser is
different than the old parser, and maybe slower because it has to go
back and forth.
You said your file is confidential. What you could do, if you have some
time to kill, go here:
http://digitalcorpora.org/corp/files/govdocs1/zipfiles/
try to find some PDF file that has this effect.
Tilman
>
> -----Mensagem original-----
> De: John Hewson [mailto:john@jahewson.com]
> Enviada em: quinta-feira, 1 de setembro de 2016 16:50
> Para: users@pdfbox.apache.org
> Assunto: Re: Trouble loading large files with PDFBox 2.0.2
>
>
> (...)
>> Is there something we're doing wrong, or we're not doing that could
>> help improve the performance?
> I\u2019d expect the fastest to be:
>
> PDDocument load(File file)
>
> \u2014 John
>
>> Thank you very much for you time.
>>
>>
>>
>> Atenciosamente,
>>
>> Gabriel Pessoa
>> Analista
>>
>> BRy Tecnologia
>> Rua Lauro Linhares, 2123 Torre B - 3 andar
>> 88036-002 - Florian�polis - SC - Brasil
>> +55 (48) 3234 6696
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: users-help@pdfbox.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: users-help@pdfbox.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
RES: Trouble loading large files with PDFBox 2.0.2
Posted by Gabriel <ga...@bry.com.br>.
Just tried with both, setupMainMemoryOnly (which is the same as just load(File)) and setupTempFileOnly, and it got a few seconds better with temp files.
load(File, MemoryUsageSetting.setupMainMemoryOnly()): 165641ms
load(File, MemoryUsageSetting.setupTempFileOnly()): 192022ms (opposed to 210154ms)
But that still is 6 times slower than 1.8.12.
-----Mensagem original-----
De: John Hewson [mailto:john@jahewson.com]
Enviada em: quinta-feira, 1 de setembro de 2016 16:50
Para: users@pdfbox.apache.org
Assunto: Re: Trouble loading large files with PDFBox 2.0.2
(...)
> Is there something we're doing wrong, or we're not doing that could
> help improve the performance?
I’d expect the fastest to be:
PDDocument load(File file)
— John
>
> Thank you very much for you time.
>
>
>
> Atenciosamente,
>
> Gabriel Pessoa
> Analista
>
> BRy Tecnologia
> Rua Lauro Linhares, 2123 Torre B - 3 andar
> 88036-002 - Florianópolis - SC - Brasil
> +55 (48) 3234 6696
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org
Re: Trouble loading large files with PDFBox 2.0.2
Posted by John Hewson <jo...@jahewson.com>.
> On 1 Sep 2016, at 11:39, Gabriel <ga...@bry.com.br> wrote:
>
> Hello there.
>
>
>
> We are in the process of migrating our application between versions 1.8.12
> to 2.0.2 and bumped into a somewhat serious problem for our application.
>
>
>
> In version 1.8.12 we were using the method load(InputStream, RandomAccess)
> for our digital signature process.
>
>
>
> With 2.0.2 we began to use the load(InputStream, MemoryUsageSetting) with
> setupTempFileOnly configuration.
>
>
>
> For small files we obtained a similar performance, but when we moved the
> tests to 1MB and 10MB files, it dropped dramatically.
>
>
>
> Analyzing via JProfiler was possible to detect the "load" method as the
> bottleneck.
>
>
>
> We have isolated the method in a test to get some performance metrics, which
> I provide in the link in addition to the source code used:
> https://drive.google.com/open?id=0B2hR5R90BDFQU1VwOXNlUmJoTnc
>
>
>
> Unfortunately I cannot provide the document used, mut it is a 10MB PDF.
>
>
>
> Is there something we're doing wrong, or we're not doing that could help
> improve the performance?
I’d expect the fastest to be:
PDDocument load(File file)
— John
>
> Thank you very much for you time.
>
>
>
> Atenciosamente,
>
> Gabriel Pessoa
> Analista
>
> BRy Tecnologia
> Rua Lauro Linhares, 2123 Torre B - 3 andar
> 88036-002 - Florianópolis - SC - Brasil
> +55 (48) 3234 6696
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org