You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pdfbox.apache.org by "Poisson, David (DGRI)" <Da...@mrnf.gouv.qc.ca.INVALID> on 2025/08/14 14:24:21 UTC

Getting IOException: expected: 'endstream' actual: '' at offset X

[Intranet logo]
Hi everyone! Sorry for the lengthy description, but I didn't want to leave out crucial details.

In one of our legacy system, we use PDFBox 1.8.12 (on Corretto JRE 11) to load a PDF (converted before reaching this system) and strip the text for processing.
The PDFs received by the legacy system are generated by a separate system that uses a 3rd party library (version 3 is in production).

We are in the process of migrating to a newer version of this library (version 4) and have started doing regression testing.
We have found that about 450 PDF/A documents out of about 11500 test documents fail to be opened by PDFBox 1.8.12.
One such PDF generates the following exception when read:
[multiple stream length is wrong message]
2025-07-25 15:45:40,763 [main] WARN  org.apache.pdfbox.pdfparser.BaseParser - Specified stream length 3686 is wrong. Fall back to reading stream until 'endstream'.
[multiple stream length is wrong message]
java.lang.Throwable: java.io.IOException: expected='endstream' actual='' at offset 93525
        [company internal class stack redacted]
Caused by: java.io.IOException: expected='endstream' actual='' at offset 93525
        at org.apache.pdfbox.pdfparser.BaseParser.parseCOSStream(BaseParser.java:609) ~[pdfbox-1.8.12.jar:1.8.12]
        at org.apache.pdfbox.pdfparser.PDFParser.parseObject(PDFParser.java:650) ~[pdfbox-1.8.12.jar:1.8.12]
        at org.apache.pdfbox.pdfparser.PDFParser.parse(PDFParser.java:203) ~[pdfbox-1.8.12.jar:1.8.12]
        [company internal class stack redacted]
        ... 9 more
The reason why I left the message "Specified stream length 3686 is wrong" in the above output is because the offset 93525 seems to fall in the "/Filter /FlateDecode" object that has a declared /Length of 3686.

I have found the following existing issue that closely ressembles my situation: https://issues.apache.org/jira/browse/PDFBOX-4704

As per issue PDFBOX-4704, I have tried to update to the latest 1.8 version (1.8.17) and replace:
document = PDDocument.load(iStream);

By the following:
RandomAccessBuffer buffer = new RandomAccessBuffer();
document = PDDocument.loadNonSeq(iStream, buffer);

When I try to process the PDF, I get a different exception:
2025-07-29 13:54:05,987 [main] ERROR org.apache.pdfbox.pdfparser.NonSequentialPDFParser - The end of the stream is out of range, using workaround to read the stream
2025-07-29 13:54:05,987 [main] ERROR org.apache.pdfbox.pdfparser.NonSequentialPDFParser - Stream start offset: 92988
2025-07-29 13:54:05,987 [main] ERROR org.apache.pdfbox.pdfparser.NonSequentialPDFParser - Expected endofstream offset: 93714
2025-07-29 13:54:05,991 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
2025-07-29 13:54:05,991 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
2025-07-29 13:54:05,991 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
2025-07-29 13:54:05,991 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
2025-07-29 13:54:05,992 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
2025-07-29 13:54:05,992 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
2025-07-29 13:54:05,992 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
2025-07-29 13:54:05,992 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
2025-07-29 13:54:05,992 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
2025-07-29 13:54:05,992 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
2025-07-29 13:54:05,992 [main] INFO  [internal class] - Exception getMessage(): null
2025-07-29 13:54:05,992 [main] INFO  [internal class]- Exception getCause(): java.util.zip.DataFormatException: too many length or distance symbols
java.lang.Throwable: java.io.IOException
        [internal stack class redacted]
Caused by: java.io.IOException
        at org.apache.pdfbox.filter.FlateFilter.decode(FlateFilter.java:108) ~[pdfbox-1.8.17.jar:1.8.17]
        at org.apache.pdfbox.cos.COSStream.doDecode(COSStream.java:379) ~[pdfbox-1.8.17.jar:1.8.17]
        at org.apache.pdfbox.cos.COSStream.doDecode(COSStream.java:291) ~[pdfbox-1.8.17.jar:1.8.17]
        at org.apache.pdfbox.cos.COSStream.getUnfilteredStream(COSStream.java:225) ~[pdfbox-1.8.17.jar:1.8.17]
        at org.apache.pdfbox.pdfparser.PDFXrefStreamParser.<init>(PDFXrefStreamParser.java:61) ~[pdfbox-1.8.17.jar:1.8.17]
        at org.apache.pdfbox.pdfparser.PDFParser.parseXrefStream(PDFParser.java:976) ~[pdfbox-1.8.17.jar:1.8.17]
        at org.apache.pdfbox.pdfparser.NonSequentialPDFParser.parseXrefObjStream(NonSequentialPDFParser.java:667) ~[pdfbox-1.8.17.jar:1.8.17]
        at org.apache.pdfbox.pdfparser.NonSequentialPDFParser.parseXref(NonSequentialPDFParser.java:621) ~[pdfbox-1.8.17.jar:1.8.17]
        at org.apache.pdfbox.pdfparser.NonSequentialPDFParser.initialParse(NonSequentialPDFParser.java:351) ~[pdfbox-1.8.17.jar:1.8.17]
        at org.apache.pdfbox.pdfparser.NonSequentialPDFParser.parse(NonSequentialPDFParser.java:928) ~[pdfbox-1.8.17.jar:1.8.17]
        at org.apache.pdfbox.pdmodel.PDDocument.loadNonSeq(PDDocument.java:1332) ~[pdfbox-1.8.17.jar:1.8.17]
        at org.apache.pdfbox.pdmodel.PDDocument.loadNonSeq(PDDocument.java:1315) ~[pdfbox-1.8.17.jar:1.8.17]
        [internal class stack redacted]
        ... 11 more

If I open the PDF in Acrobat and save it as a PDF/A, then the resulting PDF file can be opened with PDFBox 1.8.12 without a hitch.

We passed the PDF in a validator and we get the following report both the original file and after exporting in Acrobat (exact same report):
Checking against conformance level PDF/A-1a
Category: Format Message: The file contains cross reference streams. Context: file PageNo: N/A
Category: Metadata Message: The XMP property 'pdfaid:part' has the invalid value '2'. Required is '1'. Context: document metadata PageNo: N/A
Category: Metadata Message: The dictionary must not contain the key 'Filter'. Context: metadata of font file of font 'ABCDEE+CG Times' PageNo: 1
Category: Font Message: The key CIDSet is required but missing. Context: font descriptor of font 'ABCDEE+CG Times' PageNo: 1
False

Checking against conformance level PDF/A-1b
Category: Format Message: The file contains cross reference streams. Context: file PageNo: N/A
Category: Metadata Message: The XMP property 'pdfaid:part' has the invalid value '2'. Required is '1'. Context: document metadata PageNo: N/A
Category: Metadata Message: The dictionary must not contain the key 'Filter'. Context: metadata of font file of font 'ABCDEE+CG Times' PageNo: 1
Category: Font Message: The key CIDSet is required but missing. Context: font descriptor of font 'ABCDEE+CG Times' PageNo: 1
False

Checking against conformance level PDF/A-2a
True

Checking against conformance level PDF/A-2b
True

Checking against conformance level PDF/A-2u
True

Checking against conformance level PDF/A-3a
Category: Metadata Message: The XMP property 'pdfaid:part' has the invalid value '2'. Required is '3'. Context: document metadata PageNo: N/A
False

Checking against conformance level PDF/A-3b
Category: Metadata Message: The XMP property 'pdfaid:part' has the invalid value '2'. Required is '3'. Context: document metadata PageNo: N/A
False

Checking against conformance level PDF/A-3u
Category: Metadata Message: The XMP property 'pdfaid:part' has the invalid value '2'. Required is '3'. Context: document metadata PageNo: N/A
False

Of particular interest are the messages about containing cross reference streams.

Here are the PDF's in question (didn't want to add 3 PDF's to the email, so here's a link to my google drive's folder that has all 3 PDF's):
https://drive.google.com/drive/folders/1Tb136kzA5mMy5R2ti0Cy7UXWT2PQVS5z?usp=sharing
v3.PDF: conversion result using version 3 of our conversion library, works well in PDFBox 1.8.12
v4.PDF: conversion result using version 4 of our conversion library, gives errors in PDFBox
v4-fixedByAcrobat.pdf: v4.PDF opened and exported by Acrobat: works well in PDFBox 1.8.12

I'm running out of ideas of where to look for the problem/solution: Is the generated PDF corrupt or is it a PDFBox bug?

David Poisson

Re: Getting IOException: expected: 'endstream' actual: '' at offset X

Posted by Tilman Hausherr <TH...@t-online.de>.
Am 14.08.2025 um 16:24 schrieb Poisson, David (DGRI):
> RandomAccessBuffer buffer = new RandomAccessBuffer();
> document = PDDocument.loadNonSeq(iStream, buffer);

Made me wonder if there was a bug in using a scratchfile (despite that 
we don't support 1.8 anymore, but I'm curious). So I tried, and also 
used a FileInputStream as input for the PDF, but it worked too. With v3 
and v4.

Tilman


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org

Re: Getting IOException: expected: 'endstream' actual: '' at offset X

Posted by Tilman Hausherr <TH...@t-online.de>.
Am 25.08.2025 um 22:45 schrieb Poisson, David (DGRI):
>          You may disregard my last comment. I copied the wrong file and passed it through our modified legacy system, wrongfully thinking it had solved our problem. But resetting the inputStream had no impact on anything.

I tried your code on the current 1.8 version and added iStream.reset() 
before new PDFParser() and it worked with the v3 and the v4 files. 
Without it, I get this error, which is classic on empty files:

java.io.IOException: Error: End-of-File, expected line
     at 
org.apache.pdfbox.pdfparser.BaseParser.readLine(BaseParser.java:1523)
     at 
org.apache.pdfbox.pdfparser.PDFParser.parseHeader(PDFParser.java:360)
     at org.apache.pdfbox.pdfparser.PDFParser.parse(PDFParser.java:186)
     at 
com.tilman.mavenpdfboxtest18.Mail14_8_2025_Poisson_PDFLoader.openPDFv1(Mail14_8_2025_Poisson_PDFLoader.java:88)
     at 
com.tilman.mavenpdfboxtest18.Mail14_8_2025_Poisson_PDFLoader.main(Mail14_8_2025_Poisson_PDFLoader.java:34)

Tilman


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org

RE: Getting IOException: expected: 'endstream' actual: '' at offset X

Posted by "Poisson, David (DGRI)" <Da...@mrnf.gouv.qc.ca.INVALID>.
[Intranet logo]

Hi Tilman,
        You may disregard my last comment. I copied the wrong file and passed it through our modified legacy system, wrongfully thinking it had solved our problem. But resetting the inputStream had no impact on anything.

David Poisson

-----Message d'origine-----
De : Poisson, David (DGRI) <Da...@mrnf.gouv.qc.ca.INVALID>
Envoyé : 25 août 2025 10:37
À : users@pdfbox.apache.org
Objet : RE: Getting IOException: expected: 'endstream' actual: '' at offset X

[Intranet logo]

Hi Tilman,
        Sorry for the late reply.

I did more digging around. I've added a reset() call to the inputStream before parsing the document and preliminary tests shows all problematic PDF could be opened now without any problems.

I have a feeling that there are some IF-ELSE branches in the code that conditionally reset the input stream on most, but not all, cases.

David Poisson


-----Message d'origine-----
De : Tilman Hausherr <TH...@t-online.de> Envoyé : 15 août 2025 10:13 À : users@pdfbox.apache.org Objet : Re: Getting IOException: expected: 'endstream' actual: '' at offset X

Am 15.08.2025 um 15:57 schrieb Poisson, David (DGRI):
> [Intranet logo]
>
> Hi Tilman, appreciate your reply!
>
> I'm not that familiar with the internal structure of PDF files, so I appreciate the fact that you confirm that they are valid.
> When I use "file" command in Linux, it does report all files are being application/pdf.
>
> I was also able to write a sample program which loads the PDF as per the PDFBox documentation. It works well.
>
> I pulled out all the statements that work with PDFBox objects from our legacy system's code and put it in a self-contained project and I think I can reproduce the problem.
> The input stream is used for 2 passes through the document.
> The first pass goes through all pages and determines the text location on each page.
> The second pass extracts all the text, which is then cleaned up (removing what are in the margins and top/bottom of pages).
>
> When we re-used the input stream in the second pass to create the Parser, that's when we get the error.
>
> I've added an inputStream.reset() in between the two passes in my self-contained project and the error goes away.
> I'm in the process of making the modification in our legacy system and will test it to see if that helps us with the PDF files that cannot be opened.
> What I can't explain though, is why some PDF files are going through without this reset()?

I don't know, and I'm not curious enough to find out why incorrect code sometimes works.

You should be able to work with the same PDDocument object for both passes. That second pass is a bit weird. You don't need to call "new PDFParser(", this is very "old style". If you have a file in your production code, use that one. If you a byte array, use that one directly (in PDFBox 2.0 and 3.0). 3.0 should be faster with a file because it does parse on demand.

Tilman


>
> Not sure what's the best way to share this project? I've put it up on my google drive (where the PDF's were).
> It's a java maven project called PDFLoader.zip (for convenience, I
> have the 3 PDF files at the root of the project)
> https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdriv
> %2F&data=05%7C02%7CDavid.Poisson%40mrnf.gouv.qc.ca%7C8555ce14966b4285d
> b3c08dde3e4f346%7C8705e97737814f4790e1c84c8b884da1%7C0%7C0%7C638917294
> 717007981%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAu
> MDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C60000%7C%7C
> %7C&sdata=DVblZqWrK%2BUysvEm%2FMyturxnI7gMHwD2J6%2BFoIzJlKY%3D&reserve
> d=0
> e.google.com%2Fdrive%2Ffolders%2F1Tb136kzA5mMy5R2ti0Cy7UXWT2PQVS5z%3Fu
> sp%3Dsharing&data=05%7C02%7CDavid.Poisson%40mrnf.gouv.qc.ca%7Ced3da4f5
> 22354429e8a308dddc05f35c%7C8705e97737814f4790e1c84c8b884da1%7C0%7C0%7C
> 638908640312612144%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlY
> iOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%
> 7C%7C%7C&sdata=6D3bUAZsPm2k66xJujRTxbtHhagRFtgJLRKA7YIwnqQ%3D&reserved
> =0 When you run the main(), you'll get an exception with v4.PDF.
> Simply uncomment line 62 and it should work.
>
> David Poisson
>
>
> -----Message d'origine-----
> De : Tilman Hausherr <TH...@t-online.de> Envoyé : 14 août 2025
> 11:59 À : users@pdfbox.apache.org Objet : Re: Getting IOException:
> expected: 'endstream' actual: '' at offset X
>
> Am 14.08.2025 um 16:24 schrieb Poisson, David (DGRI):
>> Here are the PDF's in question (didn't want to add 3 PDF's to the email, so here's a link to my google drive's folder that has all 3 PDF's):
>> https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdri
>> %2F&data=05%7C02%7CDavid.Poisson%40mrnf.gouv.qc.ca%7C8555ce14966b4285
>> db3c08dde3e4f346%7C8705e97737814f4790e1c84c8b884da1%7C0%7C0%7C6389172
>> 94717039104%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwL
>> jAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C60000%7
>> C%7C%7C&sdata=T0SI9ArMQ0qV8RFKsrbAf4dWBB%2Fvz1gyPd1iZK0DX9g%3D&reserv
>> ed=0
>> v%2F&data=05%7C02%7CDavid.Poisson%40mrnf.gouv.qc.ca%7Ced3da4f52235442
>> 9e8a308dddc05f35c%7C8705e97737814f4790e1c84c8b884da1%7C0%7C0%7C638908
>> 640312639920%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIw
>> LjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7
>> C%7C&sdata=tXfkvUpytuDe7B7db93cj%2FIYyQxXDiXKlbuB0Fuwn%2B8%3D&reserve
>> d=0
>> e.google.com%2Fdrive%2Ffolders%2F1Tb136kzA5mMy5R2ti0Cy7UXWT2PQVS5z%3F
>> u
>> sp%3Dsharing&data=05%7C02%7CDavid.Poisson%40mrnf.gouv.qc.ca%7C5f7e76c
>> b
>> 23414628c8f808dddb4b7433%7C8705e97737814f4790e1c84c8b884da1%7C0%7C0%7
>> C
>> 638907839441012283%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIl
>> Y
>> iOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0
>> %
>> 7C%7C%7C&sdata=%2Fjl5Z3feolFod9uH7AhacRPhhmPTJIYpbeMw55POnB8%3D&reser
>> v
>> ed=0
>> v3.PDF: conversion result using version 3 of our conversion library,
>> works well in PDFBox 1.8.12
>> v4.PDF: conversion result using version 4 of our conversion library,
>> gives errors in PDFBox
>> v4-fixedByAcrobat.pdf: v4.PDF opened and exported by Acrobat: works
>> well in PDFBox 1.8.12
> I had no trouble doing a text extraction with 1.8.12, 1.8.16 and
> 1.8.17 on v3 and v4 using pdfbox-app. Makes me wonder if there's
> either a problem with PDFBox when using an input stream, or if
> something goes wrong when you read the file (maybe wrong mime type so
> it's passed as text)
>
> Re the PDF/A problems:
>
> Your file is a (correct) PDF/A-2a, and you checked it to be PDF/A-1b, which it isn't.
>
>      Checking against conformance level PDF/A-2a
>      True
>
>      Checking against conformance level PDF/A-2b
>      True
>
>      Checking against conformance level PDF/A-2u
>      True
>
> That's all you need!
>
> Tilman
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: users-help@pdfbox.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org

RE: Getting IOException: expected: 'endstream' actual: '' at offset X

Posted by "Poisson, David (DGRI)" <Da...@mrnf.gouv.qc.ca.INVALID>.
[Intranet logo]

Hi Tilman,
        Sorry for the late reply.

I did more digging around. I've added a reset() call to the inputStream before parsing the document and preliminary tests shows all problematic PDF could be opened now without any problems.

I have a feeling that there are some IF-ELSE branches in the code that conditionally reset the input stream on most, but not all, cases.

David Poisson


-----Message d'origine-----
De : Tilman Hausherr <TH...@t-online.de>
Envoyé : 15 août 2025 10:13
À : users@pdfbox.apache.org
Objet : Re: Getting IOException: expected: 'endstream' actual: '' at offset X

Am 15.08.2025 um 15:57 schrieb Poisson, David (DGRI):
> [Intranet logo]
>
> Hi Tilman, appreciate your reply!
>
> I'm not that familiar with the internal structure of PDF files, so I appreciate the fact that you confirm that they are valid.
> When I use "file" command in Linux, it does report all files are being application/pdf.
>
> I was also able to write a sample program which loads the PDF as per the PDFBox documentation. It works well.
>
> I pulled out all the statements that work with PDFBox objects from our legacy system's code and put it in a self-contained project and I think I can reproduce the problem.
> The input stream is used for 2 passes through the document.
> The first pass goes through all pages and determines the text location on each page.
> The second pass extracts all the text, which is then cleaned up (removing what are in the margins and top/bottom of pages).
>
> When we re-used the input stream in the second pass to create the Parser, that's when we get the error.
>
> I've added an inputStream.reset() in between the two passes in my self-contained project and the error goes away.
> I'm in the process of making the modification in our legacy system and will test it to see if that helps us with the PDF files that cannot be opened.
> What I can't explain though, is why some PDF files are going through without this reset()?

I don't know, and I'm not curious enough to find out why incorrect code sometimes works.

You should be able to work with the same PDDocument object for both passes. That second pass is a bit weird. You don't need to call "new PDFParser(", this is very "old style". If you have a file in your production code, use that one. If you a byte array, use that one directly (in PDFBox 2.0 and 3.0). 3.0 should be faster with a file because it does parse on demand.

Tilman


>
> Not sure what's the best way to share this project? I've put it up on my google drive (where the PDF's were).
> It's a java maven project called PDFLoader.zip (for convenience, I
> have the 3 PDF files at the root of the project)
> https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdriv
> e.google.com%2Fdrive%2Ffolders%2F1Tb136kzA5mMy5R2ti0Cy7UXWT2PQVS5z%3Fu
> sp%3Dsharing&data=05%7C02%7CDavid.Poisson%40mrnf.gouv.qc.ca%7Ced3da4f5
> 22354429e8a308dddc05f35c%7C8705e97737814f4790e1c84c8b884da1%7C0%7C0%7C
> 638908640312612144%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlY
> iOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%
> 7C%7C%7C&sdata=6D3bUAZsPm2k66xJujRTxbtHhagRFtgJLRKA7YIwnqQ%3D&reserved
> =0 When you run the main(), you'll get an exception with v4.PDF.
> Simply uncomment line 62 and it should work.
>
> David Poisson
>
>
> -----Message d'origine-----
> De : Tilman Hausherr <TH...@t-online.de> Envoyé : 14 août 2025
> 11:59 À : users@pdfbox.apache.org Objet : Re: Getting IOException:
> expected: 'endstream' actual: '' at offset X
>
> Am 14.08.2025 um 16:24 schrieb Poisson, David (DGRI):
>> Here are the PDF's in question (didn't want to add 3 PDF's to the email, so here's a link to my google drive's folder that has all 3 PDF's):
>> https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdri
>> v%2F&data=05%7C02%7CDavid.Poisson%40mrnf.gouv.qc.ca%7Ced3da4f52235442
>> 9e8a308dddc05f35c%7C8705e97737814f4790e1c84c8b884da1%7C0%7C0%7C638908
>> 640312639920%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIw
>> LjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7
>> C%7C&sdata=tXfkvUpytuDe7B7db93cj%2FIYyQxXDiXKlbuB0Fuwn%2B8%3D&reserve
>> d=0
>> e.google.com%2Fdrive%2Ffolders%2F1Tb136kzA5mMy5R2ti0Cy7UXWT2PQVS5z%3F
>> u
>> sp%3Dsharing&data=05%7C02%7CDavid.Poisson%40mrnf.gouv.qc.ca%7C5f7e76c
>> b
>> 23414628c8f808dddb4b7433%7C8705e97737814f4790e1c84c8b884da1%7C0%7C0%7
>> C
>> 638907839441012283%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIl
>> Y
>> iOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0
>> %
>> 7C%7C%7C&sdata=%2Fjl5Z3feolFod9uH7AhacRPhhmPTJIYpbeMw55POnB8%3D&reser
>> v
>> ed=0
>> v3.PDF: conversion result using version 3 of our conversion library,
>> works well in PDFBox 1.8.12
>> v4.PDF: conversion result using version 4 of our conversion library,
>> gives errors in PDFBox
>> v4-fixedByAcrobat.pdf: v4.PDF opened and exported by Acrobat: works
>> well in PDFBox 1.8.12
> I had no trouble doing a text extraction with 1.8.12, 1.8.16 and
> 1.8.17 on v3 and v4 using pdfbox-app. Makes me wonder if there's
> either a problem with PDFBox when using an input stream, or if
> something goes wrong when you read the file (maybe wrong mime type so
> it's passed as text)
>
> Re the PDF/A problems:
>
> Your file is a (correct) PDF/A-2a, and you checked it to be PDF/A-1b, which it isn't.
>
>      Checking against conformance level PDF/A-2a
>      True
>
>      Checking against conformance level PDF/A-2b
>      True
>
>      Checking against conformance level PDF/A-2u
>      True
>
> That's all you need!
>
> Tilman
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: users-help@pdfbox.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org

Re: Getting IOException: expected: 'endstream' actual: '' at offset X

Posted by Tilman Hausherr <TH...@t-online.de>.
Am 15.08.2025 um 15:57 schrieb Poisson, David (DGRI):
> [Intranet logo]
>
> Hi Tilman, appreciate your reply!
>
> I'm not that familiar with the internal structure of PDF files, so I appreciate the fact that you confirm that they are valid.
> When I use "file" command in Linux, it does report all files are being application/pdf.
>
> I was also able to write a sample program which loads the PDF as per the PDFBox documentation. It works well.
>
> I pulled out all the statements that work with PDFBox objects from our legacy system's code and put it in a self-contained project and I think I can reproduce the problem.
> The input stream is used for 2 passes through the document.
> The first pass goes through all pages and determines the text location on each page.
> The second pass extracts all the text, which is then cleaned up (removing what are in the margins and top/bottom of pages).
>
> When we re-used the input stream in the second pass to create the Parser, that's when we get the error.
>
> I've added an inputStream.reset() in between the two passes in my self-contained project and the error goes away.
> I'm in the process of making the modification in our legacy system and will test it to see if that helps us with the PDF files that cannot be opened.
> What I can't explain though, is why some PDF files are going through without this reset()?

I don't know, and I'm not curious enough to find out why incorrect code 
sometimes works.

You should be able to work with the same PDDocument object for both 
passes. That second pass is a bit weird. You don't need to call "new 
PDFParser(", this is very "old style". If you have a file in your 
production code, use that one. If you a byte array, use that one 
directly (in PDFBox 2.0 and 3.0). 3.0 should be faster with a file 
because it does parse on demand.

Tilman


>
> Not sure what's the best way to share this project? I've put it up on my google drive (where the PDF's were).
> It's a java maven project called PDFLoader.zip (for convenience, I have the 3 PDF files at the root of the project)
> https://drive.google.com/drive/folders/1Tb136kzA5mMy5R2ti0Cy7UXWT2PQVS5z?usp=sharing
> When you run the main(), you'll get an exception with v4.PDF. Simply uncomment line 62 and it should work.
>
> David Poisson
>
>
> -----Message d'origine-----
> De : Tilman Hausherr <TH...@t-online.de>
> Envoyé : 14 août 2025 11:59
> À : users@pdfbox.apache.org
> Objet : Re: Getting IOException: expected: 'endstream' actual: '' at offset X
>
> Am 14.08.2025 um 16:24 schrieb Poisson, David (DGRI):
>> Here are the PDF's in question (didn't want to add 3 PDF's to the email, so here's a link to my google drive's folder that has all 3 PDF's):
>> https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdriv
>> e.google.com%2Fdrive%2Ffolders%2F1Tb136kzA5mMy5R2ti0Cy7UXWT2PQVS5z%3Fu
>> sp%3Dsharing&data=05%7C02%7CDavid.Poisson%40mrnf.gouv.qc.ca%7C5f7e76cb
>> 23414628c8f808dddb4b7433%7C8705e97737814f4790e1c84c8b884da1%7C0%7C0%7C
>> 638907839441012283%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlY
>> iOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%
>> 7C%7C%7C&sdata=%2Fjl5Z3feolFod9uH7AhacRPhhmPTJIYpbeMw55POnB8%3D&reserv
>> ed=0
>> v3.PDF: conversion result using version 3 of our conversion library,
>> works well in PDFBox 1.8.12
>> v4.PDF: conversion result using version 4 of our conversion library,
>> gives errors in PDFBox
>> v4-fixedByAcrobat.pdf: v4.PDF opened and exported by Acrobat: works
>> well in PDFBox 1.8.12
> I had no trouble doing a text extraction with 1.8.12, 1.8.16 and 1.8.17 on v3 and v4 using pdfbox-app. Makes me wonder if there's either a problem with PDFBox when using an input stream, or if something goes wrong when you read the file (maybe wrong mime type so it's passed as text)
>
> Re the PDF/A problems:
>
> Your file is a (correct) PDF/A-2a, and you checked it to be PDF/A-1b, which it isn't.
>
>      Checking against conformance level PDF/A-2a
>      True
>
>      Checking against conformance level PDF/A-2b
>      True
>
>      Checking against conformance level PDF/A-2u
>      True
>
> That's all you need!
>
> Tilman
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
> For additional commands, e-mail: users-help@pdfbox.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org

RE: Getting IOException: expected: 'endstream' actual: '' at offset X

Posted by "Poisson, David (DGRI)" <Da...@mrnf.gouv.qc.ca.INVALID>.
[Intranet logo]

Hi Tilman, appreciate your reply!

I'm not that familiar with the internal structure of PDF files, so I appreciate the fact that you confirm that they are valid.
When I use "file" command in Linux, it does report all files are being application/pdf.

I was also able to write a sample program which loads the PDF as per the PDFBox documentation. It works well.

I pulled out all the statements that work with PDFBox objects from our legacy system's code and put it in a self-contained project and I think I can reproduce the problem.
The input stream is used for 2 passes through the document.
The first pass goes through all pages and determines the text location on each page.
The second pass extracts all the text, which is then cleaned up (removing what are in the margins and top/bottom of pages).

When we re-used the input stream in the second pass to create the Parser, that's when we get the error.

I've added an inputStream.reset() in between the two passes in my self-contained project and the error goes away.
I'm in the process of making the modification in our legacy system and will test it to see if that helps us with the PDF files that cannot be opened.
What I can't explain though, is why some PDF files are going through without this reset()?

Not sure what's the best way to share this project? I've put it up on my google drive (where the PDF's were).
It's a java maven project called PDFLoader.zip (for convenience, I have the 3 PDF files at the root of the project)
https://drive.google.com/drive/folders/1Tb136kzA5mMy5R2ti0Cy7UXWT2PQVS5z?usp=sharing
When you run the main(), you'll get an exception with v4.PDF. Simply uncomment line 62 and it should work.

David Poisson


-----Message d'origine-----
De : Tilman Hausherr <TH...@t-online.de>
Envoyé : 14 août 2025 11:59
À : users@pdfbox.apache.org
Objet : Re: Getting IOException: expected: 'endstream' actual: '' at offset X

Am 14.08.2025 um 16:24 schrieb Poisson, David (DGRI):
> Here are the PDF's in question (didn't want to add 3 PDF's to the email, so here's a link to my google drive's folder that has all 3 PDF's):
> https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdriv
> e.google.com%2Fdrive%2Ffolders%2F1Tb136kzA5mMy5R2ti0Cy7UXWT2PQVS5z%3Fu
> sp%3Dsharing&data=05%7C02%7CDavid.Poisson%40mrnf.gouv.qc.ca%7C5f7e76cb
> 23414628c8f808dddb4b7433%7C8705e97737814f4790e1c84c8b884da1%7C0%7C0%7C
> 638907839441012283%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlY
> iOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%
> 7C%7C%7C&sdata=%2Fjl5Z3feolFod9uH7AhacRPhhmPTJIYpbeMw55POnB8%3D&reserv
> ed=0
> v3.PDF: conversion result using version 3 of our conversion library,
> works well in PDFBox 1.8.12
> v4.PDF: conversion result using version 4 of our conversion library,
> gives errors in PDFBox
> v4-fixedByAcrobat.pdf: v4.PDF opened and exported by Acrobat: works
> well in PDFBox 1.8.12

I had no trouble doing a text extraction with 1.8.12, 1.8.16 and 1.8.17 on v3 and v4 using pdfbox-app. Makes me wonder if there's either a problem with PDFBox when using an input stream, or if something goes wrong when you read the file (maybe wrong mime type so it's passed as text)

Re the PDF/A problems:

Your file is a (correct) PDF/A-2a, and you checked it to be PDF/A-1b, which it isn't.

    Checking against conformance level PDF/A-2a
    True

    Checking against conformance level PDF/A-2b
    True

    Checking against conformance level PDF/A-2u
    True

That's all you need!

Tilman

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org

Re: Getting IOException: expected: 'endstream' actual: '' at offset X

Posted by Tilman Hausherr <TH...@t-online.de>.
Am 14.08.2025 um 16:24 schrieb Poisson, David (DGRI):
> Here are the PDF's in question (didn't want to add 3 PDF's to the email, so here's a link to my google drive's folder that has all 3 PDF's):
> https://drive.google.com/drive/folders/1Tb136kzA5mMy5R2ti0Cy7UXWT2PQVS5z?usp=sharing
> v3.PDF: conversion result using version 3 of our conversion library, works well in PDFBox 1.8.12
> v4.PDF: conversion result using version 4 of our conversion library, gives errors in PDFBox
> v4-fixedByAcrobat.pdf: v4.PDF opened and exported by Acrobat: works well in PDFBox 1.8.12

I had no trouble doing a text extraction with 1.8.12, 1.8.16 and 1.8.17 
on v3 and v4 using pdfbox-app. Makes me wonder if there's either a 
problem with PDFBox when using an input stream, or if something goes 
wrong when you read the file (maybe wrong mime type so it's passed as text)

Re the PDF/A problems:

Your file is a (correct) PDF/A-2a, and you checked it to be PDF/A-1b, 
which it isn't.

    Checking against conformance level PDF/A-2a
    True

    Checking against conformance level PDF/A-2b
    True

    Checking against conformance level PDF/A-2u
    True

That's all you need!

Tilman

Re: Getting IOException: expected: 'endstream' actual: '' at offset X

Posted by Andreas Lehmkühler <an...@lehmi.de.INVALID>.
Hi,

you are using an ancient version of PDFBox. 1.8.12 was released in 2016. 
You should update at least to a recent 2.0 version, but the prefered 
version is 3.0.5

Andreas

Am 14.08.25 um 16:24 schrieb Poisson, David (DGRI):
> [Intranet logo]
> Hi everyone! Sorry for the lengthy description, but I didn't want to leave out crucial details.
> 
> In one of our legacy system, we use PDFBox 1.8.12 (on Corretto JRE 11) to load a PDF (converted before reaching this system) and strip the text for processing.
> The PDFs received by the legacy system are generated by a separate system that uses a 3rd party library (version 3 is in production).
> 
> We are in the process of migrating to a newer version of this library (version 4) and have started doing regression testing.
> We have found that about 450 PDF/A documents out of about 11500 test documents fail to be opened by PDFBox 1.8.12.
> One such PDF generates the following exception when read:
> [multiple stream length is wrong message]
> 2025-07-25 15:45:40,763 [main] WARN  org.apache.pdfbox.pdfparser.BaseParser - Specified stream length 3686 is wrong. Fall back to reading stream until 'endstream'.
> [multiple stream length is wrong message]
> java.lang.Throwable: java.io.IOException: expected='endstream' actual='' at offset 93525
>          [company internal class stack redacted]
> Caused by: java.io.IOException: expected='endstream' actual='' at offset 93525
>          at org.apache.pdfbox.pdfparser.BaseParser.parseCOSStream(BaseParser.java:609) ~[pdfbox-1.8.12.jar:1.8.12]
>          at org.apache.pdfbox.pdfparser.PDFParser.parseObject(PDFParser.java:650) ~[pdfbox-1.8.12.jar:1.8.12]
>          at org.apache.pdfbox.pdfparser.PDFParser.parse(PDFParser.java:203) ~[pdfbox-1.8.12.jar:1.8.12]
>          [company internal class stack redacted]
>          ... 9 more
> The reason why I left the message "Specified stream length 3686 is wrong" in the above output is because the offset 93525 seems to fall in the "/Filter /FlateDecode" object that has a declared /Length of 3686.
> 
> I have found the following existing issue that closely ressembles my situation: https://issues.apache.org/jira/browse/PDFBOX-4704
> 
> As per issue PDFBOX-4704, I have tried to update to the latest 1.8 version (1.8.17) and replace:
> document = PDDocument.load(iStream);
> 
> By the following:
> RandomAccessBuffer buffer = new RandomAccessBuffer();
> document = PDDocument.loadNonSeq(iStream, buffer);
> 
> When I try to process the PDF, I get a different exception:
> 2025-07-29 13:54:05,987 [main] ERROR org.apache.pdfbox.pdfparser.NonSequentialPDFParser - The end of the stream is out of range, using workaround to read the stream
> 2025-07-29 13:54:05,987 [main] ERROR org.apache.pdfbox.pdfparser.NonSequentialPDFParser - Stream start offset: 92988
> 2025-07-29 13:54:05,987 [main] ERROR org.apache.pdfbox.pdfparser.NonSequentialPDFParser - Expected endofstream offset: 93714
> 2025-07-29 13:54:05,991 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
> 2025-07-29 13:54:05,991 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
> 2025-07-29 13:54:05,991 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
> 2025-07-29 13:54:05,991 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
> 2025-07-29 13:54:05,992 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
> 2025-07-29 13:54:05,992 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
> 2025-07-29 13:54:05,992 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
> 2025-07-29 13:54:05,992 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
> 2025-07-29 13:54:05,992 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
> 2025-07-29 13:54:05,992 [main] ERROR org.apache.pdfbox.filter.FlateFilter - FlateFilter: stop reading corrupt stream due to a DataFormatException
> 2025-07-29 13:54:05,992 [main] INFO  [internal class] - Exception getMessage(): null
> 2025-07-29 13:54:05,992 [main] INFO  [internal class]- Exception getCause(): java.util.zip.DataFormatException: too many length or distance symbols
> java.lang.Throwable: java.io.IOException
>          [internal stack class redacted]
> Caused by: java.io.IOException
>          at org.apache.pdfbox.filter.FlateFilter.decode(FlateFilter.java:108) ~[pdfbox-1.8.17.jar:1.8.17]
>          at org.apache.pdfbox.cos.COSStream.doDecode(COSStream.java:379) ~[pdfbox-1.8.17.jar:1.8.17]
>          at org.apache.pdfbox.cos.COSStream.doDecode(COSStream.java:291) ~[pdfbox-1.8.17.jar:1.8.17]
>          at org.apache.pdfbox.cos.COSStream.getUnfilteredStream(COSStream.java:225) ~[pdfbox-1.8.17.jar:1.8.17]
>          at org.apache.pdfbox.pdfparser.PDFXrefStreamParser.<init>(PDFXrefStreamParser.java:61) ~[pdfbox-1.8.17.jar:1.8.17]
>          at org.apache.pdfbox.pdfparser.PDFParser.parseXrefStream(PDFParser.java:976) ~[pdfbox-1.8.17.jar:1.8.17]
>          at org.apache.pdfbox.pdfparser.NonSequentialPDFParser.parseXrefObjStream(NonSequentialPDFParser.java:667) ~[pdfbox-1.8.17.jar:1.8.17]
>          at org.apache.pdfbox.pdfparser.NonSequentialPDFParser.parseXref(NonSequentialPDFParser.java:621) ~[pdfbox-1.8.17.jar:1.8.17]
>          at org.apache.pdfbox.pdfparser.NonSequentialPDFParser.initialParse(NonSequentialPDFParser.java:351) ~[pdfbox-1.8.17.jar:1.8.17]
>          at org.apache.pdfbox.pdfparser.NonSequentialPDFParser.parse(NonSequentialPDFParser.java:928) ~[pdfbox-1.8.17.jar:1.8.17]
>          at org.apache.pdfbox.pdmodel.PDDocument.loadNonSeq(PDDocument.java:1332) ~[pdfbox-1.8.17.jar:1.8.17]
>          at org.apache.pdfbox.pdmodel.PDDocument.loadNonSeq(PDDocument.java:1315) ~[pdfbox-1.8.17.jar:1.8.17]
>          [internal class stack redacted]
>          ... 11 more
> 
> If I open the PDF in Acrobat and save it as a PDF/A, then the resulting PDF file can be opened with PDFBox 1.8.12 without a hitch.
> 
> We passed the PDF in a validator and we get the following report both the original file and after exporting in Acrobat (exact same report):
> Checking against conformance level PDF/A-1a
> Category: Format Message: The file contains cross reference streams. Context: file PageNo: N/A
> Category: Metadata Message: The XMP property 'pdfaid:part' has the invalid value '2'. Required is '1'. Context: document metadata PageNo: N/A
> Category: Metadata Message: The dictionary must not contain the key 'Filter'. Context: metadata of font file of font 'ABCDEE+CG Times' PageNo: 1
> Category: Font Message: The key CIDSet is required but missing. Context: font descriptor of font 'ABCDEE+CG Times' PageNo: 1
> False
> 
> Checking against conformance level PDF/A-1b
> Category: Format Message: The file contains cross reference streams. Context: file PageNo: N/A
> Category: Metadata Message: The XMP property 'pdfaid:part' has the invalid value '2'. Required is '1'. Context: document metadata PageNo: N/A
> Category: Metadata Message: The dictionary must not contain the key 'Filter'. Context: metadata of font file of font 'ABCDEE+CG Times' PageNo: 1
> Category: Font Message: The key CIDSet is required but missing. Context: font descriptor of font 'ABCDEE+CG Times' PageNo: 1
> False
> 
> Checking against conformance level PDF/A-2a
> True
> 
> Checking against conformance level PDF/A-2b
> True
> 
> Checking against conformance level PDF/A-2u
> True
> 
> Checking against conformance level PDF/A-3a
> Category: Metadata Message: The XMP property 'pdfaid:part' has the invalid value '2'. Required is '3'. Context: document metadata PageNo: N/A
> False
> 
> Checking against conformance level PDF/A-3b
> Category: Metadata Message: The XMP property 'pdfaid:part' has the invalid value '2'. Required is '3'. Context: document metadata PageNo: N/A
> False
> 
> Checking against conformance level PDF/A-3u
> Category: Metadata Message: The XMP property 'pdfaid:part' has the invalid value '2'. Required is '3'. Context: document metadata PageNo: N/A
> False
> 
> Of particular interest are the messages about containing cross reference streams.
> 
> Here are the PDF's in question (didn't want to add 3 PDF's to the email, so here's a link to my google drive's folder that has all 3 PDF's):
> https://drive.google.com/drive/folders/1Tb136kzA5mMy5R2ti0Cy7UXWT2PQVS5z?usp=sharing
> v3.PDF: conversion result using version 3 of our conversion library, works well in PDFBox 1.8.12
> v4.PDF: conversion result using version 4 of our conversion library, gives errors in PDFBox
> v4-fixedByAcrobat.pdf: v4.PDF opened and exported by Acrobat: works well in PDFBox 1.8.12
> 
> I'm running out of ideas of where to look for the problem/solution: Is the generated PDF corrupt or is it a PDFBox bug?
> 
> David Poisson
> 
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@pdfbox.apache.org
For additional commands, e-mail: users-help@pdfbox.apache.org