You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@poi.apache.org by David North <dn...@apache.org> on 2016/08/10 22:47:09 UTC
[VOTE] Apache POI 3.15-beta3
Greetings all,
I hereby call the vote on the release of Apache POI 3.15-beta3
The release artifacts are here:
https://dist.apache.org/repos/dist/dev/poi/3.15-beta3-RC1/
The vote runs for 72 hours from now, ending 23:49 BST Saturday 13 August
2016.
Usual things need to be checked - does it work, does the packaging look
right.
I'll add my +1 once I've checked this build against our POI-using
systems at work.
Thanks,
--
David North | www.dnorth.net
RE: [VOTE] Apache POI 3.15-beta3
Posted by "Allison, Timothy B." <ta...@mitre.org>.
Not -1 worthy, but should we include the maven subdirectory in the src release so that people can easily run the maven-poms task?
From: Dominik Stadler [mailto:dominik.stadler@gmx.at]
Sent: Thursday, August 11, 2016 8:19 AM
To: POI Developers List <de...@poi.apache.org>
Subject: Re: [VOTE] Apache POI 3.15-beta3
Hi,
Unfortunately I have to vote -1 on this candidate, there is a directory "trunk" in the src-packages which seems to contain the full source again, thus making the source packages 150MB instead of 70 as before.
[Inline image 1]
[Inline image 2]
Thanks... Dominik.
On Thu, Aug 11, 2016 at 12:47 AM, David North <dn...@apache.org>> wrote:
Greetings all,
I hereby call the vote on the release of Apache POI 3.15-beta3
The release artifacts are here:
https://dist.apache.org/repos/dist/dev/poi/3.15-beta3-RC1/
The vote runs for 72 hours from now, ending 23:49 BST Saturday 13 August
2016.
Usual things need to be checked - does it work, does the packaging look
right.
I'll add my +1 once I've checked this build against our POI-using
systems at work.
Thanks,
--
David North | www.dnorth.net<http://www.dnorth.net>
Re: [VOTE] Apache POI 3.15-beta3
Posted by David North <dn...@apache.org>.
Starting now, apologies for the delay.
David
On 21/08/16 12:32, Andreas Beeker wrote:
> Hi David,
>
> what's the status? should I jump in?
>
> Andi
>
> On 19.08.2016 10:30, David North wrote:
>> Should be able to sort it this weekend, yes.
>>
>> Thanks,
>> David
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
> For additional commands, e-mail: dev-help@poi.apache.org
>
--
David North - Committer and PMC Member, Apache POI
https://home.apache.org/~dnorth/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
Re: [VOTE] Apache POI 3.15-beta3
Posted by Andreas Beeker <ki...@apache.org>.
Hi David,
what's the status? should I jump in?
Andi
On 19.08.2016 10:30, David North wrote:
> Should be able to sort it this weekend, yes.
>
> Thanks,
> David
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
Re: [VOTE] Apache POI 3.15-beta3
Posted by David North <dt...@corefiling.co.uk>.
Should be able to sort it this weekend, yes.
Thanks,
David
On 17/08/16 20:32, Dominik Stadler wrote:
> Hi,
>
> As the anniversary of POI is getting near and it would be nice to have at
> least a 3.15 release for our 15th birthday, I suggest we build a 3.15-RC1
> from latest trunk so we include the two regressions that Javen fixed and
> the additional source-directory removed.
>
> Then we might just be able to finish in time!
>
> David, are you ok to run the release-steps a 2nd time?
>
> Thanks... Dominik.
>
>
> On Tue, Aug 16, 2016 at 11:45 PM, Javen O'Neal <ja...@gmail.com> wrote:
>
>> Fixed in r1756552.
>>
>> http://svn.apache.org/viewvc?view=revision&revision=1756552
>>
>> On Aug 15, 2016 9:09 AM, "Javen O'Neal" <ja...@gmail.com> wrote:
>>
>>>> * 4 times NullPointerException in XSLFTextParagraph.
>> getDefaultFontSize()
>>> I opened bug 60005 [1] to fix the NPE in XSLFTextParagraph.
>>> getDefaultFontSize()
>>> This has been fixed.
>>>
>>>> * 456 times: ArrayIndexOutOfBoundsException in
>> SprmOperation.getOperand()
>>> The assumption that the operand length is "surely shorter than an int"
>>> seems to incorrect for some files, since the test failed with AIOOB 4,
>>> meaning the operandLength is at least 5 bytes.
>>> I would need to look at the HWPF document specification ([MS-DOC].pdf,
>>> [3]) to see if the operandLength may be longer than an int.
>>> I do not know what the best way to fix this code is
>>>
>>> // surely shorter than an int...
>>> byte operandLength = _grpprl[_gOffset + 1];
>>>
>>> // initialized to zeros by JVM
>>> byte[] codeBytes = new byte[LittleEndian.INT_SIZE];
>>> for ( int i = 0; i < operandLength; i++ )
>>> if ( _gOffset + i < _grpprl.length )
>>> codeBytes[i] = _grpprl[_gOffset + 1 + i];
>>>
>>> return LittleEndian.getInt( codeBytes, 0 );
>>>
>>>
>>> [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=60005
>>> [2] https://svn.apache.org/viewvc/poi/trunk/src/scratchpad/src/
>>> org/apache/poi/hwpf/sprm/SprmOperation.java?revision=
>>> 1753052&view=markup#l106
>>> [3] https://interoperability.blob.core.windows.net/files/
>>> OfficeFileFormatsProtocols.zip
>>>
>>> On Mon, Aug 15, 2016 at 3:09 AM, Dominik Stadler <dominik.stadler@gmx.at
>>>
>>> wrote:
>>>> Hi,
>>>>
>>>> Running the regression tests for POI 3.15-beta3 against the CommonCrawl
>>>> corpus is now finished, initial results are as follows:
>>>>
>>>> * 11966 fail because I did not add commons-collections4, I'll trigger a
>>>> re-run to get document-counts correctly show the number of regressing
>>>> documents
>>>>
>>>> * 456 times: ArrayIndexOutOfBoundsException in
>> SprmOperation.getOperand()
>>>>
>>>> java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException:
>> *
>>>> at o.a.p.hwpf.extractor.WordExtractor.getText(
>>> WordExtractor.java:317)
>>>> at o.a.p.stress.AbstractFileHandler.handleExtractingInternal(
>>> AbstractFileHandler.java:85)
>>>> at o.a.p.stress.AbstractFileHandler.handleExtracting(
>>> AbstractFileHandler.java:60)
>>>> at org.dstadler.commoncrawl.FileHandlingRunnable.run(
>>> FileHandlingRunnable.java:58)
>>>>
>>>> Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
>>>> at o.a.p.hwpf.sprm.SprmOperation.
>> getOperand(SprmOperation.java:
>>> 113)
>>>> at o.a.p.hwpf.sprm.SectionSprmUncompressor.
>>> unCompressSEPOperation(SectionSprmUncompressor.java:62)
>>>> at o.a.p.hwpf.sprm.SectionSprmUncompressor.uncompressSEP(
>>> SectionSprmUncompressor.java:44)
>>>> at o.a.p.hwpf.model.SEPX.getSectionProperties(SEPX.java:61)
>>>> at o.a.p.hwpf.usermodel.Section.(Section.java:36)
>>>> at o.a.p.hwpf.usermodel.Range.getSection(Range.java:745)
>>>> at o.a.p.hwpf.converter.AbstractWordConverter.processDocument(
>>> AbstractWordConverter.java:721)
>>>> at o.a.p.hwpf.extractor.WordExtractor.getText(
>>> WordExtractor.java:299)
>>>> ... 9 more
>>>>
>>>> * 4 times NullPointerException in XSLFTextParagraph.
>> getDefaultFontSize()
>>>>
>>>> java.lang.NullPointerException
>>>> at o.a.p.xslf.usermodel.XSLFTextParagraph.getDefaultFontSize(
>>> XSLFTextParagraph.java:935)
>>>> at o.a.p.sl.draw.DrawTextParagraph.getAttributedString(
>>> DrawTextParagraph.java:567)
>>>> at o.a.p.sl.draw.DrawTextParagraph.breakText(
>>> DrawTextParagraph.java:235)
>>>> at o.a.p.sl.draw.DrawTextShape.drawParagraphs(DrawTextShape.
>>> java:158)
>>>> at o.a.p.sl.draw.DrawTextShape.getTextHeight(DrawTextShape.
>>> java:219)
>>>> at o.a.p.sl.draw.DrawTextShape.drawContent(DrawTextShape.
>>> java:102)
>>>> at o.a.p.sl.draw.DrawSimpleShape.draw(DrawSimpleShape.java:93)
>>>> at o.a.p.sl.draw.DrawSheet.draw(DrawSheet.java:67)
>>>> at o.a.p.sl.draw.DrawSlide.draw(DrawSlide.java:39)
>>>> at o.a.p.xslf.usermodel.XSLFSlide.draw(XSLFSlide.java:301)
>>>> at o.a.p.stress.SlideShowHandler.
>> renderSlides(SlideShowHandler.
>>> java:120)
>>>> at o.a.p.stress.SlideShowHandler.handleSlideShow(
>>> SlideShowHandler.java:43)
>>>> at o.a.p.stress.XSLFFileHandler.handleFile(XSLFFileHandler.
>>> java:43)
>>>> at org.dstadler.commoncrawl.FileHandlingRunnable.run(
>>> FileHandlingRunnable.java:58)
>>>>
>>>>
>>>>
>>>> The others are probably flaky things where files caused OOM/Timeout
>>> before
>>>> and thus were not reported with these errors before.
>>>>
>>>>
>>>> See http://people.apache.org/~centic/poi_regression/reports/ and
>>>> http://people.apache.org/~centic/poi_regression/reportsAll/ for
>> detailed
>>>> results.
>>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
Re: [VOTE] Apache POI 3.15-beta3
Posted by Dominik Stadler <do...@gmx.at>.
Hi,
As the anniversary of POI is getting near and it would be nice to have at
least a 3.15 release for our 15th birthday, I suggest we build a 3.15-RC1
from latest trunk so we include the two regressions that Javen fixed and
the additional source-directory removed.
Then we might just be able to finish in time!
David, are you ok to run the release-steps a 2nd time?
Thanks... Dominik.
On Tue, Aug 16, 2016 at 11:45 PM, Javen O'Neal <ja...@gmail.com> wrote:
> Fixed in r1756552.
>
> http://svn.apache.org/viewvc?view=revision&revision=1756552
>
> On Aug 15, 2016 9:09 AM, "Javen O'Neal" <ja...@gmail.com> wrote:
>
> > > * 4 times NullPointerException in XSLFTextParagraph.
> getDefaultFontSize()
> > I opened bug 60005 [1] to fix the NPE in XSLFTextParagraph.
> > getDefaultFontSize()
> > This has been fixed.
> >
> > > * 456 times: ArrayIndexOutOfBoundsException in
> SprmOperation.getOperand()
> > The assumption that the operand length is "surely shorter than an int"
> > seems to incorrect for some files, since the test failed with AIOOB 4,
> > meaning the operandLength is at least 5 bytes.
> > I would need to look at the HWPF document specification ([MS-DOC].pdf,
> > [3]) to see if the operandLength may be longer than an int.
> > I do not know what the best way to fix this code is
> >
> > // surely shorter than an int...
> > byte operandLength = _grpprl[_gOffset + 1];
> >
> > // initialized to zeros by JVM
> > byte[] codeBytes = new byte[LittleEndian.INT_SIZE];
> > for ( int i = 0; i < operandLength; i++ )
> > if ( _gOffset + i < _grpprl.length )
> > codeBytes[i] = _grpprl[_gOffset + 1 + i];
> >
> > return LittleEndian.getInt( codeBytes, 0 );
> >
> >
> > [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=60005
> > [2] https://svn.apache.org/viewvc/poi/trunk/src/scratchpad/src/
> > org/apache/poi/hwpf/sprm/SprmOperation.java?revision=
> > 1753052&view=markup#l106
> > [3] https://interoperability.blob.core.windows.net/files/
> > OfficeFileFormatsProtocols.zip
> >
> > On Mon, Aug 15, 2016 at 3:09 AM, Dominik Stadler <dominik.stadler@gmx.at
> >
> > wrote:
> > > Hi,
> > >
> > > Running the regression tests for POI 3.15-beta3 against the CommonCrawl
> > > corpus is now finished, initial results are as follows:
> > >
> > > * 11966 fail because I did not add commons-collections4, I'll trigger a
> > > re-run to get document-counts correctly show the number of regressing
> > > documents
> > >
> > > * 456 times: ArrayIndexOutOfBoundsException in
> SprmOperation.getOperand()
> > >
> > > java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException:
> *
> > > at o.a.p.hwpf.extractor.WordExtractor.getText(
> > WordExtractor.java:317)
> > > at o.a.p.stress.AbstractFileHandler.handleExtractingInternal(
> > AbstractFileHandler.java:85)
> > > at o.a.p.stress.AbstractFileHandler.handleExtracting(
> > AbstractFileHandler.java:60)
> > > at org.dstadler.commoncrawl.FileHandlingRunnable.run(
> > FileHandlingRunnable.java:58)
> > >
> > > Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
> > > at o.a.p.hwpf.sprm.SprmOperation.
> getOperand(SprmOperation.java:
> > 113)
> > > at o.a.p.hwpf.sprm.SectionSprmUncompressor.
> > unCompressSEPOperation(SectionSprmUncompressor.java:62)
> > > at o.a.p.hwpf.sprm.SectionSprmUncompressor.uncompressSEP(
> > SectionSprmUncompressor.java:44)
> > > at o.a.p.hwpf.model.SEPX.getSectionProperties(SEPX.java:61)
> > > at o.a.p.hwpf.usermodel.Section.(Section.java:36)
> > > at o.a.p.hwpf.usermodel.Range.getSection(Range.java:745)
> > > at o.a.p.hwpf.converter.AbstractWordConverter.processDocument(
> > AbstractWordConverter.java:721)
> > > at o.a.p.hwpf.extractor.WordExtractor.getText(
> > WordExtractor.java:299)
> > > ... 9 more
> > >
> > > * 4 times NullPointerException in XSLFTextParagraph.
> getDefaultFontSize()
> > >
> > > java.lang.NullPointerException
> > > at o.a.p.xslf.usermodel.XSLFTextParagraph.getDefaultFontSize(
> > XSLFTextParagraph.java:935)
> > > at o.a.p.sl.draw.DrawTextParagraph.getAttributedString(
> > DrawTextParagraph.java:567)
> > > at o.a.p.sl.draw.DrawTextParagraph.breakText(
> > DrawTextParagraph.java:235)
> > > at o.a.p.sl.draw.DrawTextShape.drawParagraphs(DrawTextShape.
> > java:158)
> > > at o.a.p.sl.draw.DrawTextShape.getTextHeight(DrawTextShape.
> > java:219)
> > > at o.a.p.sl.draw.DrawTextShape.drawContent(DrawTextShape.
> > java:102)
> > > at o.a.p.sl.draw.DrawSimpleShape.draw(DrawSimpleShape.java:93)
> > > at o.a.p.sl.draw.DrawSheet.draw(DrawSheet.java:67)
> > > at o.a.p.sl.draw.DrawSlide.draw(DrawSlide.java:39)
> > > at o.a.p.xslf.usermodel.XSLFSlide.draw(XSLFSlide.java:301)
> > > at o.a.p.stress.SlideShowHandler.
> renderSlides(SlideShowHandler.
> > java:120)
> > > at o.a.p.stress.SlideShowHandler.handleSlideShow(
> > SlideShowHandler.java:43)
> > > at o.a.p.stress.XSLFFileHandler.handleFile(XSLFFileHandler.
> > java:43)
> > > at org.dstadler.commoncrawl.FileHandlingRunnable.run(
> > FileHandlingRunnable.java:58)
> > >
> > >
> > >
> > > The others are probably flaky things where files caused OOM/Timeout
> > before
> > > and thus were not reported with these errors before.
> > >
> > >
> > > See http://people.apache.org/~centic/poi_regression/reports/ and
> > > http://people.apache.org/~centic/poi_regression/reportsAll/ for
> detailed
> > > results.
> >
>
Re: [VOTE] Apache POI 3.15-beta3
Posted by Javen O'Neal <ja...@gmail.com>.
Fixed in r1756552.
http://svn.apache.org/viewvc?view=revision&revision=1756552
On Aug 15, 2016 9:09 AM, "Javen O'Neal" <ja...@gmail.com> wrote:
> > * 4 times NullPointerException in XSLFTextParagraph.getDefaultFontSize()
> I opened bug 60005 [1] to fix the NPE in XSLFTextParagraph.
> getDefaultFontSize()
> This has been fixed.
>
> > * 456 times: ArrayIndexOutOfBoundsException in SprmOperation.getOperand()
> The assumption that the operand length is "surely shorter than an int"
> seems to incorrect for some files, since the test failed with AIOOB 4,
> meaning the operandLength is at least 5 bytes.
> I would need to look at the HWPF document specification ([MS-DOC].pdf,
> [3]) to see if the operandLength may be longer than an int.
> I do not know what the best way to fix this code is
>
> // surely shorter than an int...
> byte operandLength = _grpprl[_gOffset + 1];
>
> // initialized to zeros by JVM
> byte[] codeBytes = new byte[LittleEndian.INT_SIZE];
> for ( int i = 0; i < operandLength; i++ )
> if ( _gOffset + i < _grpprl.length )
> codeBytes[i] = _grpprl[_gOffset + 1 + i];
>
> return LittleEndian.getInt( codeBytes, 0 );
>
>
> [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=60005
> [2] https://svn.apache.org/viewvc/poi/trunk/src/scratchpad/src/
> org/apache/poi/hwpf/sprm/SprmOperation.java?revision=
> 1753052&view=markup#l106
> [3] https://interoperability.blob.core.windows.net/files/
> OfficeFileFormatsProtocols.zip
>
> On Mon, Aug 15, 2016 at 3:09 AM, Dominik Stadler <do...@gmx.at>
> wrote:
> > Hi,
> >
> > Running the regression tests for POI 3.15-beta3 against the CommonCrawl
> > corpus is now finished, initial results are as follows:
> >
> > * 11966 fail because I did not add commons-collections4, I'll trigger a
> > re-run to get document-counts correctly show the number of regressing
> > documents
> >
> > * 456 times: ArrayIndexOutOfBoundsException in SprmOperation.getOperand()
> >
> > java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: *
> > at o.a.p.hwpf.extractor.WordExtractor.getText(
> WordExtractor.java:317)
> > at o.a.p.stress.AbstractFileHandler.handleExtractingInternal(
> AbstractFileHandler.java:85)
> > at o.a.p.stress.AbstractFileHandler.handleExtracting(
> AbstractFileHandler.java:60)
> > at org.dstadler.commoncrawl.FileHandlingRunnable.run(
> FileHandlingRunnable.java:58)
> >
> > Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
> > at o.a.p.hwpf.sprm.SprmOperation.getOperand(SprmOperation.java:
> 113)
> > at o.a.p.hwpf.sprm.SectionSprmUncompressor.
> unCompressSEPOperation(SectionSprmUncompressor.java:62)
> > at o.a.p.hwpf.sprm.SectionSprmUncompressor.uncompressSEP(
> SectionSprmUncompressor.java:44)
> > at o.a.p.hwpf.model.SEPX.getSectionProperties(SEPX.java:61)
> > at o.a.p.hwpf.usermodel.Section.(Section.java:36)
> > at o.a.p.hwpf.usermodel.Range.getSection(Range.java:745)
> > at o.a.p.hwpf.converter.AbstractWordConverter.processDocument(
> AbstractWordConverter.java:721)
> > at o.a.p.hwpf.extractor.WordExtractor.getText(
> WordExtractor.java:299)
> > ... 9 more
> >
> > * 4 times NullPointerException in XSLFTextParagraph.getDefaultFontSize()
> >
> > java.lang.NullPointerException
> > at o.a.p.xslf.usermodel.XSLFTextParagraph.getDefaultFontSize(
> XSLFTextParagraph.java:935)
> > at o.a.p.sl.draw.DrawTextParagraph.getAttributedString(
> DrawTextParagraph.java:567)
> > at o.a.p.sl.draw.DrawTextParagraph.breakText(
> DrawTextParagraph.java:235)
> > at o.a.p.sl.draw.DrawTextShape.drawParagraphs(DrawTextShape.
> java:158)
> > at o.a.p.sl.draw.DrawTextShape.getTextHeight(DrawTextShape.
> java:219)
> > at o.a.p.sl.draw.DrawTextShape.drawContent(DrawTextShape.
> java:102)
> > at o.a.p.sl.draw.DrawSimpleShape.draw(DrawSimpleShape.java:93)
> > at o.a.p.sl.draw.DrawSheet.draw(DrawSheet.java:67)
> > at o.a.p.sl.draw.DrawSlide.draw(DrawSlide.java:39)
> > at o.a.p.xslf.usermodel.XSLFSlide.draw(XSLFSlide.java:301)
> > at o.a.p.stress.SlideShowHandler.renderSlides(SlideShowHandler.
> java:120)
> > at o.a.p.stress.SlideShowHandler.handleSlideShow(
> SlideShowHandler.java:43)
> > at o.a.p.stress.XSLFFileHandler.handleFile(XSLFFileHandler.
> java:43)
> > at org.dstadler.commoncrawl.FileHandlingRunnable.run(
> FileHandlingRunnable.java:58)
> >
> >
> >
> > The others are probably flaky things where files caused OOM/Timeout
> before
> > and thus were not reported with these errors before.
> >
> >
> > See http://people.apache.org/~centic/poi_regression/reports/ and
> > http://people.apache.org/~centic/poi_regression/reportsAll/ for detailed
> > results.
>
Re: [VOTE] Apache POI 3.15-beta3
Posted by Javen O'Neal <ja...@gmail.com>.
I agree. Premature optimization here. These getters must have side-effects.
*grumble*
On Aug 16, 2016 1:42 PM, "Dominik Stadler" <do...@gmx.at> wrote:
> Hi,
>
> The 2nd item is a regression, so I didn't think it should be related to
> those code-pieces as we did not change them compared to previous versions.
>
> Therefore I git-bisected this and it seems the following change causes this
> regression:
>
> https://svn.apache.org/viewvc?view=rev&rev=1753048
>
> It seems the extraction of the call to getOperand()/getOperation() outside
> of the switch causes it to be called in more cases than before, namely in
> some of the switch-cases it was not called before, but is now, thus
> triggering this issue.
>
> Javen, I think we can simply undo this change for now as it was not
> intended to change behavior, or? The additional log can stay in place,
> naturally.
>
> Dominik.
>
>
>
> On Mon, Aug 15, 2016 at 6:09 PM, Javen O'Neal <ja...@gmail.com>
> wrote:
>
> > > * 4 times NullPointerException in XSLFTextParagraph.
> getDefaultFontSize()
> > I opened bug 60005 [1] to fix the NPE in XSLFTextParagraph.
> > getDefaultFontSize()
> > This has been fixed.
> >
> > > * 456 times: ArrayIndexOutOfBoundsException in
> SprmOperation.getOperand()
> > The assumption that the operand length is "surely shorter than an int"
> > seems to incorrect for some files, since the test failed with AIOOB 4,
> > meaning the operandLength is at least 5 bytes.
> > I would need to look at the HWPF document specification ([MS-DOC].pdf,
> > [3]) to see if the operandLength may be longer than an int.
> > I do not know what the best way to fix this code is
> >
> > // surely shorter than an int...
> > byte operandLength = _grpprl[_gOffset + 1];
> >
> > // initialized to zeros by JVM
> > byte[] codeBytes = new byte[LittleEndian.INT_SIZE];
> > for ( int i = 0; i < operandLength; i++ )
> > if ( _gOffset + i < _grpprl.length )
> > codeBytes[i] = _grpprl[_gOffset + 1 + i];
> >
> > return LittleEndian.getInt( codeBytes, 0 );
> >
> >
> > [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=60005
> > [2] https://svn.apache.org/viewvc/poi/trunk/src/scratchpad/src/
> > org/apache/poi/hwpf/sprm/SprmOperation.java?revision=
> > 1753052&view=markup#l106
> > [3] https://interoperability.blob.core.windows.net/files/
> > OfficeFileFormatsProtocols.zip
> >
> > On Mon, Aug 15, 2016 at 3:09 AM, Dominik Stadler <dominik.stadler@gmx.at
> >
> > wrote:
> > > Hi,
> > >
> > > Running the regression tests for POI 3.15-beta3 against the CommonCrawl
> > > corpus is now finished, initial results are as follows:
> > >
> > > * 11966 fail because I did not add commons-collections4, I'll trigger a
> > > re-run to get document-counts correctly show the number of regressing
> > > documents
> > >
> > > * 456 times: ArrayIndexOutOfBoundsException in
> SprmOperation.getOperand()
> > >
> > > java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException:
> *
> > > at o.a.p.hwpf.extractor.WordExtractor.getText(
> > WordExtractor.java:317)
> > > at o.a.p.stress.AbstractFileHandler.handleExtractingInternal(
> > AbstractFileHandler.java:85)
> > > at o.a.p.stress.AbstractFileHandler.handleExtracting(
> > AbstractFileHandler.java:60)
> > > at org.dstadler.commoncrawl.FileHandlingRunnable.run(
> > FileHandlingRunnable.java:58)
> > >
> > > Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
> > > at o.a.p.hwpf.sprm.SprmOperation.
> getOperand(SprmOperation.java:
> > 113)
> > > at o.a.p.hwpf.sprm.SectionSprmUncompressor.
> > unCompressSEPOperation(SectionSprmUncompressor.java:62)
> > > at o.a.p.hwpf.sprm.SectionSprmUncompressor.uncompressSEP(
> > SectionSprmUncompressor.java:44)
> > > at o.a.p.hwpf.model.SEPX.getSectionProperties(SEPX.java:61)
> > > at o.a.p.hwpf.usermodel.Section.(Section.java:36)
> > > at o.a.p.hwpf.usermodel.Range.getSection(Range.java:745)
> > > at o.a.p.hwpf.converter.AbstractWordConverter.processDocument(
> > AbstractWordConverter.java:721)
> > > at o.a.p.hwpf.extractor.WordExtractor.getText(
> > WordExtractor.java:299)
> > > ... 9 more
> > >
> > > * 4 times NullPointerException in XSLFTextParagraph.
> getDefaultFontSize()
> > >
> > > java.lang.NullPointerException
> > > at o.a.p.xslf.usermodel.XSLFTextParagraph.getDefaultFontSize(
> > XSLFTextParagraph.java:935)
> > > at o.a.p.sl.draw.DrawTextParagraph.getAttributedString(
> > DrawTextParagraph.java:567)
> > > at o.a.p.sl.draw.DrawTextParagraph.breakText(
> > DrawTextParagraph.java:235)
> > > at o.a.p.sl.draw.DrawTextShape.drawParagraphs(DrawTextShape.
> > java:158)
> > > at o.a.p.sl.draw.DrawTextShape.getTextHeight(DrawTextShape.
> > java:219)
> > > at o.a.p.sl.draw.DrawTextShape.drawContent(DrawTextShape.
> > java:102)
> > > at o.a.p.sl.draw.DrawSimpleShape.draw(DrawSimpleShape.java:93)
> > > at o.a.p.sl.draw.DrawSheet.draw(DrawSheet.java:67)
> > > at o.a.p.sl.draw.DrawSlide.draw(DrawSlide.java:39)
> > > at o.a.p.xslf.usermodel.XSLFSlide.draw(XSLFSlide.java:301)
> > > at o.a.p.stress.SlideShowHandler.
> renderSlides(SlideShowHandler.
> > java:120)
> > > at o.a.p.stress.SlideShowHandler.handleSlideShow(
> > SlideShowHandler.java:43)
> > > at o.a.p.stress.XSLFFileHandler.handleFile(XSLFFileHandler.
> > java:43)
> > > at org.dstadler.commoncrawl.FileHandlingRunnable.run(
> > FileHandlingRunnable.java:58)
> > >
> > >
> > >
> > > The others are probably flaky things where files caused OOM/Timeout
> > before
> > > and thus were not reported with these errors before.
> > >
> > >
> > > See http://people.apache.org/~centic/poi_regression/reports/ and
> > > http://people.apache.org/~centic/poi_regression/reportsAll/ for
> detailed
> > > results.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
> > For additional commands, e-mail: dev-help@poi.apache.org
> >
> >
>
Re: [VOTE] Apache POI 3.15-beta3
Posted by Dominik Stadler <do...@gmx.at>.
Hi,
The 2nd item is a regression, so I didn't think it should be related to
those code-pieces as we did not change them compared to previous versions.
Therefore I git-bisected this and it seems the following change causes this
regression:
https://svn.apache.org/viewvc?view=rev&rev=1753048
It seems the extraction of the call to getOperand()/getOperation() outside
of the switch causes it to be called in more cases than before, namely in
some of the switch-cases it was not called before, but is now, thus
triggering this issue.
Javen, I think we can simply undo this change for now as it was not
intended to change behavior, or? The additional log can stay in place,
naturally.
Dominik.
On Mon, Aug 15, 2016 at 6:09 PM, Javen O'Neal <ja...@gmail.com> wrote:
> > * 4 times NullPointerException in XSLFTextParagraph.getDefaultFontSize()
> I opened bug 60005 [1] to fix the NPE in XSLFTextParagraph.
> getDefaultFontSize()
> This has been fixed.
>
> > * 456 times: ArrayIndexOutOfBoundsException in SprmOperation.getOperand()
> The assumption that the operand length is "surely shorter than an int"
> seems to incorrect for some files, since the test failed with AIOOB 4,
> meaning the operandLength is at least 5 bytes.
> I would need to look at the HWPF document specification ([MS-DOC].pdf,
> [3]) to see if the operandLength may be longer than an int.
> I do not know what the best way to fix this code is
>
> // surely shorter than an int...
> byte operandLength = _grpprl[_gOffset + 1];
>
> // initialized to zeros by JVM
> byte[] codeBytes = new byte[LittleEndian.INT_SIZE];
> for ( int i = 0; i < operandLength; i++ )
> if ( _gOffset + i < _grpprl.length )
> codeBytes[i] = _grpprl[_gOffset + 1 + i];
>
> return LittleEndian.getInt( codeBytes, 0 );
>
>
> [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=60005
> [2] https://svn.apache.org/viewvc/poi/trunk/src/scratchpad/src/
> org/apache/poi/hwpf/sprm/SprmOperation.java?revision=
> 1753052&view=markup#l106
> [3] https://interoperability.blob.core.windows.net/files/
> OfficeFileFormatsProtocols.zip
>
> On Mon, Aug 15, 2016 at 3:09 AM, Dominik Stadler <do...@gmx.at>
> wrote:
> > Hi,
> >
> > Running the regression tests for POI 3.15-beta3 against the CommonCrawl
> > corpus is now finished, initial results are as follows:
> >
> > * 11966 fail because I did not add commons-collections4, I'll trigger a
> > re-run to get document-counts correctly show the number of regressing
> > documents
> >
> > * 456 times: ArrayIndexOutOfBoundsException in SprmOperation.getOperand()
> >
> > java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: *
> > at o.a.p.hwpf.extractor.WordExtractor.getText(
> WordExtractor.java:317)
> > at o.a.p.stress.AbstractFileHandler.handleExtractingInternal(
> AbstractFileHandler.java:85)
> > at o.a.p.stress.AbstractFileHandler.handleExtracting(
> AbstractFileHandler.java:60)
> > at org.dstadler.commoncrawl.FileHandlingRunnable.run(
> FileHandlingRunnable.java:58)
> >
> > Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
> > at o.a.p.hwpf.sprm.SprmOperation.getOperand(SprmOperation.java:
> 113)
> > at o.a.p.hwpf.sprm.SectionSprmUncompressor.
> unCompressSEPOperation(SectionSprmUncompressor.java:62)
> > at o.a.p.hwpf.sprm.SectionSprmUncompressor.uncompressSEP(
> SectionSprmUncompressor.java:44)
> > at o.a.p.hwpf.model.SEPX.getSectionProperties(SEPX.java:61)
> > at o.a.p.hwpf.usermodel.Section.(Section.java:36)
> > at o.a.p.hwpf.usermodel.Range.getSection(Range.java:745)
> > at o.a.p.hwpf.converter.AbstractWordConverter.processDocument(
> AbstractWordConverter.java:721)
> > at o.a.p.hwpf.extractor.WordExtractor.getText(
> WordExtractor.java:299)
> > ... 9 more
> >
> > * 4 times NullPointerException in XSLFTextParagraph.getDefaultFontSize()
> >
> > java.lang.NullPointerException
> > at o.a.p.xslf.usermodel.XSLFTextParagraph.getDefaultFontSize(
> XSLFTextParagraph.java:935)
> > at o.a.p.sl.draw.DrawTextParagraph.getAttributedString(
> DrawTextParagraph.java:567)
> > at o.a.p.sl.draw.DrawTextParagraph.breakText(
> DrawTextParagraph.java:235)
> > at o.a.p.sl.draw.DrawTextShape.drawParagraphs(DrawTextShape.
> java:158)
> > at o.a.p.sl.draw.DrawTextShape.getTextHeight(DrawTextShape.
> java:219)
> > at o.a.p.sl.draw.DrawTextShape.drawContent(DrawTextShape.
> java:102)
> > at o.a.p.sl.draw.DrawSimpleShape.draw(DrawSimpleShape.java:93)
> > at o.a.p.sl.draw.DrawSheet.draw(DrawSheet.java:67)
> > at o.a.p.sl.draw.DrawSlide.draw(DrawSlide.java:39)
> > at o.a.p.xslf.usermodel.XSLFSlide.draw(XSLFSlide.java:301)
> > at o.a.p.stress.SlideShowHandler.renderSlides(SlideShowHandler.
> java:120)
> > at o.a.p.stress.SlideShowHandler.handleSlideShow(
> SlideShowHandler.java:43)
> > at o.a.p.stress.XSLFFileHandler.handleFile(XSLFFileHandler.
> java:43)
> > at org.dstadler.commoncrawl.FileHandlingRunnable.run(
> FileHandlingRunnable.java:58)
> >
> >
> >
> > The others are probably flaky things where files caused OOM/Timeout
> before
> > and thus were not reported with these errors before.
> >
> >
> > See http://people.apache.org/~centic/poi_regression/reports/ and
> > http://people.apache.org/~centic/poi_regression/reportsAll/ for detailed
> > results.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
> For additional commands, e-mail: dev-help@poi.apache.org
>
>
Re: [VOTE] Apache POI 3.15-beta3
Posted by Javen O'Neal <ja...@gmail.com>.
> * 4 times NullPointerException in XSLFTextParagraph.getDefaultFontSize()
I opened bug 60005 [1] to fix the NPE in XSLFTextParagraph.getDefaultFontSize()
This has been fixed.
> * 456 times: ArrayIndexOutOfBoundsException in SprmOperation.getOperand()
The assumption that the operand length is "surely shorter than an int"
seems to incorrect for some files, since the test failed with AIOOB 4,
meaning the operandLength is at least 5 bytes.
I would need to look at the HWPF document specification ([MS-DOC].pdf,
[3]) to see if the operandLength may be longer than an int.
I do not know what the best way to fix this code is
// surely shorter than an int...
byte operandLength = _grpprl[_gOffset + 1];
// initialized to zeros by JVM
byte[] codeBytes = new byte[LittleEndian.INT_SIZE];
for ( int i = 0; i < operandLength; i++ )
if ( _gOffset + i < _grpprl.length )
codeBytes[i] = _grpprl[_gOffset + 1 + i];
return LittleEndian.getInt( codeBytes, 0 );
[1] https://bz.apache.org/bugzilla/show_bug.cgi?id=60005
[2] https://svn.apache.org/viewvc/poi/trunk/src/scratchpad/src/org/apache/poi/hwpf/sprm/SprmOperation.java?revision=1753052&view=markup#l106
[3] https://interoperability.blob.core.windows.net/files/OfficeFileFormatsProtocols.zip
On Mon, Aug 15, 2016 at 3:09 AM, Dominik Stadler <do...@gmx.at> wrote:
> Hi,
>
> Running the regression tests for POI 3.15-beta3 against the CommonCrawl
> corpus is now finished, initial results are as follows:
>
> * 11966 fail because I did not add commons-collections4, I'll trigger a
> re-run to get document-counts correctly show the number of regressing
> documents
>
> * 456 times: ArrayIndexOutOfBoundsException in SprmOperation.getOperand()
>
> java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: *
> at o.a.p.hwpf.extractor.WordExtractor.getText(WordExtractor.java:317)
> at o.a.p.stress.AbstractFileHandler.handleExtractingInternal(AbstractFileHandler.java:85)
> at o.a.p.stress.AbstractFileHandler.handleExtracting(AbstractFileHandler.java:60)
> at org.dstadler.commoncrawl.FileHandlingRunnable.run(FileHandlingRunnable.java:58)
>
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
> at o.a.p.hwpf.sprm.SprmOperation.getOperand(SprmOperation.java:113)
> at o.a.p.hwpf.sprm.SectionSprmUncompressor.unCompressSEPOperation(SectionSprmUncompressor.java:62)
> at o.a.p.hwpf.sprm.SectionSprmUncompressor.uncompressSEP(SectionSprmUncompressor.java:44)
> at o.a.p.hwpf.model.SEPX.getSectionProperties(SEPX.java:61)
> at o.a.p.hwpf.usermodel.Section.(Section.java:36)
> at o.a.p.hwpf.usermodel.Range.getSection(Range.java:745)
> at o.a.p.hwpf.converter.AbstractWordConverter.processDocument(AbstractWordConverter.java:721)
> at o.a.p.hwpf.extractor.WordExtractor.getText(WordExtractor.java:299)
> ... 9 more
>
> * 4 times NullPointerException in XSLFTextParagraph.getDefaultFontSize()
>
> java.lang.NullPointerException
> at o.a.p.xslf.usermodel.XSLFTextParagraph.getDefaultFontSize(XSLFTextParagraph.java:935)
> at o.a.p.sl.draw.DrawTextParagraph.getAttributedString(DrawTextParagraph.java:567)
> at o.a.p.sl.draw.DrawTextParagraph.breakText(DrawTextParagraph.java:235)
> at o.a.p.sl.draw.DrawTextShape.drawParagraphs(DrawTextShape.java:158)
> at o.a.p.sl.draw.DrawTextShape.getTextHeight(DrawTextShape.java:219)
> at o.a.p.sl.draw.DrawTextShape.drawContent(DrawTextShape.java:102)
> at o.a.p.sl.draw.DrawSimpleShape.draw(DrawSimpleShape.java:93)
> at o.a.p.sl.draw.DrawSheet.draw(DrawSheet.java:67)
> at o.a.p.sl.draw.DrawSlide.draw(DrawSlide.java:39)
> at o.a.p.xslf.usermodel.XSLFSlide.draw(XSLFSlide.java:301)
> at o.a.p.stress.SlideShowHandler.renderSlides(SlideShowHandler.java:120)
> at o.a.p.stress.SlideShowHandler.handleSlideShow(SlideShowHandler.java:43)
> at o.a.p.stress.XSLFFileHandler.handleFile(XSLFFileHandler.java:43)
> at org.dstadler.commoncrawl.FileHandlingRunnable.run(FileHandlingRunnable.java:58)
>
>
>
> The others are probably flaky things where files caused OOM/Timeout before
> and thus were not reported with these errors before.
>
>
> See http://people.apache.org/~centic/poi_regression/reports/ and
> http://people.apache.org/~centic/poi_regression/reportsAll/ for detailed
> results.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
RE: [VOTE] Apache POI 3.15-beta3
Posted by Dominik Stadler <do...@gmx.at>.
To be honest I would rather release now, any additional fix carries a small
risk of regressions.
Dominik
On Sep 15, 2016 2:59 AM, "Javen O'Neal" <ja...@gmail.com> wrote:
> I don't mind putting off 3.15 final if it means we have a solid,
> universally usable release. We could do another beta release and wait for a
> final release when it's ready. That would allow us to go back to committing
> changes that we have put off for the last couple months.
>
> Releasing a beta now and postponing a final release would allow us to
> attempt to get an API breakage checker software in place.
>
> On Sep 14, 2016 4:48 PM, "Allison, Timothy B." <ta...@mitre.org> wrote:
>
> > Y, thank you for figuring out what is going on there. I think this is
> low
> > enough priority to put off until 3.16-beta1. Unless there are
> objections...
> >
> > -----Original Message-----
> > From: Javen O'Neal [mailto:onealj@apache.org]
> > Sent: Saturday, September 10, 2016 3:20 AM
> > To: POI Developers List <de...@poi.apache.org>
> > Subject: Re: [VOTE] Apache POI 3.15-beta3
> >
> > Bug 60003 is still open and is a regression if POI should be extracting
> > Prague from the test slideshow.
> >
> > https://bz.apache.org/bugzilla/show_bug.cgi?id=60003
> >
> > On Fri, Sep 9, 2016 at 11:44 AM, Allison, Timothy B. <tallison@mitre.org
> >
> > wrote:
> > > Thank you, Dominik, for catching these! 3 cheers for mass regression
> > testing!
> > >
> > >
> > > I'm finally back from break and catching up on emails...
> > >
> > > -----Original Message-----
> > > From: Dominik Stadler [mailto:dominik.stadler@gmx.at]
> > > Sent: Monday, August 15, 2016 6:09 AM
> > > To: POI Developers List <de...@poi.apache.org>
> > > Subject: Re: [VOTE] Apache POI 3.15-beta3
> > >
> > > Hi,
> > >
> > > Running the regression tests for POI 3.15-beta3 against the CommonCrawl
> > corpus is now finished, initial results are as follows:
> > >
> > > * 11966 fail because I did not add commons-collections4, I'll trigger
> > > a re-run to get document-counts correctly show the number of
> > > regressing documents
> > >
> > > * 456 times: ArrayIndexOutOfBoundsException in
> > > SprmOperation.getOperand()
> > >
> > > java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException:
> *
> > > at o.a.p.hwpf.extractor.WordExtractor.getText(
> > WordExtractor.java:317)
> > > at o.a.p.stress.AbstractFileHandler.handleExtractingInternal(
> > AbstractFileHandler.java:85)
> > > at o.a.p.stress.AbstractFileHandler.handleExtracting(
> > AbstractFileHandler.java:60)
> > > at
> > > org.dstadler.commoncrawl.FileHandlingRunnable.run(FileHandlingRunnable
> > > .java:58)
> > >
> > > Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
> > > at o.a.p.hwpf.sprm.SprmOperation.
> getOperand(SprmOperation.java:
> > 113)
> > > at o.a.p.hwpf.sprm.SectionSprmUncompressor.
> > unCompressSEPOperation(SectionSprmUncompressor.java:62)
> > > at o.a.p.hwpf.sprm.SectionSprmUncompressor.uncompressSEP(
> > SectionSprmUncompressor.java:44)
> > > at o.a.p.hwpf.model.SEPX.getSectionProperties(SEPX.java:61)
> > > at o.a.p.hwpf.usermodel.Section.(Section.java:36)
> > > at o.a.p.hwpf.usermodel.Range.getSection(Range.java:745)
> > > at o.a.p.hwpf.converter.AbstractWordConverter.processDocument(
> > AbstractWordConverter.java:721)
> > > at o.a.p.hwpf.extractor.WordExtractor.getText(
> > WordExtractor.java:299)
> > > ... 9 more
> > >
> > > * 4 times NullPointerException in
> > > XSLFTextParagraph.getDefaultFontSize()
> > >
> > > java.lang.NullPointerException
> > > at o.a.p.xslf.usermodel.XSLFTextParagraph.getDefaultFontSize(
> > XSLFTextParagraph.java:935)
> > > at o.a.p.sl.draw.DrawTextParagraph.getAttributedString(
> > DrawTextParagraph.java:567)
> > > at o.a.p.sl.draw.DrawTextParagraph.breakText(
> > DrawTextParagraph.java:235)
> > > at o.a.p.sl.draw.DrawTextShape.drawParagraphs(DrawTextShape.
> > java:158)
> > > at o.a.p.sl.draw.DrawTextShape.getTextHeight(DrawTextShape.
> > java:219)
> > > at o.a.p.sl.draw.DrawTextShape.drawContent(DrawTextShape.
> > java:102)
> > > at o.a.p.sl.draw.DrawSimpleShape.draw(DrawSimpleShape.java:93)
> > > at o.a.p.sl.draw.DrawSheet.draw(DrawSheet.java:67)
> > > at o.a.p.sl.draw.DrawSlide.draw(DrawSlide.java:39)
> > > at o.a.p.xslf.usermodel.XSLFSlide.draw(XSLFSlide.java:301)
> > > at o.a.p.stress.SlideShowHandler.
> renderSlides(SlideShowHandler.
> > java:120)
> > > at o.a.p.stress.SlideShowHandler.handleSlideShow(
> > SlideShowHandler.java:43)
> > > at o.a.p.stress.XSLFFileHandler.handleFile(XSLFFileHandler.
> > java:43)
> > > at
> > > org.dstadler.commoncrawl.FileHandlingRunnable.run(FileHandlingRunnable
> > > .java:58)
> > >
> > >
> > >
> > > The others are probably flaky things where files caused OOM/Timeout
> > before and thus were not reported with these errors before.
> > >
> > >
> > > See http://people.apache.org/~centic/poi_regression/reports/ and
> > http://people.apache.org/~centic/poi_regression/reportsAll/ for detailed
> > results.
> > >
> > >
> > > Thanks... Dominik.
> > >
> > >
> > > On Mon, Aug 15, 2016 at 4:16 AM, Javen O'Neal <on...@apache.org>
> wrote:
> > >
> > >> Correction: HSLF. This is a ppt/OLE2 file.
> > >>
> > >> On Sun, Aug 14, 2016 at 6:58 PM, Javen O'Neal <on...@apache.org>
> > wrote:
> > >> > Tim,
> > >> >
> > >> > I have extracted the pptx PowerPoint file containing the Prague
> > >> > footer. I'm want to write a unit test for POI to find the Prague
> > >> > string so I can figure why Prague was not included in the Tika
> > >> > regression test using POI 3.15 beta 3 but was found by POI 3.15
> > >> > beta 1.
> > >> >
> > >> > Could you point me to the Tika code that generated the potential
> > >> > regressions zip file in TIKA-2013, or the POI class/function that
> > >> > is used to extract the text from a document?
> > >> >
> > >> > Also, is the pptx file shareable and ASL 2.0 licensed so that it
> > >> > can be included as part of POI's unit test suite?
> > >> >
> > >> > On Fri, Aug 12, 2016 at 6:52 PM, Javen O'Neal
> > >> > <ja...@gmail.com>
> > >> wrote:
> > >> >> On Aug 12, 2016 11:39, "Allison, Timothy B." <ta...@mitre.org>
> > >> wrote:
> > >> >>>...the two potential content regressions may be caused by
> > >> >>>something at
> > >> the
> > >> >>> Tika level. If anyone has time to take a look, that'd be great.
> > >> >>
> > >> >> I can take a look this weekend.
> > >> >>
> > >> >> Did you use the same Tika code with different POI versions for
> > >> >> these
> > >> tests
> > >> >> (so that we can attribute the change in behavior to a POI commit,
> > >> regardless
> > >> >> of whether the bug is in Tika or POI)?
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org For additional
> > >> commands, e-mail: dev-help@poi.apache.org
> > >>
> > >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org For additional
> > commands, e-mail: dev-help@poi.apache.org
> >
> >
>
RE: [VOTE] Apache POI 3.15-beta3
Posted by Javen O'Neal <ja...@gmail.com>.
I don't mind putting off 3.15 final if it means we have a solid,
universally usable release. We could do another beta release and wait for a
final release when it's ready. That would allow us to go back to committing
changes that we have put off for the last couple months.
Releasing a beta now and postponing a final release would allow us to
attempt to get an API breakage checker software in place.
On Sep 14, 2016 4:48 PM, "Allison, Timothy B." <ta...@mitre.org> wrote:
> Y, thank you for figuring out what is going on there. I think this is low
> enough priority to put off until 3.16-beta1. Unless there are objections...
>
> -----Original Message-----
> From: Javen O'Neal [mailto:onealj@apache.org]
> Sent: Saturday, September 10, 2016 3:20 AM
> To: POI Developers List <de...@poi.apache.org>
> Subject: Re: [VOTE] Apache POI 3.15-beta3
>
> Bug 60003 is still open and is a regression if POI should be extracting
> Prague from the test slideshow.
>
> https://bz.apache.org/bugzilla/show_bug.cgi?id=60003
>
> On Fri, Sep 9, 2016 at 11:44 AM, Allison, Timothy B. <ta...@mitre.org>
> wrote:
> > Thank you, Dominik, for catching these! 3 cheers for mass regression
> testing!
> >
> >
> > I'm finally back from break and catching up on emails...
> >
> > -----Original Message-----
> > From: Dominik Stadler [mailto:dominik.stadler@gmx.at]
> > Sent: Monday, August 15, 2016 6:09 AM
> > To: POI Developers List <de...@poi.apache.org>
> > Subject: Re: [VOTE] Apache POI 3.15-beta3
> >
> > Hi,
> >
> > Running the regression tests for POI 3.15-beta3 against the CommonCrawl
> corpus is now finished, initial results are as follows:
> >
> > * 11966 fail because I did not add commons-collections4, I'll trigger
> > a re-run to get document-counts correctly show the number of
> > regressing documents
> >
> > * 456 times: ArrayIndexOutOfBoundsException in
> > SprmOperation.getOperand()
> >
> > java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: *
> > at o.a.p.hwpf.extractor.WordExtractor.getText(
> WordExtractor.java:317)
> > at o.a.p.stress.AbstractFileHandler.handleExtractingInternal(
> AbstractFileHandler.java:85)
> > at o.a.p.stress.AbstractFileHandler.handleExtracting(
> AbstractFileHandler.java:60)
> > at
> > org.dstadler.commoncrawl.FileHandlingRunnable.run(FileHandlingRunnable
> > .java:58)
> >
> > Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
> > at o.a.p.hwpf.sprm.SprmOperation.getOperand(SprmOperation.java:
> 113)
> > at o.a.p.hwpf.sprm.SectionSprmUncompressor.
> unCompressSEPOperation(SectionSprmUncompressor.java:62)
> > at o.a.p.hwpf.sprm.SectionSprmUncompressor.uncompressSEP(
> SectionSprmUncompressor.java:44)
> > at o.a.p.hwpf.model.SEPX.getSectionProperties(SEPX.java:61)
> > at o.a.p.hwpf.usermodel.Section.(Section.java:36)
> > at o.a.p.hwpf.usermodel.Range.getSection(Range.java:745)
> > at o.a.p.hwpf.converter.AbstractWordConverter.processDocument(
> AbstractWordConverter.java:721)
> > at o.a.p.hwpf.extractor.WordExtractor.getText(
> WordExtractor.java:299)
> > ... 9 more
> >
> > * 4 times NullPointerException in
> > XSLFTextParagraph.getDefaultFontSize()
> >
> > java.lang.NullPointerException
> > at o.a.p.xslf.usermodel.XSLFTextParagraph.getDefaultFontSize(
> XSLFTextParagraph.java:935)
> > at o.a.p.sl.draw.DrawTextParagraph.getAttributedString(
> DrawTextParagraph.java:567)
> > at o.a.p.sl.draw.DrawTextParagraph.breakText(
> DrawTextParagraph.java:235)
> > at o.a.p.sl.draw.DrawTextShape.drawParagraphs(DrawTextShape.
> java:158)
> > at o.a.p.sl.draw.DrawTextShape.getTextHeight(DrawTextShape.
> java:219)
> > at o.a.p.sl.draw.DrawTextShape.drawContent(DrawTextShape.
> java:102)
> > at o.a.p.sl.draw.DrawSimpleShape.draw(DrawSimpleShape.java:93)
> > at o.a.p.sl.draw.DrawSheet.draw(DrawSheet.java:67)
> > at o.a.p.sl.draw.DrawSlide.draw(DrawSlide.java:39)
> > at o.a.p.xslf.usermodel.XSLFSlide.draw(XSLFSlide.java:301)
> > at o.a.p.stress.SlideShowHandler.renderSlides(SlideShowHandler.
> java:120)
> > at o.a.p.stress.SlideShowHandler.handleSlideShow(
> SlideShowHandler.java:43)
> > at o.a.p.stress.XSLFFileHandler.handleFile(XSLFFileHandler.
> java:43)
> > at
> > org.dstadler.commoncrawl.FileHandlingRunnable.run(FileHandlingRunnable
> > .java:58)
> >
> >
> >
> > The others are probably flaky things where files caused OOM/Timeout
> before and thus were not reported with these errors before.
> >
> >
> > See http://people.apache.org/~centic/poi_regression/reports/ and
> http://people.apache.org/~centic/poi_regression/reportsAll/ for detailed
> results.
> >
> >
> > Thanks... Dominik.
> >
> >
> > On Mon, Aug 15, 2016 at 4:16 AM, Javen O'Neal <on...@apache.org> wrote:
> >
> >> Correction: HSLF. This is a ppt/OLE2 file.
> >>
> >> On Sun, Aug 14, 2016 at 6:58 PM, Javen O'Neal <on...@apache.org>
> wrote:
> >> > Tim,
> >> >
> >> > I have extracted the pptx PowerPoint file containing the Prague
> >> > footer. I'm want to write a unit test for POI to find the Prague
> >> > string so I can figure why Prague was not included in the Tika
> >> > regression test using POI 3.15 beta 3 but was found by POI 3.15
> >> > beta 1.
> >> >
> >> > Could you point me to the Tika code that generated the potential
> >> > regressions zip file in TIKA-2013, or the POI class/function that
> >> > is used to extract the text from a document?
> >> >
> >> > Also, is the pptx file shareable and ASL 2.0 licensed so that it
> >> > can be included as part of POI's unit test suite?
> >> >
> >> > On Fri, Aug 12, 2016 at 6:52 PM, Javen O'Neal
> >> > <ja...@gmail.com>
> >> wrote:
> >> >> On Aug 12, 2016 11:39, "Allison, Timothy B." <ta...@mitre.org>
> >> wrote:
> >> >>>...the two potential content regressions may be caused by
> >> >>>something at
> >> the
> >> >>> Tika level. If anyone has time to take a look, that'd be great.
> >> >>
> >> >> I can take a look this weekend.
> >> >>
> >> >> Did you use the same Tika code with different POI versions for
> >> >> these
> >> tests
> >> >> (so that we can attribute the change in behavior to a POI commit,
> >> regardless
> >> >> of whether the bug is in Tika or POI)?
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org For additional
> >> commands, e-mail: dev-help@poi.apache.org
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org For additional
> commands, e-mail: dev-help@poi.apache.org
>
>
RE: [VOTE] Apache POI 3.15-beta3
Posted by "Allison, Timothy B." <ta...@mitre.org>.
Y, thank you for figuring out what is going on there. I think this is low enough priority to put off until 3.16-beta1. Unless there are objections...
-----Original Message-----
From: Javen O'Neal [mailto:onealj@apache.org]
Sent: Saturday, September 10, 2016 3:20 AM
To: POI Developers List <de...@poi.apache.org>
Subject: Re: [VOTE] Apache POI 3.15-beta3
Bug 60003 is still open and is a regression if POI should be extracting Prague from the test slideshow.
https://bz.apache.org/bugzilla/show_bug.cgi?id=60003
On Fri, Sep 9, 2016 at 11:44 AM, Allison, Timothy B. <ta...@mitre.org> wrote:
> Thank you, Dominik, for catching these! 3 cheers for mass regression testing!
>
>
> I'm finally back from break and catching up on emails...
>
> -----Original Message-----
> From: Dominik Stadler [mailto:dominik.stadler@gmx.at]
> Sent: Monday, August 15, 2016 6:09 AM
> To: POI Developers List <de...@poi.apache.org>
> Subject: Re: [VOTE] Apache POI 3.15-beta3
>
> Hi,
>
> Running the regression tests for POI 3.15-beta3 against the CommonCrawl corpus is now finished, initial results are as follows:
>
> * 11966 fail because I did not add commons-collections4, I'll trigger
> a re-run to get document-counts correctly show the number of
> regressing documents
>
> * 456 times: ArrayIndexOutOfBoundsException in
> SprmOperation.getOperand()
>
> java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: *
> at o.a.p.hwpf.extractor.WordExtractor.getText(WordExtractor.java:317)
> at o.a.p.stress.AbstractFileHandler.handleExtractingInternal(AbstractFileHandler.java:85)
> at o.a.p.stress.AbstractFileHandler.handleExtracting(AbstractFileHandler.java:60)
> at
> org.dstadler.commoncrawl.FileHandlingRunnable.run(FileHandlingRunnable
> .java:58)
>
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
> at o.a.p.hwpf.sprm.SprmOperation.getOperand(SprmOperation.java:113)
> at o.a.p.hwpf.sprm.SectionSprmUncompressor.unCompressSEPOperation(SectionSprmUncompressor.java:62)
> at o.a.p.hwpf.sprm.SectionSprmUncompressor.uncompressSEP(SectionSprmUncompressor.java:44)
> at o.a.p.hwpf.model.SEPX.getSectionProperties(SEPX.java:61)
> at o.a.p.hwpf.usermodel.Section.(Section.java:36)
> at o.a.p.hwpf.usermodel.Range.getSection(Range.java:745)
> at o.a.p.hwpf.converter.AbstractWordConverter.processDocument(AbstractWordConverter.java:721)
> at o.a.p.hwpf.extractor.WordExtractor.getText(WordExtractor.java:299)
> ... 9 more
>
> * 4 times NullPointerException in
> XSLFTextParagraph.getDefaultFontSize()
>
> java.lang.NullPointerException
> at o.a.p.xslf.usermodel.XSLFTextParagraph.getDefaultFontSize(XSLFTextParagraph.java:935)
> at o.a.p.sl.draw.DrawTextParagraph.getAttributedString(DrawTextParagraph.java:567)
> at o.a.p.sl.draw.DrawTextParagraph.breakText(DrawTextParagraph.java:235)
> at o.a.p.sl.draw.DrawTextShape.drawParagraphs(DrawTextShape.java:158)
> at o.a.p.sl.draw.DrawTextShape.getTextHeight(DrawTextShape.java:219)
> at o.a.p.sl.draw.DrawTextShape.drawContent(DrawTextShape.java:102)
> at o.a.p.sl.draw.DrawSimpleShape.draw(DrawSimpleShape.java:93)
> at o.a.p.sl.draw.DrawSheet.draw(DrawSheet.java:67)
> at o.a.p.sl.draw.DrawSlide.draw(DrawSlide.java:39)
> at o.a.p.xslf.usermodel.XSLFSlide.draw(XSLFSlide.java:301)
> at o.a.p.stress.SlideShowHandler.renderSlides(SlideShowHandler.java:120)
> at o.a.p.stress.SlideShowHandler.handleSlideShow(SlideShowHandler.java:43)
> at o.a.p.stress.XSLFFileHandler.handleFile(XSLFFileHandler.java:43)
> at
> org.dstadler.commoncrawl.FileHandlingRunnable.run(FileHandlingRunnable
> .java:58)
>
>
>
> The others are probably flaky things where files caused OOM/Timeout before and thus were not reported with these errors before.
>
>
> See http://people.apache.org/~centic/poi_regression/reports/ and http://people.apache.org/~centic/poi_regression/reportsAll/ for detailed results.
>
>
> Thanks... Dominik.
>
>
> On Mon, Aug 15, 2016 at 4:16 AM, Javen O'Neal <on...@apache.org> wrote:
>
>> Correction: HSLF. This is a ppt/OLE2 file.
>>
>> On Sun, Aug 14, 2016 at 6:58 PM, Javen O'Neal <on...@apache.org> wrote:
>> > Tim,
>> >
>> > I have extracted the pptx PowerPoint file containing the Prague
>> > footer. I'm want to write a unit test for POI to find the Prague
>> > string so I can figure why Prague was not included in the Tika
>> > regression test using POI 3.15 beta 3 but was found by POI 3.15
>> > beta 1.
>> >
>> > Could you point me to the Tika code that generated the potential
>> > regressions zip file in TIKA-2013, or the POI class/function that
>> > is used to extract the text from a document?
>> >
>> > Also, is the pptx file shareable and ASL 2.0 licensed so that it
>> > can be included as part of POI's unit test suite?
>> >
>> > On Fri, Aug 12, 2016 at 6:52 PM, Javen O'Neal
>> > <ja...@gmail.com>
>> wrote:
>> >> On Aug 12, 2016 11:39, "Allison, Timothy B." <ta...@mitre.org>
>> wrote:
>> >>>...the two potential content regressions may be caused by
>> >>>something at
>> the
>> >>> Tika level. If anyone has time to take a look, that'd be great.
>> >>
>> >> I can take a look this weekend.
>> >>
>> >> Did you use the same Tika code with different POI versions for
>> >> these
>> tests
>> >> (so that we can attribute the change in behavior to a POI commit,
>> regardless
>> >> of whether the bug is in Tika or POI)?
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org For additional
>> commands, e-mail: dev-help@poi.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org For additional commands, e-mail: dev-help@poi.apache.org
Re: [VOTE] Apache POI 3.15-beta3
Posted by Javen O'Neal <on...@apache.org>.
Bug 60003 is still open and is a regression if POI should be
extracting Prague from the test slideshow.
https://bz.apache.org/bugzilla/show_bug.cgi?id=60003
On Fri, Sep 9, 2016 at 11:44 AM, Allison, Timothy B. <ta...@mitre.org> wrote:
> Thank you, Dominik, for catching these! 3 cheers for mass regression testing!
>
>
> I'm finally back from break and catching up on emails...
>
> -----Original Message-----
> From: Dominik Stadler [mailto:dominik.stadler@gmx.at]
> Sent: Monday, August 15, 2016 6:09 AM
> To: POI Developers List <de...@poi.apache.org>
> Subject: Re: [VOTE] Apache POI 3.15-beta3
>
> Hi,
>
> Running the regression tests for POI 3.15-beta3 against the CommonCrawl corpus is now finished, initial results are as follows:
>
> * 11966 fail because I did not add commons-collections4, I'll trigger a re-run to get document-counts correctly show the number of regressing documents
>
> * 456 times: ArrayIndexOutOfBoundsException in SprmOperation.getOperand()
>
> java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: *
> at o.a.p.hwpf.extractor.WordExtractor.getText(WordExtractor.java:317)
> at o.a.p.stress.AbstractFileHandler.handleExtractingInternal(AbstractFileHandler.java:85)
> at o.a.p.stress.AbstractFileHandler.handleExtracting(AbstractFileHandler.java:60)
> at org.dstadler.commoncrawl.FileHandlingRunnable.run(FileHandlingRunnable.java:58)
>
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
> at o.a.p.hwpf.sprm.SprmOperation.getOperand(SprmOperation.java:113)
> at o.a.p.hwpf.sprm.SectionSprmUncompressor.unCompressSEPOperation(SectionSprmUncompressor.java:62)
> at o.a.p.hwpf.sprm.SectionSprmUncompressor.uncompressSEP(SectionSprmUncompressor.java:44)
> at o.a.p.hwpf.model.SEPX.getSectionProperties(SEPX.java:61)
> at o.a.p.hwpf.usermodel.Section.(Section.java:36)
> at o.a.p.hwpf.usermodel.Range.getSection(Range.java:745)
> at o.a.p.hwpf.converter.AbstractWordConverter.processDocument(AbstractWordConverter.java:721)
> at o.a.p.hwpf.extractor.WordExtractor.getText(WordExtractor.java:299)
> ... 9 more
>
> * 4 times NullPointerException in XSLFTextParagraph.getDefaultFontSize()
>
> java.lang.NullPointerException
> at o.a.p.xslf.usermodel.XSLFTextParagraph.getDefaultFontSize(XSLFTextParagraph.java:935)
> at o.a.p.sl.draw.DrawTextParagraph.getAttributedString(DrawTextParagraph.java:567)
> at o.a.p.sl.draw.DrawTextParagraph.breakText(DrawTextParagraph.java:235)
> at o.a.p.sl.draw.DrawTextShape.drawParagraphs(DrawTextShape.java:158)
> at o.a.p.sl.draw.DrawTextShape.getTextHeight(DrawTextShape.java:219)
> at o.a.p.sl.draw.DrawTextShape.drawContent(DrawTextShape.java:102)
> at o.a.p.sl.draw.DrawSimpleShape.draw(DrawSimpleShape.java:93)
> at o.a.p.sl.draw.DrawSheet.draw(DrawSheet.java:67)
> at o.a.p.sl.draw.DrawSlide.draw(DrawSlide.java:39)
> at o.a.p.xslf.usermodel.XSLFSlide.draw(XSLFSlide.java:301)
> at o.a.p.stress.SlideShowHandler.renderSlides(SlideShowHandler.java:120)
> at o.a.p.stress.SlideShowHandler.handleSlideShow(SlideShowHandler.java:43)
> at o.a.p.stress.XSLFFileHandler.handleFile(XSLFFileHandler.java:43)
> at org.dstadler.commoncrawl.FileHandlingRunnable.run(FileHandlingRunnable.java:58)
>
>
>
> The others are probably flaky things where files caused OOM/Timeout before and thus were not reported with these errors before.
>
>
> See http://people.apache.org/~centic/poi_regression/reports/ and http://people.apache.org/~centic/poi_regression/reportsAll/ for detailed results.
>
>
> Thanks... Dominik.
>
>
> On Mon, Aug 15, 2016 at 4:16 AM, Javen O'Neal <on...@apache.org> wrote:
>
>> Correction: HSLF. This is a ppt/OLE2 file.
>>
>> On Sun, Aug 14, 2016 at 6:58 PM, Javen O'Neal <on...@apache.org> wrote:
>> > Tim,
>> >
>> > I have extracted the pptx PowerPoint file containing the Prague
>> > footer. I'm want to write a unit test for POI to find the Prague
>> > string so I can figure why Prague was not included in the Tika
>> > regression test using POI 3.15 beta 3 but was found by POI 3.15 beta
>> > 1.
>> >
>> > Could you point me to the Tika code that generated the potential
>> > regressions zip file in TIKA-2013, or the POI class/function that is
>> > used to extract the text from a document?
>> >
>> > Also, is the pptx file shareable and ASL 2.0 licensed so that it can
>> > be included as part of POI's unit test suite?
>> >
>> > On Fri, Aug 12, 2016 at 6:52 PM, Javen O'Neal <ja...@gmail.com>
>> wrote:
>> >> On Aug 12, 2016 11:39, "Allison, Timothy B." <ta...@mitre.org>
>> wrote:
>> >>>...the two potential content regressions may be caused by something
>> >>>at
>> the
>> >>> Tika level. If anyone has time to take a look, that'd be great.
>> >>
>> >> I can take a look this weekend.
>> >>
>> >> Did you use the same Tika code with different POI versions for
>> >> these
>> tests
>> >> (so that we can attribute the change in behavior to a POI commit,
>> regardless
>> >> of whether the bug is in Tika or POI)?
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org For additional
>> commands, e-mail: dev-help@poi.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
RE: [VOTE] Apache POI 3.15-beta3
Posted by "Allison, Timothy B." <ta...@mitre.org>.
Thank you, Dominik, for catching these! 3 cheers for mass regression testing!
I'm finally back from break and catching up on emails...
-----Original Message-----
From: Dominik Stadler [mailto:dominik.stadler@gmx.at]
Sent: Monday, August 15, 2016 6:09 AM
To: POI Developers List <de...@poi.apache.org>
Subject: Re: [VOTE] Apache POI 3.15-beta3
Hi,
Running the regression tests for POI 3.15-beta3 against the CommonCrawl corpus is now finished, initial results are as follows:
* 11966 fail because I did not add commons-collections4, I'll trigger a re-run to get document-counts correctly show the number of regressing documents
* 456 times: ArrayIndexOutOfBoundsException in SprmOperation.getOperand()
java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: *
at o.a.p.hwpf.extractor.WordExtractor.getText(WordExtractor.java:317)
at o.a.p.stress.AbstractFileHandler.handleExtractingInternal(AbstractFileHandler.java:85)
at o.a.p.stress.AbstractFileHandler.handleExtracting(AbstractFileHandler.java:60)
at org.dstadler.commoncrawl.FileHandlingRunnable.run(FileHandlingRunnable.java:58)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
at o.a.p.hwpf.sprm.SprmOperation.getOperand(SprmOperation.java:113)
at o.a.p.hwpf.sprm.SectionSprmUncompressor.unCompressSEPOperation(SectionSprmUncompressor.java:62)
at o.a.p.hwpf.sprm.SectionSprmUncompressor.uncompressSEP(SectionSprmUncompressor.java:44)
at o.a.p.hwpf.model.SEPX.getSectionProperties(SEPX.java:61)
at o.a.p.hwpf.usermodel.Section.(Section.java:36)
at o.a.p.hwpf.usermodel.Range.getSection(Range.java:745)
at o.a.p.hwpf.converter.AbstractWordConverter.processDocument(AbstractWordConverter.java:721)
at o.a.p.hwpf.extractor.WordExtractor.getText(WordExtractor.java:299)
... 9 more
* 4 times NullPointerException in XSLFTextParagraph.getDefaultFontSize()
java.lang.NullPointerException
at o.a.p.xslf.usermodel.XSLFTextParagraph.getDefaultFontSize(XSLFTextParagraph.java:935)
at o.a.p.sl.draw.DrawTextParagraph.getAttributedString(DrawTextParagraph.java:567)
at o.a.p.sl.draw.DrawTextParagraph.breakText(DrawTextParagraph.java:235)
at o.a.p.sl.draw.DrawTextShape.drawParagraphs(DrawTextShape.java:158)
at o.a.p.sl.draw.DrawTextShape.getTextHeight(DrawTextShape.java:219)
at o.a.p.sl.draw.DrawTextShape.drawContent(DrawTextShape.java:102)
at o.a.p.sl.draw.DrawSimpleShape.draw(DrawSimpleShape.java:93)
at o.a.p.sl.draw.DrawSheet.draw(DrawSheet.java:67)
at o.a.p.sl.draw.DrawSlide.draw(DrawSlide.java:39)
at o.a.p.xslf.usermodel.XSLFSlide.draw(XSLFSlide.java:301)
at o.a.p.stress.SlideShowHandler.renderSlides(SlideShowHandler.java:120)
at o.a.p.stress.SlideShowHandler.handleSlideShow(SlideShowHandler.java:43)
at o.a.p.stress.XSLFFileHandler.handleFile(XSLFFileHandler.java:43)
at org.dstadler.commoncrawl.FileHandlingRunnable.run(FileHandlingRunnable.java:58)
The others are probably flaky things where files caused OOM/Timeout before and thus were not reported with these errors before.
See http://people.apache.org/~centic/poi_regression/reports/ and http://people.apache.org/~centic/poi_regression/reportsAll/ for detailed results.
Thanks... Dominik.
On Mon, Aug 15, 2016 at 4:16 AM, Javen O'Neal <on...@apache.org> wrote:
> Correction: HSLF. This is a ppt/OLE2 file.
>
> On Sun, Aug 14, 2016 at 6:58 PM, Javen O'Neal <on...@apache.org> wrote:
> > Tim,
> >
> > I have extracted the pptx PowerPoint file containing the Prague
> > footer. I'm want to write a unit test for POI to find the Prague
> > string so I can figure why Prague was not included in the Tika
> > regression test using POI 3.15 beta 3 but was found by POI 3.15 beta
> > 1.
> >
> > Could you point me to the Tika code that generated the potential
> > regressions zip file in TIKA-2013, or the POI class/function that is
> > used to extract the text from a document?
> >
> > Also, is the pptx file shareable and ASL 2.0 licensed so that it can
> > be included as part of POI's unit test suite?
> >
> > On Fri, Aug 12, 2016 at 6:52 PM, Javen O'Neal <ja...@gmail.com>
> wrote:
> >> On Aug 12, 2016 11:39, "Allison, Timothy B." <ta...@mitre.org>
> wrote:
> >>>...the two potential content regressions may be caused by something
> >>>at
> the
> >>> Tika level. If anyone has time to take a look, that'd be great.
> >>
> >> I can take a look this weekend.
> >>
> >> Did you use the same Tika code with different POI versions for
> >> these
> tests
> >> (so that we can attribute the change in behavior to a POI commit,
> regardless
> >> of whether the bug is in Tika or POI)?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org For additional
> commands, e-mail: dev-help@poi.apache.org
>
>
Re: [VOTE] Apache POI 3.15-beta3
Posted by Dominik Stadler <do...@gmx.at>.
Hi,
Running the regression tests for POI 3.15-beta3 against the CommonCrawl
corpus is now finished, initial results are as follows:
* 11966 fail because I did not add commons-collections4, I'll trigger a
re-run to get document-counts correctly show the number of regressing
documents
* 456 times: ArrayIndexOutOfBoundsException in SprmOperation.getOperand()
java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: *
at o.a.p.hwpf.extractor.WordExtractor.getText(WordExtractor.java:317)
at o.a.p.stress.AbstractFileHandler.handleExtractingInternal(AbstractFileHandler.java:85)
at o.a.p.stress.AbstractFileHandler.handleExtracting(AbstractFileHandler.java:60)
at org.dstadler.commoncrawl.FileHandlingRunnable.run(FileHandlingRunnable.java:58)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
at o.a.p.hwpf.sprm.SprmOperation.getOperand(SprmOperation.java:113)
at o.a.p.hwpf.sprm.SectionSprmUncompressor.unCompressSEPOperation(SectionSprmUncompressor.java:62)
at o.a.p.hwpf.sprm.SectionSprmUncompressor.uncompressSEP(SectionSprmUncompressor.java:44)
at o.a.p.hwpf.model.SEPX.getSectionProperties(SEPX.java:61)
at o.a.p.hwpf.usermodel.Section.(Section.java:36)
at o.a.p.hwpf.usermodel.Range.getSection(Range.java:745)
at o.a.p.hwpf.converter.AbstractWordConverter.processDocument(AbstractWordConverter.java:721)
at o.a.p.hwpf.extractor.WordExtractor.getText(WordExtractor.java:299)
... 9 more
* 4 times NullPointerException in XSLFTextParagraph.getDefaultFontSize()
java.lang.NullPointerException
at o.a.p.xslf.usermodel.XSLFTextParagraph.getDefaultFontSize(XSLFTextParagraph.java:935)
at o.a.p.sl.draw.DrawTextParagraph.getAttributedString(DrawTextParagraph.java:567)
at o.a.p.sl.draw.DrawTextParagraph.breakText(DrawTextParagraph.java:235)
at o.a.p.sl.draw.DrawTextShape.drawParagraphs(DrawTextShape.java:158)
at o.a.p.sl.draw.DrawTextShape.getTextHeight(DrawTextShape.java:219)
at o.a.p.sl.draw.DrawTextShape.drawContent(DrawTextShape.java:102)
at o.a.p.sl.draw.DrawSimpleShape.draw(DrawSimpleShape.java:93)
at o.a.p.sl.draw.DrawSheet.draw(DrawSheet.java:67)
at o.a.p.sl.draw.DrawSlide.draw(DrawSlide.java:39)
at o.a.p.xslf.usermodel.XSLFSlide.draw(XSLFSlide.java:301)
at o.a.p.stress.SlideShowHandler.renderSlides(SlideShowHandler.java:120)
at o.a.p.stress.SlideShowHandler.handleSlideShow(SlideShowHandler.java:43)
at o.a.p.stress.XSLFFileHandler.handleFile(XSLFFileHandler.java:43)
at org.dstadler.commoncrawl.FileHandlingRunnable.run(FileHandlingRunnable.java:58)
The others are probably flaky things where files caused OOM/Timeout before
and thus were not reported with these errors before.
See http://people.apache.org/~centic/poi_regression/reports/ and
http://people.apache.org/~centic/poi_regression/reportsAll/ for detailed
results.
Thanks... Dominik.
On Mon, Aug 15, 2016 at 4:16 AM, Javen O'Neal <on...@apache.org> wrote:
> Correction: HSLF. This is a ppt/OLE2 file.
>
> On Sun, Aug 14, 2016 at 6:58 PM, Javen O'Neal <on...@apache.org> wrote:
> > Tim,
> >
> > I have extracted the pptx PowerPoint file containing the Prague
> > footer. I'm want to write a unit test for POI to find the Prague
> > string so I can figure why Prague was not included in the Tika
> > regression test using POI 3.15 beta 3 but was found by POI 3.15 beta
> > 1.
> >
> > Could you point me to the Tika code that generated the potential
> > regressions zip file in TIKA-2013, or the POI class/function that is
> > used to extract the text from a document?
> >
> > Also, is the pptx file shareable and ASL 2.0 licensed so that it can
> > be included as part of POI's unit test suite?
> >
> > On Fri, Aug 12, 2016 at 6:52 PM, Javen O'Neal <ja...@gmail.com>
> wrote:
> >> On Aug 12, 2016 11:39, "Allison, Timothy B." <ta...@mitre.org>
> wrote:
> >>>...the two potential content regressions may be caused by something at
> the
> >>> Tika level. If anyone has time to take a look, that'd be great.
> >>
> >> I can take a look this weekend.
> >>
> >> Did you use the same Tika code with different POI versions for these
> tests
> >> (so that we can attribute the change in behavior to a POI commit,
> regardless
> >> of whether the bug is in Tika or POI)?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
> For additional commands, e-mail: dev-help@poi.apache.org
>
>
Re: [VOTE] Apache POI 3.15-beta3
Posted by Javen O'Neal <on...@apache.org>.
I have spent some time on TIKA-2013 Issue #1 (extracting footer text
from a OLE2 power point slide master) and I think the issue is small
enough to proceed with the beta release with this issue open (#60003).
I do not know which files to look at for Issue #2 (some numbers in XLS
are being corrupted), so I cannot make a recommendation either way if
it is necessary to postpone the release until we have a fix.
On Sun, Aug 14, 2016 at 7:16 PM, Javen O'Neal <on...@apache.org> wrote:
> Correction: HSLF. This is a ppt/OLE2 file.
>
> On Sun, Aug 14, 2016 at 6:58 PM, Javen O'Neal <on...@apache.org> wrote:
>> Tim,
>>
>> I have extracted the pptx PowerPoint file containing the Prague
>> footer. I'm want to write a unit test for POI to find the Prague
>> string so I can figure why Prague was not included in the Tika
>> regression test using POI 3.15 beta 3 but was found by POI 3.15 beta
>> 1.
>>
>> Could you point me to the Tika code that generated the potential
>> regressions zip file in TIKA-2013, or the POI class/function that is
>> used to extract the text from a document?
>>
>> Also, is the pptx file shareable and ASL 2.0 licensed so that it can
>> be included as part of POI's unit test suite?
>>
>> On Fri, Aug 12, 2016 at 6:52 PM, Javen O'Neal <ja...@gmail.com> wrote:
>>> On Aug 12, 2016 11:39, "Allison, Timothy B." <ta...@mitre.org> wrote:
>>>>...the two potential content regressions may be caused by something at the
>>>> Tika level. If anyone has time to take a look, that'd be great.
>>>
>>> I can take a look this weekend.
>>>
>>> Did you use the same Tika code with different POI versions for these tests
>>> (so that we can attribute the change in behavior to a POI commit, regardless
>>> of whether the bug is in Tika or POI)?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
Re: [VOTE] Apache POI 3.15-beta3
Posted by Javen O'Neal <on...@apache.org>.
Correction: HSLF. This is a ppt/OLE2 file.
On Sun, Aug 14, 2016 at 6:58 PM, Javen O'Neal <on...@apache.org> wrote:
> Tim,
>
> I have extracted the pptx PowerPoint file containing the Prague
> footer. I'm want to write a unit test for POI to find the Prague
> string so I can figure why Prague was not included in the Tika
> regression test using POI 3.15 beta 3 but was found by POI 3.15 beta
> 1.
>
> Could you point me to the Tika code that generated the potential
> regressions zip file in TIKA-2013, or the POI class/function that is
> used to extract the text from a document?
>
> Also, is the pptx file shareable and ASL 2.0 licensed so that it can
> be included as part of POI's unit test suite?
>
> On Fri, Aug 12, 2016 at 6:52 PM, Javen O'Neal <ja...@gmail.com> wrote:
>> On Aug 12, 2016 11:39, "Allison, Timothy B." <ta...@mitre.org> wrote:
>>>...the two potential content regressions may be caused by something at the
>>> Tika level. If anyone has time to take a look, that'd be great.
>>
>> I can take a look this weekend.
>>
>> Did you use the same Tika code with different POI versions for these tests
>> (so that we can attribute the change in behavior to a POI commit, regardless
>> of whether the bug is in Tika or POI)?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
Re: [VOTE] Apache POI 3.15-beta3
Posted by Javen O'Neal <on...@apache.org>.
Tim,
I have extracted the pptx PowerPoint file containing the Prague
footer. I'm want to write a unit test for POI to find the Prague
string so I can figure why Prague was not included in the Tika
regression test using POI 3.15 beta 3 but was found by POI 3.15 beta
1.
Could you point me to the Tika code that generated the potential
regressions zip file in TIKA-2013, or the POI class/function that is
used to extract the text from a document?
Also, is the pptx file shareable and ASL 2.0 licensed so that it can
be included as part of POI's unit test suite?
On Fri, Aug 12, 2016 at 6:52 PM, Javen O'Neal <ja...@gmail.com> wrote:
> On Aug 12, 2016 11:39, "Allison, Timothy B." <ta...@mitre.org> wrote:
>>...the two potential content regressions may be caused by something at the
>> Tika level. If anyone has time to take a look, that'd be great.
>
> I can take a look this weekend.
>
> Did you use the same Tika code with different POI versions for these tests
> (so that we can attribute the change in behavior to a POI commit, regardless
> of whether the bug is in Tika or POI)?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
RE: [VOTE] Apache POI 3.15-beta3
Posted by Javen O'Neal <ja...@gmail.com>.
On Aug 12, 2016 11:39, "Allison, Timothy B." <ta...@mitre.org> wrote:
>...the two potential content regressions may be caused by something at the
Tika level. If anyone has time to take a look, that'd be great.
I can take a look this weekend.
Did you use the same Tika code with different POI versions for these tests
(so that we can attribute the change in behavior to a POI commit,
regardless of whether the bug is in Tika or POI)?
RE: [VOTE] Apache POI 3.15-beta3
Posted by "Allison, Timothy B." <ta...@mitre.org>.
All,
I just finished a comparison of -beta1 and the pre-release -beta3 in Tika (we skipped -beta2). See my post here [1], which includes example docs and a link to the full reports. I haven’t (and probably won't) have a chance to dig into this with pure POI...the two potential content regressions may be caused by something at the Tika level. If anyone has time to take a look, that'd be great.
If not, onward!
Cheers,
Tim
[1] https://issues.apache.org/jira/browse/TIKA-2013?focusedCommentId=15419274&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15419274
-----Original Message-----
From: David North [mailto:dtn-poi@corefiling.co.uk]
Sent: Thursday, August 11, 2016 8:54 AM
To: POI Developers List <de...@poi.apache.org>
Subject: Re: [VOTE] Apache POI 3.15-beta3
Thanks, I'll see if I can fix that tomorrow and issue a new RC
On 11/08/16 13:19, Dominik Stadler wrote:
> Hi,
>
> Unfortunately I have to vote -1 on this candidate, there is a
> directory "trunk" in the src-packages which seems to contain the full
> source again, thus making the source packages 150MB instead of 70 as before.
>
> Inline image 1
>
> Inline image 2
>
> Thanks... Dominik.
>
> On Thu, Aug 11, 2016 at 12:47 AM, David North <dnorth@apache.org
> <ma...@apache.org>> wrote:
>
> Greetings all,
>
> I hereby call the vote on the release of Apache POI 3.15-beta3
>
> The release artifacts are here:
>
> https://dist.apache.org/repos/dist/dev/poi/3.15-beta3-RC1/
> <https://dist.apache.org/repos/dist/dev/poi/3.15-beta3-RC1/>
>
> The vote runs for 72 hours from now, ending 23:49 BST Saturday 13 August
> 2016.
>
> Usual things need to be checked - does it work, does the packaging look
> right.
>
> I'll add my +1 once I've checked this build against our POI-using
> systems at work.
>
> Thanks,
>
> --
> David North | www.dnorth.net <http://www.dnorth.net>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org For additional commands, e-mail: dev-help@poi.apache.org
Re: [VOTE] Apache POI 3.15-beta3
Posted by David North <dt...@corefiling.co.uk>.
Thanks, I'll see if I can fix that tomorrow and issue a new RC
On 11/08/16 13:19, Dominik Stadler wrote:
> Hi,
>
> Unfortunately I have to vote -1 on this candidate, there is a directory
> "trunk" in the src-packages which seems to contain the full source
> again, thus making the source packages 150MB instead of 70 as before.
>
> Inline image 1
>
> Inline image 2
>
> Thanks... Dominik.
>
> On Thu, Aug 11, 2016 at 12:47 AM, David North <dnorth@apache.org
> <ma...@apache.org>> wrote:
>
> Greetings all,
>
> I hereby call the vote on the release of Apache POI 3.15-beta3
>
> The release artifacts are here:
>
> https://dist.apache.org/repos/dist/dev/poi/3.15-beta3-RC1/
> <https://dist.apache.org/repos/dist/dev/poi/3.15-beta3-RC1/>
>
> The vote runs for 72 hours from now, ending 23:49 BST Saturday 13 August
> 2016.
>
> Usual things need to be checked - does it work, does the packaging look
> right.
>
> I'll add my +1 once I've checked this build against our POI-using
> systems at work.
>
> Thanks,
>
> --
> David North | www.dnorth.net <http://www.dnorth.net>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org
Re: [VOTE] Apache POI 3.15-beta3
Posted by Dominik Stadler <do...@gmx.at>.
Hi,
Unfortunately I have to vote -1 on this candidate, there is a directory
"trunk" in the src-packages which seems to contain the full source again,
thus making the source packages 150MB instead of 70 as before.
[image: Inline image 1]
[image: Inline image 2]
Thanks... Dominik.
On Thu, Aug 11, 2016 at 12:47 AM, David North <dn...@apache.org> wrote:
> Greetings all,
>
> I hereby call the vote on the release of Apache POI 3.15-beta3
>
> The release artifacts are here:
>
> https://dist.apache.org/repos/dist/dev/poi/3.15-beta3-RC1/
>
> The vote runs for 72 hours from now, ending 23:49 BST Saturday 13 August
> 2016.
>
> Usual things need to be checked - does it work, does the packaging look
> right.
>
> I'll add my +1 once I've checked this build against our POI-using
> systems at work.
>
> Thanks,
>
> --
> David North | www.dnorth.net
>
>
Re: [VOTE] Apache POI 3.15-beta3
Posted by Javen O'Neal <ja...@gmail.com>.
I updated the site documentation in r1755900. [1]
I tested this release on a POI-powered application at work with no new
issues.
Changing my 0 to a +1.
[1] http://svn.apache.org/viewvc?view=revision&revision=1755900
On Aug 10, 2016 7:33 PM, wrote:
> For those testing this, we have a new runtime dependency: Apache
> commons-collections4, which is subtly documented on [1].
>
> Besides highlighting the new dependency introduced in 3.15 beta 3 on the
> overview page, we need to clearly mention this change in the release notes
> [2] and change log [3].
>
> [1] https://poi.apache.org/overview.html#components
> [2] https://dist.apache.org/repos/dist/dev/poi/3.15-beta3-RC1/
> RELEASE-NOTES.txt
> [3] https://poi.apache.org/changes.html
>
> 0 from me until we fix the website documentation, though no need for an
> RC2 just to update release notes for this beta. However, 3.15 final release
> really should have the new dependency mentioned in the release notes.
>
> On Aug 10, 2016 5:07 PM, "Mark Murphy" <jm...@gmail.com> wrote:
>
> Since I don't have anything in this yet, I will go ahead and +1
>
> On Wed, Aug 10, 2016 at 6:47 PM, David North <dn...@apache.org> wrote:
>
> > Greetings all,
> >
> > I hereby call the vote on the release of Apache POI 3.15-beta3
> >
> > The release artifacts are here:
> >
> > https://dist.apache.org/repos/dist/dev/poi/3.15-beta3-RC1/
> >
> > The vote runs for 72 hours from now, ending 23:49 BST Saturday 13 August
> > 2016.
> >
> > Usual things need to be checked - does it work, does the packaging look
> > right.
> >
> > I'll add my +1 once I've checked this build against our POI-using
> > systems at work.
> >
> > Thanks,
> >
> > --
> > David North | www.dnorth.net
> >
> >
>
>
>
Re: [VOTE] Apache POI 3.15-beta3
Posted by Javen O'Neal <ja...@gmail.com>.
For those testing this, we have a new runtime dependency: Apache
commons-collections4, which is subtly documented on [1].
Besides highlighting the new dependency introduced in 3.15 beta 3 on the
overview page, we need to clearly mention this change in the release notes
[2] and change log [3].
[1] https://poi.apache.org/overview.html#components
[2]
https://dist.apache.org/repos/dist/dev/poi/3.15-beta3-RC1/RELEASE-NOTES.txt
[3] https://poi.apache.org/changes.html
0 from me until we fix the website documentation, though no need for an RC2
just to update release notes for this beta. However, 3.15 final release
really should have the new dependency mentioned in the release notes.
On Aug 10, 2016 5:07 PM, "Mark Murphy" <jm...@gmail.com> wrote:
Since I don't have anything in this yet, I will go ahead and +1
On Wed, Aug 10, 2016 at 6:47 PM, David North <dn...@apache.org> wrote:
> Greetings all,
>
> I hereby call the vote on the release of Apache POI 3.15-beta3
>
> The release artifacts are here:
>
> https://dist.apache.org/repos/dist/dev/poi/3.15-beta3-RC1/
>
> The vote runs for 72 hours from now, ending 23:49 BST Saturday 13 August
> 2016.
>
> Usual things need to be checked - does it work, does the packaging look
> right.
>
> I'll add my +1 once I've checked this build against our POI-using
> systems at work.
>
> Thanks,
>
> --
> David North | www.dnorth.net
>
>
Re: [VOTE] Apache POI 3.15-beta3
Posted by Mark Murphy <jm...@gmail.com>.
Since I don't have anything in this yet, I will go ahead and +1
On Wed, Aug 10, 2016 at 6:47 PM, David North <dn...@apache.org> wrote:
> Greetings all,
>
> I hereby call the vote on the release of Apache POI 3.15-beta3
>
> The release artifacts are here:
>
> https://dist.apache.org/repos/dist/dev/poi/3.15-beta3-RC1/
>
> The vote runs for 72 hours from now, ending 23:49 BST Saturday 13 August
> 2016.
>
> Usual things need to be checked - does it work, does the packaging look
> right.
>
> I'll add my +1 once I've checked this build against our POI-using
> systems at work.
>
> Thanks,
>
> --
> David North | www.dnorth.net
>
>