You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tika.apache.org by Tyler Palsulich <tp...@apache.org> on 2015/04/08 01:54:27 UTC
[VOTE] Release Apache Tika 1.8 Candidate #1
Hi Folks,
A candidate for the Tika 1.8 release is available at:
https://dist.apache.org/repos/dist/dev/tika/
The release candidate is a zip archive of the sources in:
http://svn.apache.org/repos/asf/tika/tags/1.8-rc1/
The SHA1 checksum of the archive is
ddeb3b43ca1c1ef346658a7005434019507e096f.
In addition, a staged maven repository is available here:
https://repository.apache.org/content/repositories/orgapachetika-1008
Please vote on releasing this package as Apache Tika 1.8.
The vote is open for the next 72 hours and passes if a majority of at
least three +1 Tika PMC votes are cast.
[ ] +1 Release this package as Apache Tika 1.8
[ ] -1 Do not release this package because...
Have a good night!
Tyler
Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by Konstantin Gribov <gr...@gmail.com>.
Hi, all.
All tests pass (w/ tesseract) on Arch Linux, Oracle JDK 7u75.
[x] +1 Release this package as Apache Tika 1.8
[ ] -1 Do not release this package because...
Also, I've found strange behavior while testing this RC on some RTFs. See
https://issues.apache.org/jira/browse/TIKA-1597. But I think that it should
not prevent releasing this RC.
--
Best regards,
Konstantin Gribov
ср, 8 апр. 2015 г. в 9:07, Oleg Tikhonov <ol...@apache.org>:
> Hi,
> [x] +1 Release this package as Apache Tika 1.8.
>
> Tested on: Ubuntu 14.10, x86_64. Java 1.7 (Oracle)
> Don't we want to update the following dependencies:
> biz.aQute:bndlib ..................... 1.43.0 -> 2.0.0.20130123-133441
> org.apache.felix:org.apache.felix.scr.annotations .... 1.6.0 -> 1.9.10
> org.osgi:org.osgi.compendium .......................... 4.0.0 -> 5.0.0
> org.osgi:org.osgi.core ................................ 4.0.0 -> 6.0.0
> com.drewnoakes:metadata-extractor ..................... 2.7.2 -> 2.8.0
> com.google.guava:guava ................................ 10.0.1 -> 18.0
> edu.ucar:grib ........................................ 4.5.5 -> 8.0.29
> org.ow2.asm:asm-debug-all ............................... 4.1 -> 5.0.3
> commons-io:commons-io ..................................... 2.1 -> 2.4
> javax.mail:mail ................................... 1.4.4 -> 1.5.0-b01
> org.apache.cxf:cxf-rt-frontend-jaxrs .................. 2.7.8 -> 3.0.4
>
> BR,
> Oleg
>
>
>
>
> On Wed, Apr 8, 2015 at 2:55 AM, Tyler Palsulich <tp...@apache.org>
> wrote:
>
> > CC'ing user@tika for visibility.
> >
> > Tyler
> >
> > On Tue, Apr 7, 2015 at 4:54 PM, Tyler Palsulich <tp...@apache.org>
> > wrote:
> >
> > > Hi Folks,
> > >
> > > A candidate for the Tika 1.8 release is available at:
> > > https://dist.apache.org/repos/dist/dev/tika/
> > >
> > > The release candidate is a zip archive of the sources in:
> > > http://svn.apache.org/repos/asf/tika/tags/1.8-rc1/
> > >
> > > The SHA1 checksum of the archive is
> > > ddeb3b43ca1c1ef346658a7005434019507e096f.
> > >
> > > In addition, a staged maven repository is available here:
> > > https://repository.apache.org/content/repositories/
> orgapachetika-1008
> > >
> > > Please vote on releasing this package as Apache Tika 1.8.
> > > The vote is open for the next 72 hours and passes if a majority of at
> > > least three +1 Tika PMC votes are cast.
> > >
> > > [ ] +1 Release this package as Apache Tika 1.8
> > > [ ] -1 Do not release this package because...
> > >
> > > Have a good night!
> > > Tyler
> > >
> >
>
Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by Oleg Tikhonov <ol...@apache.org>.
Hi,
[x] +1 Release this package as Apache Tika 1.8.
Tested on: Ubuntu 14.10, x86_64. Java 1.7 (Oracle)
Don't we want to update the following dependencies:
biz.aQute:bndlib ..................... 1.43.0 -> 2.0.0.20130123-133441
org.apache.felix:org.apache.felix.scr.annotations .... 1.6.0 -> 1.9.10
org.osgi:org.osgi.compendium .......................... 4.0.0 -> 5.0.0
org.osgi:org.osgi.core ................................ 4.0.0 -> 6.0.0
com.drewnoakes:metadata-extractor ..................... 2.7.2 -> 2.8.0
com.google.guava:guava ................................ 10.0.1 -> 18.0
edu.ucar:grib ........................................ 4.5.5 -> 8.0.29
org.ow2.asm:asm-debug-all ............................... 4.1 -> 5.0.3
commons-io:commons-io ..................................... 2.1 -> 2.4
javax.mail:mail ................................... 1.4.4 -> 1.5.0-b01
org.apache.cxf:cxf-rt-frontend-jaxrs .................. 2.7.8 -> 3.0.4
BR,
Oleg
On Wed, Apr 8, 2015 at 2:55 AM, Tyler Palsulich <tp...@apache.org>
wrote:
> CC'ing user@tika for visibility.
>
> Tyler
>
> On Tue, Apr 7, 2015 at 4:54 PM, Tyler Palsulich <tp...@apache.org>
> wrote:
>
> > Hi Folks,
> >
> > A candidate for the Tika 1.8 release is available at:
> > https://dist.apache.org/repos/dist/dev/tika/
> >
> > The release candidate is a zip archive of the sources in:
> > http://svn.apache.org/repos/asf/tika/tags/1.8-rc1/
> >
> > The SHA1 checksum of the archive is
> > ddeb3b43ca1c1ef346658a7005434019507e096f.
> >
> > In addition, a staged maven repository is available here:
> > https://repository.apache.org/content/repositories/orgapachetika-1008
> >
> > Please vote on releasing this package as Apache Tika 1.8.
> > The vote is open for the next 72 hours and passes if a majority of at
> > least three +1 Tika PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Tika 1.8
> > [ ] -1 Do not release this package because...
> >
> > Have a good night!
> > Tyler
> >
>
Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by Tyler Palsulich <tp...@apache.org>.
CC'ing user@tika for visibility.
Tyler
On Tue, Apr 7, 2015 at 4:54 PM, Tyler Palsulich <tp...@apache.org>
wrote:
> Hi Folks,
>
> A candidate for the Tika 1.8 release is available at:
> https://dist.apache.org/repos/dist/dev/tika/
>
> The release candidate is a zip archive of the sources in:
> http://svn.apache.org/repos/asf/tika/tags/1.8-rc1/
>
> The SHA1 checksum of the archive is
> ddeb3b43ca1c1ef346658a7005434019507e096f.
>
> In addition, a staged maven repository is available here:
> https://repository.apache.org/content/repositories/orgapachetika-1008
>
> Please vote on releasing this package as Apache Tika 1.8.
> The vote is open for the next 72 hours and passes if a majority of at
> least three +1 Tika PMC votes are cast.
>
> [ ] +1 Release this package as Apache Tika 1.8
> [ ] -1 Do not release this package because...
>
> Have a good night!
> Tyler
>
Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by Hong-Thai Nguyen <th...@gmail.com>.
Hi folks,
-1 for me.
I've checked with an integration tests on our same corpus to compare with
recent: Tika 1.7, Poi 3.11, pdfbox 1.8.8. Here's result:
===================== TIKA 1.8 poi 3.12 beta-1 pdfbox 1.8.9
========================================
15-04-09 18:29:46 INFO (DocumentConversionErrorPlugin.java : 117)
[pool-3-thread-1] Summary of document conversion errors:
- pdf (5)
* (3) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: TIKA-198: Illegal IOException from
org.apache.tika.parser.ParserDecorator$1@4c9f1e40
* (1) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@4c9f1e40
* (1) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unable to extract PDF content
- pptx (10)
* (1) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@6b03495a
* (9) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Error creating OOXML extractor
- doc (2)
* (2) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@6ca94962
- ppt (14)
* (13) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@6ca94962
* (1) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: TIKA-198: Illegal IOException from
org.apache.tika.parser.ParserDecorator$1@6ca94962
- xls (9)
* (9) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@6ca94962
- chm (2)
* (2) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: CHM file extract error: extracted
Length is wrong.
- dwg (4)
* (4) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unsupported AutoCAD drawing
version: AC1014
- odp (6)
* (4) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@6457a807
* (2) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: TIKA-198: Illegal IOException from
org.apache.tika.parser.ParserDecorator$1@6457a807
- odt (217)
* (217) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@6457a807
- pps (2)
* (2) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@6ca94962
- chw (1)
* (1) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@5485ccc1
============= Tika 1.7, Poi 3.11, pdfbox 1.8.8
=================================================
15-01-16 18:05:40 INFO (DocumentConversionErrorPlugin.java : 116)
[pool-3-thread-2] Summary of document conversion errors:
- pdf (4)
* (2) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: TIKA-198: Illegal IOException from
org.apache.tika.parser.ParserDecorator$1@796cd440
* (1) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@796cd440
* (1) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unable to extract PDF content
- pptx (10)
* (9) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Error creating OOXML extractor
* (1) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@4ef4bfa8
- doc (6)
* (6) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@40937aa3
- ppt (14)
* (1) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: TIKA-198: Illegal IOException from
org.apache.tika.parser.ParserDecorator$1@40937aa3
* (13) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@40937aa3
- xls (9)
* (9) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@40937aa3
- chm (2)
* (1) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: CHM file extract error: extracted
Length is wrong.
* (1) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: 22
- dwg (4)
* (4) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unsupported AutoCAD drawing
version: AC1014
- odp (2)
* (2) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: TIKA-198: Illegal IOException from
org.apache.tika.parser.ParserDecorator$1@4d12dad5
- pps (2)
* (2) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@40937aa3
- chw (1)
* (1) com.polyspot.document.converter.ConversionException:
org.apache.tika.exception.TikaException: Unexpected RuntimeException from
org.apache.tika.parser.ParserDecorator$1@3ef505f
It seems that we have a big regression on ODT files. I tested manually some
of failed odt files with tika-app (sample file:
https://drive.google.com/file/d/0B8g_W4fhTMCJNmpVbG12QW05UWM/view?usp=sharing),
here's the stack trace:
Apache Tika was unable to parse the document
at C:\Users\hong-thai.nguyen\Downloads\Manuel_koha.odt.
The full exception stack trace is included below:
org.apache.tika.exception.TikaException: Failed to close temporary resources
at
org.apache.tika.io.TemporaryResources.dispose(TemporaryResources.java:152)
at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:127)
at org.apache.tika.gui.TikaGUI.handleStream(TikaGUI.java:342)
at org.apache.tika.gui.TikaGUI.openFile(TikaGUI.java:299)
at org.apache.tika.gui.TikaGUI.actionPerformed(TikaGUI.java:256)
at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
at javax.swing.AbstractButton.doClick(Unknown Source)
at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source)
at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown
Source)
at java.awt.Component.processMouseEvent(Unknown Source)
at javax.swing.JComponent.processMouseEvent(Unknown Source)
at java.awt.Component.processEvent(Unknown Source)
at java.awt.Container.processEvent(Unknown Source)
at java.awt.Component.dispatchEventImpl(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Window.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
at java.awt.EventQueue.access$400(Unknown Source)
at java.awt.EventQueue$3.run(Unknown Source)
at java.awt.EventQueue$3.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue$4.run(Unknown Source)
at java.awt.EventQueue$4.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)
Caused by: java.io.IOException: Could not delete temporary file
C:\Users\HONG-T~1.NGU\AppData\Local\Temp\apache-tika-2891340188156641845.tmp
at org.apache.tika.io.TemporaryResources$1.close(TemporaryResources.java:70)
at org.apache.tika.io.TemporaryResources.close(TemporaryResources.java:121)
at
org.apache.tika.io.TemporaryResources.dispose(TemporaryResources.java:150)
... 42 more
Anyone have same problem ?
Regards,
On Wed, Apr 8, 2015 at 1:54 AM, Tyler Palsulich <tp...@apache.org>
wrote:
> Hi Folks,
>
> A candidate for the Tika 1.8 release is available at:
> https://dist.apache.org/repos/dist/dev/tika/
>
> The release candidate is a zip archive of the sources in:
> http://svn.apache.org/repos/asf/tika/tags/1.8-rc1/
>
> The SHA1 checksum of the archive is
> ddeb3b43ca1c1ef346658a7005434019507e096f.
>
> In addition, a staged maven repository is available here:
> https://repository.apache.org/content/repositories/orgapachetika-1008
>
> Please vote on releasing this package as Apache Tika 1.8.
> The vote is open for the next 72 hours and passes if a majority of at
> least three +1 Tika PMC votes are cast.
>
> [ ] +1 Release this package as Apache Tika 1.8
> [ ] -1 Do not release this package because...
>
> Have a good night!
> Tyler
>
--
--------------
Hong-Thai
Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by Tyler Palsulich <tp...@apache.org>.
CC'ing user@tika for visibility.
Tyler
On Tue, Apr 7, 2015 at 4:54 PM, Tyler Palsulich <tp...@apache.org>
wrote:
> Hi Folks,
>
> A candidate for the Tika 1.8 release is available at:
> https://dist.apache.org/repos/dist/dev/tika/
>
> The release candidate is a zip archive of the sources in:
> http://svn.apache.org/repos/asf/tika/tags/1.8-rc1/
>
> The SHA1 checksum of the archive is
> ddeb3b43ca1c1ef346658a7005434019507e096f.
>
> In addition, a staged maven repository is available here:
> https://repository.apache.org/content/repositories/orgapachetika-1008
>
> Please vote on releasing this package as Apache Tika 1.8.
> The vote is open for the next 72 hours and passes if a majority of at
> least three +1 Tika PMC votes are cast.
>
> [ ] +1 Release this package as Apache Tika 1.8
> [ ] -1 Do not release this package because...
>
> Have a good night!
> Tyler
>
Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by Tyler Palsulich <tp...@gmail.com>.
Hi Folks,
Marking this VOTE as failed. Now that the above issues have been addressed,
I'll cut a new release.
Please let me know if you find any other blockers.
Thanks,
Tyler
On Mon, Apr 13, 2015 at 12:45 AM, Hong-Thai Nguyen <
hnguyen@customermatrix.com> wrote:
> Not yet, I'm investigating more on TIKA-1600 today.
>
> Hong-Thai
>
> -----Message d'origine-----
> De : Allison, Timothy B. [mailto:tallison@mitre.org]
> Envoyé : lundi 13 avril 2015 01:07
> À : dev@tika.apache.org
> Objet : RE: [VOTE] Release Apache Tika 1.8 Candidate #1
>
> I don't think we've solved TIKA-1600, yet, or have we?
>
> -----Original Message-----
> From: Tyler Palsulich [mailto:tpalsulich@gmail.com]
> Sent: Sunday, April 12, 2015 12:12 AM
> To: dev@tika.apache.org
> Subject: Re: [VOTE] Release Apache Tika 1.8 Candidate #1
>
> Are we ready for another RC? I'd like to make sure the above issues are
> (believed to be) settled before the next cut.
>
> Thanks,
> Tyler
> On Apr 10, 2015 4:55 PM, "David Meikle" <lo...@gmail.com> wrote:
>
> >
> > > On 10 Apr 2015, at 11:38, Allison, Timothy B. <ta...@mitre.org>
> > wrote:
> > >
> > > I agree that the ODT issue might require a respin. What do others
> > think?
> >
> > +1 for re-spin.
> >
> > >
> > > Unfortunately, there might be 2 odt docs (mime type:
> > “application/vnd.oasis.opendocument.text”?) in govdocs1…so we wouldn't
> > see that problem.
> > >
> > >
> > >
> > > I did do a comparison of 1.7 vs 1.8-rc1, and the results are here:
> > >
> > >
> > https://github.com/tballison/share/blob/master/tika_comparisons/tika_1
> > _7_v_1_8-rc1.zip
> > <
> > https://github.com/tballison/share/blob/master/tika_comparisons/tika_1
> > _7_v_1_8-rc1.zip
> > >
> > >
> > > I encourage folks (if you haven't, and if you care :) ) to take a
> > > look
> > and see if you see something that I don’t.
> >
> > Thanks for this Tim. About to get on a flight, so will check through
> > on that.
> >
> > Cheers,
> > Dave
> >
> >
>
RE: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by Hong-Thai Nguyen <hn...@customermatrix.com>.
Not yet, I'm investigating more on TIKA-1600 today.
Hong-Thai
-----Message d'origine-----
De : Allison, Timothy B. [mailto:tallison@mitre.org]
Envoyé : lundi 13 avril 2015 01:07
À : dev@tika.apache.org
Objet : RE: [VOTE] Release Apache Tika 1.8 Candidate #1
I don't think we've solved TIKA-1600, yet, or have we?
-----Original Message-----
From: Tyler Palsulich [mailto:tpalsulich@gmail.com]
Sent: Sunday, April 12, 2015 12:12 AM
To: dev@tika.apache.org
Subject: Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Are we ready for another RC? I'd like to make sure the above issues are (believed to be) settled before the next cut.
Thanks,
Tyler
On Apr 10, 2015 4:55 PM, "David Meikle" <lo...@gmail.com> wrote:
>
> > On 10 Apr 2015, at 11:38, Allison, Timothy B. <ta...@mitre.org>
> wrote:
> >
> > I agree that the ODT issue might require a respin. What do others
> think?
>
> +1 for re-spin.
>
> >
> > Unfortunately, there might be 2 odt docs (mime type:
> “application/vnd.oasis.opendocument.text”?) in govdocs1…so we wouldn't
> see that problem.
> >
> >
> >
> > I did do a comparison of 1.7 vs 1.8-rc1, and the results are here:
> >
> >
> https://github.com/tballison/share/blob/master/tika_comparisons/tika_1
> _7_v_1_8-rc1.zip
> <
> https://github.com/tballison/share/blob/master/tika_comparisons/tika_1
> _7_v_1_8-rc1.zip
> >
> >
> > I encourage folks (if you haven't, and if you care :) ) to take a
> > look
> and see if you see something that I don’t.
>
> Thanks for this Tim. About to get on a flight, so will check through
> on that.
>
> Cheers,
> Dave
>
>
RE: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by "Allison, Timothy B." <ta...@mitre.org>.
I don't think we've solved TIKA-1600, yet, or have we?
-----Original Message-----
From: Tyler Palsulich [mailto:tpalsulich@gmail.com]
Sent: Sunday, April 12, 2015 12:12 AM
To: dev@tika.apache.org
Subject: Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Are we ready for another RC? I'd like to make sure the above issues are
(believed to be) settled before the next cut.
Thanks,
Tyler
On Apr 10, 2015 4:55 PM, "David Meikle" <lo...@gmail.com> wrote:
>
> > On 10 Apr 2015, at 11:38, Allison, Timothy B. <ta...@mitre.org>
> wrote:
> >
> > I agree that the ODT issue might require a respin. What do others
> think?
>
> +1 for re-spin.
>
> >
> > Unfortunately, there might be 2 odt docs (mime type:
> “application/vnd.oasis.opendocument.text”?) in govdocs1…so we wouldn't see
> that problem.
> >
> >
> >
> > I did do a comparison of 1.7 vs 1.8-rc1, and the results are here:
> >
> >
> https://github.com/tballison/share/blob/master/tika_comparisons/tika_1_7_v_1_8-rc1.zip
> <
> https://github.com/tballison/share/blob/master/tika_comparisons/tika_1_7_v_1_8-rc1.zip
> >
> >
> > I encourage folks (if you haven't, and if you care :) ) to take a look
> and see if you see something that I don’t.
>
> Thanks for this Tim. About to get on a flight, so will check through on
> that.
>
> Cheers,
> Dave
>
>
Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.
+1, will hope to VOTE on that one! :)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Tyler Palsulich <tp...@gmail.com>
Reply-To: "dev@tika.apache.org" <de...@tika.apache.org>
Date: Saturday, April 11, 2015 at 9:12 PM
To: "dev@tika.apache.org" <de...@tika.apache.org>
Subject: Re: [VOTE] Release Apache Tika 1.8 Candidate #1
>Are we ready for another RC? I'd like to make sure the above issues are
>(believed to be) settled before the next cut.
>
>Thanks,
>Tyler
>On Apr 10, 2015 4:55 PM, "David Meikle" <lo...@gmail.com> wrote:
>
>>
>> > On 10 Apr 2015, at 11:38, Allison, Timothy B. <ta...@mitre.org>
>> wrote:
>> >
>> > I agree that the ODT issue might require a respin. What do others
>> think?
>>
>> +1 for re-spin.
>>
>> >
>> > Unfortunately, there might be 2 odt docs (mime type:
>> “application/vnd.oasis.opendocument.text”?) in govdocs1…so we wouldn't
>>see
>> that problem.
>> >
>> >
>> >
>> > I did do a comparison of 1.7 vs 1.8-rc1, and the results are here:
>> >
>> >
>>
>>https://github.com/tballison/share/blob/master/tika_comparisons/tika_1_7_
>>v_1_8-rc1.zip
>> <
>>
>>https://github.com/tballison/share/blob/master/tika_comparisons/tika_1_7_
>>v_1_8-rc1.zip
>> >
>> >
>> > I encourage folks (if you haven't, and if you care :) ) to take a look
>> and see if you see something that I don’t.
>>
>> Thanks for this Tim. About to get on a flight, so will check through on
>> that.
>>
>> Cheers,
>> Dave
>>
>>
Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by Tyler Palsulich <tp...@gmail.com>.
Are we ready for another RC? I'd like to make sure the above issues are
(believed to be) settled before the next cut.
Thanks,
Tyler
On Apr 10, 2015 4:55 PM, "David Meikle" <lo...@gmail.com> wrote:
>
> > On 10 Apr 2015, at 11:38, Allison, Timothy B. <ta...@mitre.org>
> wrote:
> >
> > I agree that the ODT issue might require a respin. What do others
> think?
>
> +1 for re-spin.
>
> >
> > Unfortunately, there might be 2 odt docs (mime type:
> “application/vnd.oasis.opendocument.text”?) in govdocs1…so we wouldn't see
> that problem.
> >
> >
> >
> > I did do a comparison of 1.7 vs 1.8-rc1, and the results are here:
> >
> >
> https://github.com/tballison/share/blob/master/tika_comparisons/tika_1_7_v_1_8-rc1.zip
> <
> https://github.com/tballison/share/blob/master/tika_comparisons/tika_1_7_v_1_8-rc1.zip
> >
> >
> > I encourage folks (if you haven't, and if you care :) ) to take a look
> and see if you see something that I don’t.
>
> Thanks for this Tim. About to get on a flight, so will check through on
> that.
>
> Cheers,
> Dave
>
>
Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by David Meikle <lo...@gmail.com>.
> On 10 Apr 2015, at 11:38, Allison, Timothy B. <ta...@mitre.org> wrote:
>
> I agree that the ODT issue might require a respin. What do others think?
+1 for re-spin.
>
> Unfortunately, there might be 2 odt docs (mime type: “application/vnd.oasis.opendocument.text”?) in govdocs1…so we wouldn't see that problem.
>
>
>
> I did do a comparison of 1.7 vs 1.8-rc1, and the results are here:
>
> https://github.com/tballison/share/blob/master/tika_comparisons/tika_1_7_v_1_8-rc1.zip <https://github.com/tballison/share/blob/master/tika_comparisons/tika_1_7_v_1_8-rc1.zip>
>
> I encourage folks (if you haven't, and if you care :) ) to take a look and see if you see something that I don’t.
Thanks for this Tim. About to get on a flight, so will check through on that.
Cheers,
Dave
Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by Konstantin Gribov <gr...@gmail.com>.
Hi, folks.
I have to change my vote to -1, because of TIKA-1597. I'll apply fix
suggested by Tim to trunk.
--
Best regards,
Konstantin Gribov
пт, 10 апр. 2015 г. в 18:39, Allison, Timothy B. <ta...@mitre.org>:
> Hi Tyler,
>
> I agree that the ODT issue might require a respin. What do others think?
>
>
>
> Unfortunately, there might be 2 odt docs (mime type:
> “application/vnd.oasis.opendocument.text”?) in govdocs1…so we wouldn't
> see that problem.
>
>
>
> I did do a comparison of 1.7 vs 1.8-rc1, and the results are here:
>
> https://github.com/tballison/share/blob/master/tika_
> comparisons/tika_1_7_v_1_8-rc1.zip
>
>
>
> I encourage folks (if you haven't, and if you care :) ) to take a look and
> see if you see something that I don’t.
>
>
>
> This is a list of counts of mime-types detected with 1.8-rc1:
>
>
> DETECTED_CONTENT_TYPE_B <http://localhost:8082/query.do?jsessionid=
> 71edd928846d15aa0f0a4aad16922326>
>
> CNT <http://localhost:8082/query.do?jsessionid=
> 71edd928846d15aa0f0a4aad16922326>
>
> application/pdf
>
> 230627
>
> image/jpeg
>
> 109180
>
> text/html; charset=ISO-8859-1
>
> 86263
>
> application/msword
>
> 76946
>
> text/plain; charset=ISO-8859-1
>
> 64124
>
> application/vnd.ms-excel
>
> 58939
>
> application/vnd.ms-powerpoint
>
> 50615
>
> text/html; charset=windows-1252
>
> 50224
>
> text/plain; charset=windows-1252
>
> 42952
>
> text/html; charset=UTF-8
>
> 40983
>
> image/gif
>
> 36279
>
> application/xhtml+xml
>
> 34782
>
> application/xml
>
> 34124
>
> application/postscript
>
> 26579
>
> application/gzip
>
> 13722
>
> image/png
>
> 4193
>
> application/x-shockwave-flash
>
> 3473
>
> application/vnd.ms-excel.sheet.4
>
> 2574
>
> application/octet-stream
>
> 2386
>
> application/rtf
>
> 1117
>
> text/html; charset=US-ASCII
>
> 1087
>
> application/fits
>
> 1055
>
> message/rfc822
>
> 998
>
> application/vnd.google-earth.kml+xml
>
> 980
>
> application/zip
>
> 636
>
> model/vnd.dwf
>
> 299
>
> text/x-java-source
>
> 289
>
> text/plain; charset=UTF-8
>
> 287
>
> text/plain; charset=GB18030
>
> 273
>
> application/vnd.openxmlformats-officedocument.presentationml.presentation
>
> 217
>
> application/rss+xml
>
> 187
>
> application/vnd.ms-excel.sheet.3
>
> 175
>
> application/msword2
>
> 165
>
> application/vnd.openxmlformats-officedocument.wordprocessingml.document
>
> 163
>
> text/html; charset=GB18030
>
> 112
>
> application/x-tika-msoffice
>
> 107
>
> application/rdf+xml
>
> 105
>
> text/html; charset=UTF-16
>
> 80
>
> image/x-ms-bmp
>
> 73
>
> application/x-tex
>
> 59
>
> text/html; charset=windows-1250
>
> 55
>
> text/html; charset=windows-1251
>
> 48
>
> application/x-tika-msoffice-embedded; format=ole10_native
>
> 47
>
> application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
>
> 34
>
> application/x-msmetafile
>
> 33
>
> image/tiff
>
> 31
>
> text/plain; charset=ISO-8859-15
>
> 31
>
> text/html; charset=UTF-16LE
>
> 29
>
> application/vnd.framemaker
>
> 25
>
> application/x-tika-msoffice-embedded; format=comp_obj
>
> 25
>
> text/html; charset=IBM437
>
> 23
>
> text/plain; charset=windows-1250
>
> 23
>
> image/x-portable-bitmap
>
> 15
>
> text/plain; charset=EUC-KR
>
> 14
>
> text/html; charset=GB2312
>
> 13
>
> text/plain; charset=KOI8-R
>
> 13
>
> text/html; charset=windows-1256
>
> 13
>
> application/x-123
>
> 12
>
> text/html; charset=Shift_JIS
>
> 12
>
> application/vnd.stardivision.calc
>
> 12
>
> text/plain; charset=Shift_JIS
>
> 11
>
> text/html; charset=EUC-KR
>
> 11
>
> text/html; charset=IBM866
>
> 10
>
> text/plain; charset=windows-1255
>
> 10
>
> text/html; charset=ISO-8859-15
>
> 9
>
> text/dif+xml
>
> 9
>
> text/html; charset=EUC-JP
>
> 9
>
> text/plain; charset=ISO-8859-5
>
> 9
>
> application/x-bibtex-text-file
>
> 9
>
> text/html; charset=ISO-2022-JP
>
> 8
>
> application/vnd.stardivision.draw
>
> 7
>
> video/x-ms-wmv
>
> 6
>
> text/plain; charset=IBM866
>
> 6
>
> application/x-emf
>
> 6
>
> text/plain; charset=IBM855
>
> 6
>
> application/xspf+xml
>
> 5
>
> null
>
> 5
>
> text/plain; charset=UTF-16LE
>
> 5
>
> application/x-compress
>
> 5
>
> text/plain; charset=ISO-8859-2
>
> 5
>
> text/html; charset=ISO-8859-2
>
> 5
>
> text/html; charset=windows-1254
>
> 4
>
> text/plain; charset=Big5
>
> 4
>
> text/html; charset=Big5
>
> 4
>
> application/xslt+xml
>
> 3
>
> text/plain; charset=ISO-2022-JP
>
> 3
>
> application/vnd.visio
>
> 3
>
> image/x-pict
>
> 3
>
> application/x-tika-msworks-spreadsheet
>
> 2
>
> text/html; charset=ISO-8859-9
>
> 2
>
> text/html; charset=ISO-8859-7
>
> 2
>
> application/vnd.sun.xml.impress
>
> 2
>
> image/vnd.adobe.photoshop
>
> 2
>
> text/html; charset=KOI8-R
>
> 2
>
> application/x-elc
>
> 2
>
> text/plain; charset=ISO-8859-8
>
> 2
>
> application/vnd.fdf
>
> 2
>
> application/vnd.oasis.opendocument.text
>
> 2
>
> text/plain; charset=ISO-8859-7
>
> 2
>
> image/g3fax
>
> 2
>
> image/vnd.dwg
>
> 2
>
> application/vnd.stardivision.writer
>
> 2
>
> application/vnd.openxmlformats-officedocument.presentationml.slideshow
>
> 1
>
> application/x-mspublisher
>
> 1
>
> image/x-portable-pixmap
>
> 1
>
> application/x-grib
>
> 1
>
> text/plain; charset=UTF-16BE
>
> 1
>
> text/html; charset=windows-1258
>
> 1
>
> application/x-executable
>
> 1
>
> text/html; charset=windows-1255
>
> 1
>
> image/vnd.microsoft.icon
>
> 1
>
> application/x-stuffit
>
> 1
>
> application/vnd.ms-word.document.macroenabled.12
>
> 1
>
> text/plain; charset=windows-1253
>
> 1
>
> text/plain; charset=windows-1251
>
> 1
>
> audio/x-ms-wma
>
> 1
>
> application/x-iso9660-image
>
> 1
>
> text/html; charset=IBM855
>
> 1
>
> text/html; charset=ISO-8859-5
>
> 1
>
> text/html; charset=ISO-8859-6
>
> 1
>
> application/x-archive
>
> 1
>
> application/msword5
>
> 1
>
> text/html; charset=IBM500
>
> 1
>
> application/vnd.ms-xpsdocument
>
> 1
>
> text/plain; charset=ISO-8859-9
>
> 1
>
> text/html; charset=UTF-32LE
>
> 1
>
> application/dita+xml; format=topic
>
> 1
>
> application/vnd.ms-powerpoint.presentation.macroenabled.12
>
> 1
>
> text/plain; charset=IBM500
>
> 1
>
>
>
>
>
>
RE: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by "Allison, Timothy B." <ta...@mitre.org>.
Hi Tyler,
I agree that the ODT issue might require a respin. What do others think?
Unfortunately, there might be 2 odt docs (mime type: “application/vnd.oasis.opendocument.text”?) in govdocs1…so we wouldn't see that problem.
I did do a comparison of 1.7 vs 1.8-rc1, and the results are here:
https://github.com/tballison/share/blob/master/tika_comparisons/tika_1_7_v_1_8-rc1.zip
I encourage folks (if you haven't, and if you care :) ) to take a look and see if you see something that I don’t.
This is a list of counts of mime-types detected with 1.8-rc1:
DETECTED_CONTENT_TYPE_B <http://localhost:8082/query.do?jsessionid=71edd928846d15aa0f0a4aad16922326>
CNT <http://localhost:8082/query.do?jsessionid=71edd928846d15aa0f0a4aad16922326>
application/pdf
230627
image/jpeg
109180
text/html; charset=ISO-8859-1
86263
application/msword
76946
text/plain; charset=ISO-8859-1
64124
application/vnd.ms-excel
58939
application/vnd.ms-powerpoint
50615
text/html; charset=windows-1252
50224
text/plain; charset=windows-1252
42952
text/html; charset=UTF-8
40983
image/gif
36279
application/xhtml+xml
34782
application/xml
34124
application/postscript
26579
application/gzip
13722
image/png
4193
application/x-shockwave-flash
3473
application/vnd.ms-excel.sheet.4
2574
application/octet-stream
2386
application/rtf
1117
text/html; charset=US-ASCII
1087
application/fits
1055
message/rfc822
998
application/vnd.google-earth.kml+xml
980
application/zip
636
model/vnd.dwf
299
text/x-java-source
289
text/plain; charset=UTF-8
287
text/plain; charset=GB18030
273
application/vnd.openxmlformats-officedocument.presentationml.presentation
217
application/rss+xml
187
application/vnd.ms-excel.sheet.3
175
application/msword2
165
application/vnd.openxmlformats-officedocument.wordprocessingml.document
163
text/html; charset=GB18030
112
application/x-tika-msoffice
107
application/rdf+xml
105
text/html; charset=UTF-16
80
image/x-ms-bmp
73
application/x-tex
59
text/html; charset=windows-1250
55
text/html; charset=windows-1251
48
application/x-tika-msoffice-embedded; format=ole10_native
47
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
34
application/x-msmetafile
33
image/tiff
31
text/plain; charset=ISO-8859-15
31
text/html; charset=UTF-16LE
29
application/vnd.framemaker
25
application/x-tika-msoffice-embedded; format=comp_obj
25
text/html; charset=IBM437
23
text/plain; charset=windows-1250
23
image/x-portable-bitmap
15
text/plain; charset=EUC-KR
14
text/html; charset=GB2312
13
text/plain; charset=KOI8-R
13
text/html; charset=windows-1256
13
application/x-123
12
text/html; charset=Shift_JIS
12
application/vnd.stardivision.calc
12
text/plain; charset=Shift_JIS
11
text/html; charset=EUC-KR
11
text/html; charset=IBM866
10
text/plain; charset=windows-1255
10
text/html; charset=ISO-8859-15
9
text/dif+xml
9
text/html; charset=EUC-JP
9
text/plain; charset=ISO-8859-5
9
application/x-bibtex-text-file
9
text/html; charset=ISO-2022-JP
8
application/vnd.stardivision.draw
7
video/x-ms-wmv
6
text/plain; charset=IBM866
6
application/x-emf
6
text/plain; charset=IBM855
6
application/xspf+xml
5
null
5
text/plain; charset=UTF-16LE
5
application/x-compress
5
text/plain; charset=ISO-8859-2
5
text/html; charset=ISO-8859-2
5
text/html; charset=windows-1254
4
text/plain; charset=Big5
4
text/html; charset=Big5
4
application/xslt+xml
3
text/plain; charset=ISO-2022-JP
3
application/vnd.visio
3
image/x-pict
3
application/x-tika-msworks-spreadsheet
2
text/html; charset=ISO-8859-9
2
text/html; charset=ISO-8859-7
2
application/vnd.sun.xml.impress
2
image/vnd.adobe.photoshop
2
text/html; charset=KOI8-R
2
application/x-elc
2
text/plain; charset=ISO-8859-8
2
application/vnd.fdf
2
application/vnd.oasis.opendocument.text
2
text/plain; charset=ISO-8859-7
2
image/g3fax
2
image/vnd.dwg
2
application/vnd.stardivision.writer
2
application/vnd.openxmlformats-officedocument.presentationml.slideshow
1
application/x-mspublisher
1
image/x-portable-pixmap
1
application/x-grib
1
text/plain; charset=UTF-16BE
1
text/html; charset=windows-1258
1
application/x-executable
1
text/html; charset=windows-1255
1
image/vnd.microsoft.icon
1
application/x-stuffit
1
application/vnd.ms-word.document.macroenabled.12
1
text/plain; charset=windows-1253
1
text/plain; charset=windows-1251
1
audio/x-ms-wma
1
application/x-iso9660-image
1
text/html; charset=IBM855
1
text/html; charset=ISO-8859-5
1
text/html; charset=ISO-8859-6
1
application/x-archive
1
application/msword5
1
text/html; charset=IBM500
1
application/vnd.ms-xpsdocument
1
text/plain; charset=ISO-8859-9
1
text/html; charset=UTF-32LE
1
application/dita+xml; format=topic
1
application/vnd.ms-powerpoint.presentation.macroenabled.12
1
text/plain; charset=IBM500
1
Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by Tyler Palsulich <tp...@gmail.com>.
Hi All,
RE: charset, +0. I don't know enough about standards and use cases to say
what the correct behavior is.
RE: ODT files, can we run 1.8-RC1 against govdocs and see if there are any
broader regressions? A major ODT parsing regression seems like a blocking
issue.
Of course, it's not a problem to roll another RC if we want to fix either
issue!
Tyler
On Fri, Apr 10, 2015 at 6:07 AM, Allison, Timothy B. <ta...@mitre.org>
wrote:
> +1
>
> I'm happy to make the fix if anyone thinks we need to respin to modify
> this change in 1.8:
>
> text/html; charset=iso-8859-1 -> application/xhtml+xml
>
> Thank you, Tyler!
>
> Best,
>
> Tim
>
RE: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by "Allison, Timothy B." <ta...@mitre.org>.
+1
I'm happy to make the fix if anyone thinks we need to respin to modify this change in 1.8:
text/html; charset=iso-8859-1 -> application/xhtml+xml
Thank you, Tyler!
Best,
Tim
RE: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by Ken Krugler <kk...@transpac.com>.
> From: Allison, Timothy B.
> Sent: April 9, 2015 9:02:44am PDT
> To: dev@tika.apache.org
> Subject: RE: [VOTE] Release Apache Tika 1.8 Candidate #1
>
> I just finished the against govdocs1 with 1.7 vs. 1.8-rc1, and all looks good with one major change... on first glance.
>
> Because of my "fix" on TIKA-1519 and the law of unintended consequences, files that start like so:
>
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
> <html xmlns="http://www.w3.org/1999/xhtml">
> <head>
> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
>
> Have different Content-Type(s) between the
>
> In Tika 1.7, they used to have a Content-Type of: text/html; charset=iso-8859-1
>
> In Tika 1.8-rc1, they now have a Content-Type of: application/xhtml+xml
>
> This is a major change.
>
> Do we want this?
>
> Or do we want to revert to the old behavior but add some kind of filter to prevent crazy Content-Type information like the following from overwriting what the detector detected:
> <meta http-equiv="Content-Type" content="application/pdf" />
> or
> <meta http-equiv="Content-Type" content="anythingIFeelLikeInserting" />
I'd have to vote for option #2, as currently there are a number of places in my various crawl projects (that uses Tika for mime type detection) which use text/html to mean HTML pages.
Though we've mostly transitioned to using HtmlParser.getSupportedTypes() to generate this list dynamically based on what Tika reports.
-- Ken
> -----Original Message-----
> From: David Meikle [mailto:loompa@gmail.com]
> Sent: Wednesday, April 08, 2015 8:06 PM
> To: dev@tika.apache.org
> Subject: Re: [VOTE] Release Apache Tika 1.8 Candidate #1
>
> Hey Tyler,
>
>> On 7 Apr 2015, at 19:54, Tyler Palsulich <tp...@apache.org> wrote:
>>
>> [ ] +1 Release this package as Apache Tika 1.8
>> [ ] -1 Do not release this package because...
>
> Whilst my testing with the release is good so far on Mac and Linux with Windows to go, and I am inclined to +1, it would be good if you were able to get your code signing key signed by someone nearby to avoid the warning below?
>
> amadeaus-air:release david$ gpg --verify tika-1.8-src.zip.asc
> gpg: Signature made Tue 7 Apr 19:45:15 2015 EDT using RSA key ID D4F10117
> gpg: Good signature from "Tyler Palsulich <tp...@apache.org>"
> gpg: WARNING: This key is not certified with a trusted signature!
> gpg: There is no indication that the signature belongs to the owner.
> Primary key fingerprint: 1D32 9CC2 D69C 821B FBE4 183E 8810 BB19 D4F1 0117
>
> Not sure if Chris, Lewis et al are near you and do this quickly?
>
> Cheers,
> Dave
--------------------------
Ken Krugler
+1 530-210-6378
http://www.scaleunlimited.com
custom big data solutions & training
Hadoop, Cascading, Cassandra & Solr
--------------------------
Ken Krugler
+1 530-210-6378
http://www.scaleunlimited.com
custom big data solutions & training
Hadoop, Cascading, Cassandra & Solr
Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by Konstantin Gribov <gr...@gmail.com>.
Hi, Tim.
I think, whitelisting on content-type from meta tag can be a solution. We
can whitelist "text/html" + options (like "text/html; charset=...") and
"application/xhtml+xml" + options. So, users, who had valid (text/html or
application/xhtml+xml in <meta http-equiv="Content-Type" ...>) will have
same behavior as it was in 1.7 and wouldn't have some weird content-type in
case of invalid html meta tag.
On the other hand, we can provide some configuration switch to choose
between Content-Type source preference for html/xhtml parsers. But
content-type whitelisting is still a good idea, IMHO.
--
Best regards,
Konstantin Gribov
чт, 9 апр. 2015 г. в 19:03, Allison, Timothy B. <ta...@mitre.org>:
> I just finished the against govdocs1 with 1.7 vs. 1.8-rc1, and all looks
> good with one major change... on first glance.
>
> Because of my "fix" on TIKA-1519 and the law of unintended consequences,
> files that start like so:
>
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "
> http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
> <html xmlns="http://www.w3.org/1999/xhtml">
> <head>
> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
>
> Have different Content-Type(s) between the
>
> In Tika 1.7, they used to have a Content-Type of: text/html;
> charset=iso-8859-1
>
> In Tika 1.8-rc1, they now have a Content-Type of: application/xhtml+xml
>
> This is a major change.
>
> Do we want this?
>
> Or do we want to revert to the old behavior but add some kind of filter
> to prevent crazy Content-Type information like the following from
> overwriting what the detector detected:
> <meta http-equiv="Content-Type" content="application/pdf" />
> or
> <meta http-equiv="Content-Type" content="anythingIFeelLikeInserting" />
>
> -----Original Message-----
> From: David Meikle [mailto:loompa@gmail.com]
> Sent: Wednesday, April 08, 2015 8:06 PM
> To: dev@tika.apache.org
> Subject: Re: [VOTE] Release Apache Tika 1.8 Candidate #1
>
> Hey Tyler,
>
> > On 7 Apr 2015, at 19:54, Tyler Palsulich <tp...@apache.org> wrote:
> >
> > [ ] +1 Release this package as Apache Tika 1.8
> > [ ] -1 Do not release this package because...
>
> Whilst my testing with the release is good so far on Mac and Linux with
> Windows to go, and I am inclined to +1, it would be good if you were able
> to get your code signing key signed by someone nearby to avoid the warning
> below?
>
> amadeaus-air:release david$ gpg --verify tika-1.8-src.zip.asc
> gpg: Signature made Tue 7 Apr 19:45:15 2015 EDT using RSA key ID D4F10117
> gpg: Good signature from "Tyler Palsulich <tp...@apache.org>"
> gpg: WARNING: This key is not certified with a trusted signature!
> gpg: There is no indication that the signature belongs to the
> owner.
> Primary key fingerprint: 1D32 9CC2 D69C 821B FBE4 183E 8810 BB19 D4F1 0117
>
> Not sure if Chris, Lewis et al are near you and do this quickly?
>
> Cheers,
> Dave
>
RE: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by "Allison, Timothy B." <ta...@mitre.org>.
For those who want to take a look at the reports (much more work is needed on processing stack traces for SORT_STACK_TRACE):
https://github.com/tballison/share/blob/master/tika_comparisons/tika_1_7_v_1_8-rc1.zip
RE: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by "Allison, Timothy B." <ta...@mitre.org>.
I just finished the against govdocs1 with 1.7 vs. 1.8-rc1, and all looks good with one major change... on first glance.
Because of my "fix" on TIKA-1519 and the law of unintended consequences, files that start like so:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
Have different Content-Type(s) between the
In Tika 1.7, they used to have a Content-Type of: text/html; charset=iso-8859-1
In Tika 1.8-rc1, they now have a Content-Type of: application/xhtml+xml
This is a major change.
Do we want this?
Or do we want to revert to the old behavior but add some kind of filter to prevent crazy Content-Type information like the following from overwriting what the detector detected:
<meta http-equiv="Content-Type" content="application/pdf" />
or
<meta http-equiv="Content-Type" content="anythingIFeelLikeInserting" />
-----Original Message-----
From: David Meikle [mailto:loompa@gmail.com]
Sent: Wednesday, April 08, 2015 8:06 PM
To: dev@tika.apache.org
Subject: Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Hey Tyler,
> On 7 Apr 2015, at 19:54, Tyler Palsulich <tp...@apache.org> wrote:
>
> [ ] +1 Release this package as Apache Tika 1.8
> [ ] -1 Do not release this package because...
Whilst my testing with the release is good so far on Mac and Linux with Windows to go, and I am inclined to +1, it would be good if you were able to get your code signing key signed by someone nearby to avoid the warning below?
amadeaus-air:release david$ gpg --verify tika-1.8-src.zip.asc
gpg: Signature made Tue 7 Apr 19:45:15 2015 EDT using RSA key ID D4F10117
gpg: Good signature from "Tyler Palsulich <tp...@apache.org>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 1D32 9CC2 D69C 821B FBE4 183E 8810 BB19 D4F1 0117
Not sure if Chris, Lewis et al are near you and do this quickly?
Cheers,
Dave
Re: [VOTE] Release Apache Tika 1.8 Candidate #1
Posted by David Meikle <lo...@gmail.com>.
Hey Tyler,
> On 7 Apr 2015, at 19:54, Tyler Palsulich <tp...@apache.org> wrote:
>
> [ ] +1 Release this package as Apache Tika 1.8
> [ ] -1 Do not release this package because...
Whilst my testing with the release is good so far on Mac and Linux with Windows to go, and I am inclined to +1, it would be good if you were able to get your code signing key signed by someone nearby to avoid the warning below?
amadeaus-air:release david$ gpg --verify tika-1.8-src.zip.asc
gpg: Signature made Tue 7 Apr 19:45:15 2015 EDT using RSA key ID D4F10117
gpg: Good signature from "Tyler Palsulich <tp...@apache.org>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 1D32 9CC2 D69C 821B FBE4 183E 8810 BB19 D4F1 0117
Not sure if Chris, Lewis et al are near you and do this quickly?
Cheers,
Dave