You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tika.apache.org by "Erik Hetzner (JIRA)" <ji...@apache.org> on 2011/02/20 00:29:38 UTC
[jira] Created: (TIKA-599) Thread issue with autodetect parser
Thread issue with autodetect parser
-----------------------------------
Key: TIKA-599
URL: https://issues.apache.org/jira/browse/TIKA-599
Project: Tika
Issue Type: Bug
Components: parser
Affects Versions: 0.9
Environment: SunOS xxx 5.10 Generic_142901-13 i86pc i386 i86pc Solaris
java version "1.6.0_14"
Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
Java HotSpot(TM) Server VM (build 14.0-b16, mixed mode)
Reporter: Erik Hetzner
When using tika to index web crawls, I seem to have run across a thread-safety issue with the autodetect parser. My indexer has gone into an apparent loop with all 5 threads in the same bit of code:
"Thread-5" prio=3 tid=0x0898a400 nid=0x19 runnable [0x75238000]
java.lang.Thread.State: RUNNABLE
at java.util.HashMap.get(HashMap.java:303)
at org.ccil.cowan.tagsoup.Schema.getElementType(Schema.java:122)
at org.ccil.cowan.tagsoup.Parser.gi(Parser.java:959)
at org.ccil.cowan.tagsoup.HTMLScanner.scan(HTMLScanner.java:505)
at org.ccil.cowan.tagsoup.Parser.parse(Parser.java:449)
at org.apache.tika.parser.html.HtmlParser.parse(HtmlParser.java:198)
at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:197)
at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:197)
at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:135)
....
This is the same line in HashMap as we see in TIKA-374.
Let me know if you need any more information. This is with the latest tika-0.9.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (TIKA-599) Thread issue with autodetect parser
Posted by "Erik Hetzner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TIKA-599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13004965#comment-13004965 ]
Erik Hetzner commented on TIKA-599:
-----------------------------------
Thanks for fixing this!
> Thread issue with autodetect parser
> -----------------------------------
>
> Key: TIKA-599
> URL: https://issues.apache.org/jira/browse/TIKA-599
> Project: Tika
> Issue Type: Bug
> Components: parser
> Affects Versions: 0.9
> Environment: SunOS xxx 5.10 Generic_142901-13 i86pc i386 i86pc Solaris
> java version "1.6.0_14"
> Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
> Java HotSpot(TM) Server VM (build 14.0-b16, mixed mode)
> Reporter: Erik Hetzner
> Assignee: Jukka Zitting
> Fix For: 1.0
>
>
> When using tika to index web crawls, I seem to have run across a thread-safety issue with the autodetect parser. My indexer has gone into an apparent loop with all 5 threads in the same bit of code:
> "Thread-5" prio=3 tid=0x0898a400 nid=0x19 runnable [0x75238000]
> java.lang.Thread.State: RUNNABLE
> at java.util.HashMap.get(HashMap.java:303)
> at org.ccil.cowan.tagsoup.Schema.getElementType(Schema.java:122)
> at org.ccil.cowan.tagsoup.Parser.gi(Parser.java:959)
> at org.ccil.cowan.tagsoup.HTMLScanner.scan(HTMLScanner.java:505)
> at org.ccil.cowan.tagsoup.Parser.parse(Parser.java:449)
> at org.apache.tika.parser.html.HtmlParser.parse(HtmlParser.java:198)
> at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:197)
> at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:197)
> at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:135)
> ....
> This is the same line in HashMap as we see in TIKA-374.
> Let me know if you need any more information. This is with the latest tika-0.9.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (TIKA-599) Thread issue with autodetect parser
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TIKA-599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jukka Zitting resolved TIKA-599.
--------------------------------
Resolution: Fixed
Fix Version/s: 1.0
Assignee: Jukka Zitting
It turned out that the shared HTMLSchema instance introduced in TIKA-528 was actually
not thread-safe as we assumed. To make it truly thread-safe, in revision 1079915 I enabled the ignoreBogons feature that prevents tagsoup from adding encountered unknown HTML tags to the schema.
> Thread issue with autodetect parser
> -----------------------------------
>
> Key: TIKA-599
> URL: https://issues.apache.org/jira/browse/TIKA-599
> Project: Tika
> Issue Type: Bug
> Components: parser
> Affects Versions: 0.9
> Environment: SunOS xxx 5.10 Generic_142901-13 i86pc i386 i86pc Solaris
> java version "1.6.0_14"
> Java(TM) SE Runtime Environment (build 1.6.0_14-b08)
> Java HotSpot(TM) Server VM (build 14.0-b16, mixed mode)
> Reporter: Erik Hetzner
> Assignee: Jukka Zitting
> Fix For: 1.0
>
>
> When using tika to index web crawls, I seem to have run across a thread-safety issue with the autodetect parser. My indexer has gone into an apparent loop with all 5 threads in the same bit of code:
> "Thread-5" prio=3 tid=0x0898a400 nid=0x19 runnable [0x75238000]
> java.lang.Thread.State: RUNNABLE
> at java.util.HashMap.get(HashMap.java:303)
> at org.ccil.cowan.tagsoup.Schema.getElementType(Schema.java:122)
> at org.ccil.cowan.tagsoup.Parser.gi(Parser.java:959)
> at org.ccil.cowan.tagsoup.HTMLScanner.scan(HTMLScanner.java:505)
> at org.ccil.cowan.tagsoup.Parser.parse(Parser.java:449)
> at org.apache.tika.parser.html.HtmlParser.parse(HtmlParser.java:198)
> at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:197)
> at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:197)
> at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:135)
> ....
> This is the same line in HashMap as we see in TIKA-374.
> Let me know if you need any more information. This is with the latest tika-0.9.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira