You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tika.apache.org by "Radek (JIRA)" <ji...@apache.org> on 2010/08/24 02:42:16 UTC
[jira] Created: (TIKA-495) Metadata constructor is slow
Metadata constructor is slow
----------------------------
Key: TIKA-495
URL: https://issues.apache.org/jira/browse/TIKA-495
Project: Tika
Issue Type: Bug
Components: metadata
Affects Versions: 0.8
Reporter: Radek
Metadata constructor initialises a bunch of SimpleTimeFormat objects. This is slow - they need to parse their configuration strings, system must be queried for timezone, and so on. In my simple test in which I create new Metadata object for each new document this accounts for approximately 25-30% of all time spent in Tika.
As pointed out by Nick Burch, SimpleTimeFormat is not threadsafe, so they can't be static.
Possible solutions include using org.apache.commons.lang.time.FastDateFormat instead (if its features are sufficient), or using thread local objects.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (TIKA-495) Metadata constructor is slow
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TIKA-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jukka Zitting updated TIKA-495:
-------------------------------
Attachment: TIKA-495.patch
In fact the format instances can also be static, we just need to protect them against concurrent access.
Since the date formats are typically only accessed a few times per each parsed document, using normal synchronization shouldn't cause any lock contention.
See the attached patch for a proposed solution.
> Metadata constructor is slow
> ----------------------------
>
> Key: TIKA-495
> URL: https://issues.apache.org/jira/browse/TIKA-495
> Project: Tika
> Issue Type: Bug
> Components: metadata
> Affects Versions: 0.8
> Reporter: Radek
> Attachments: TIKA-495.patch
>
>
> Metadata constructor initialises a bunch of SimpleTimeFormat objects. This is slow - they need to parse their configuration strings, system must be queried for timezone, and so on. In my simple test in which I create new Metadata object for each new document this accounts for approximately 25-30% of all time spent in Tika.
> As pointed out by Nick Burch, SimpleTimeFormat is not threadsafe, so they can't be static.
> Possible solutions include using org.apache.commons.lang.time.FastDateFormat instead (if its features are sufficient), or using thread local objects.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (TIKA-495) Metadata constructor is slow
Posted by "Radek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TIKA-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901810#action_12901810 ]
Radek commented on TIKA-495:
----------------------------
[edit] ugh I'm blind, those methods are static now, it's all good. thanks for the patch!
> Metadata constructor is slow
> ----------------------------
>
> Key: TIKA-495
> URL: https://issues.apache.org/jira/browse/TIKA-495
> Project: Tika
> Issue Type: Bug
> Components: metadata
> Affects Versions: 0.8
> Reporter: Radek
> Attachments: TIKA-495.patch
>
>
> Metadata constructor initialises a bunch of SimpleTimeFormat objects. This is slow - they need to parse their configuration strings, system must be queried for timezone, and so on. In my simple test in which I create new Metadata object for each new document this accounts for approximately 25-30% of all time spent in Tika.
> As pointed out by Nick Burch, SimpleTimeFormat is not threadsafe, so they can't be static.
> Possible solutions include using org.apache.commons.lang.time.FastDateFormat instead (if its features are sufficient), or using thread local objects.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (TIKA-495) Metadata constructor is slow
Posted by "Radek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TIKA-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901808#action_12901808 ]
Radek commented on TIKA-495:
----------------------------
It's funny how I omitted this simple solution in my report :)
However, shouldn't you synchronize on those static objects (or the array) rather than on |this|? Please correct me if I'm wrong (it's been a long day...) but attached patch will only protect against concurrent access from one instance of Metadata, but not across instances.
:)
> Metadata constructor is slow
> ----------------------------
>
> Key: TIKA-495
> URL: https://issues.apache.org/jira/browse/TIKA-495
> Project: Tika
> Issue Type: Bug
> Components: metadata
> Affects Versions: 0.8
> Reporter: Radek
> Attachments: TIKA-495.patch
>
>
> Metadata constructor initialises a bunch of SimpleTimeFormat objects. This is slow - they need to parse their configuration strings, system must be queried for timezone, and so on. In my simple test in which I create new Metadata object for each new document this accounts for approximately 25-30% of all time spent in Tika.
> As pointed out by Nick Burch, SimpleTimeFormat is not threadsafe, so they can't be static.
> Possible solutions include using org.apache.commons.lang.time.FastDateFormat instead (if its features are sufficient), or using thread local objects.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (TIKA-495) Metadata constructor is slow
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TIKA-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jukka Zitting resolved TIKA-495.
--------------------------------
Assignee: Jukka Zitting
Fix Version/s: 0.8
Resolution: Fixed
Patch committed in revision 988934.
> Metadata constructor is slow
> ----------------------------
>
> Key: TIKA-495
> URL: https://issues.apache.org/jira/browse/TIKA-495
> Project: Tika
> Issue Type: Bug
> Components: metadata
> Affects Versions: 0.8
> Reporter: Radek
> Assignee: Jukka Zitting
> Fix For: 0.8
>
> Attachments: TIKA-495.patch
>
>
> Metadata constructor initialises a bunch of SimpleTimeFormat objects. This is slow - they need to parse their configuration strings, system must be queried for timezone, and so on. In my simple test in which I create new Metadata object for each new document this accounts for approximately 25-30% of all time spent in Tika.
> As pointed out by Nick Burch, SimpleTimeFormat is not threadsafe, so they can't be static.
> Possible solutions include using org.apache.commons.lang.time.FastDateFormat instead (if its features are sufficient), or using thread local objects.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (TIKA-495) Metadata constructor is slow
Posted by "Radek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TIKA-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Radek updated TIKA-495:
-----------------------
Comment: was deleted
(was: It's funny how I omitted this simple solution in my report :)
However, shouldn't you synchronize on those static objects (or the array) rather than on |this|? Please correct me if I'm wrong (it's been a long day...) but attached patch will only protect against concurrent access from one instance of Metadata, but not across instances.
:))
> Metadata constructor is slow
> ----------------------------
>
> Key: TIKA-495
> URL: https://issues.apache.org/jira/browse/TIKA-495
> Project: Tika
> Issue Type: Bug
> Components: metadata
> Affects Versions: 0.8
> Reporter: Radek
> Attachments: TIKA-495.patch
>
>
> Metadata constructor initialises a bunch of SimpleTimeFormat objects. This is slow - they need to parse their configuration strings, system must be queried for timezone, and so on. In my simple test in which I create new Metadata object for each new document this accounts for approximately 25-30% of all time spent in Tika.
> As pointed out by Nick Burch, SimpleTimeFormat is not threadsafe, so they can't be static.
> Possible solutions include using org.apache.commons.lang.time.FastDateFormat instead (if its features are sufficient), or using thread local objects.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.