You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by "Roman Puchkovskiy (JIRA)" <ji...@apache.org> on 2008/06/26 08:08:45 UTC
[jira] Created: (JCR-1663) REFERENCE properties produce duplicate
strings in memory
REFERENCE properties produce duplicate strings in memory
--------------------------------------------------------
Key: JCR-1663
URL: https://issues.apache.org/jira/browse/JCR-1663
Project: Jackrabbit
Issue Type: Improvement
Components: jackrabbit-core, jackrabbit-spi-commons
Affects Versions: 1.4, core 1.4.5
Reporter: Roman Puchkovskiy
When reference property is loaded from PM, Serializer.deserialize(NodeReferences, InputStream) is called, which calls PropertyId.valueOf(String), which in turn calls NameFactoryImpl.create(String) which finally splits a full property name to namespace and local name. Namespace is internalized, but local name is not (comments say that this is done to avoid perm space overfilling).
So, in the end, a new String instance is created for local name. This leads to considerable memory waste when repository has a lot of nodes with REFERENCE properties.
It seems that local name part could be internalized here too because in the most repositories it's not allowed to create properties with arbitrary names, so the danger of perm space exhaust does not seem to be an argument.
As for ways to resolve this, maybe a new NameFactory implementation could be created which would be used for properties only (and, possibly, mainly in the PropertyId.valueOf(String)) which would extend an existing NameFactoryImpl overriding its create(String) method.
What do you think about all this?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1663) REFERENCE properties produce duplicate
strings in memory
Posted by "Thomas Mueller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12611685#action_12611685 ]
Thomas Mueller commented on JCR-1663:
-------------------------------------
Hi,
This patch looks good, but there is a flaw. Not a bug, but it might lead to a bug later on when somebody (for whatever reason) changes the code later on. See http://developers.sun.com/learning/javaoneonline/2007/pdf/TS-2707.pdf page 38ff. I don't want to spoil it in case you like to solve the puzzle yourself... By the way, I can highly recommend the book "Java Puzzlers".
Something completely different: did you know that % (mod) is slower than & (bitwise and)? Also the following wouldn't require Math.abs. You _need_ to use a power of 2 size then, but you could document that as follows:
private final static int SIZE_POWER_OF_2 = 1 << 10;
private final Name[] names = new Name[SIZE_POWER_OF_2];
...
int position = name.hashCode() & (SIZE_POWER_OF_2 - 1);
Regards,
Thomas
> REFERENCE properties produce duplicate strings in memory
> --------------------------------------------------------
>
> Key: JCR-1663
> URL: https://issues.apache.org/jira/browse/JCR-1663
> Project: Jackrabbit
> Issue Type: Improvement
> Components: jackrabbit-core, jackrabbit-spi-commons
> Affects Versions: 1.4, core 1.4.5
> Reporter: Roman Puchkovskiy
> Attachments: JCR-1663.patch
>
>
> When reference property is loaded from PM, Serializer.deserialize(NodeReferences, InputStream) is called, which calls PropertyId.valueOf(String), which in turn calls NameFactoryImpl.create(String) which finally splits a full property name to namespace and local name. Namespace is internalized, but local name is not (comments say that this is done to avoid perm space overfilling).
> So, in the end, a new String instance is created for local name. This leads to considerable memory waste when repository has a lot of nodes with REFERENCE properties.
> It seems that local name part could be internalized here too because in the most repositories it's not allowed to create properties with arbitrary names, so the danger of perm space exhaust does not seem to be an argument.
> As for ways to resolve this, maybe a new NameFactory implementation could be created which would be used for properties only (and, possibly, mainly in the PropertyId.valueOf(String)) which would extend an existing NameFactoryImpl overriding its create(String) method.
> What do you think about all this?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1663) REFERENCE properties produce duplicate
strings in memory
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12608661#action_12608661 ]
Jukka Zitting commented on JCR-1663:
------------------------------------
Re: improvement. Sure, so far I just did the easiest thing. Also, it could well be that a smart JVM could defer the creation of the new object... or perhaps I'm counting too much on the optimizations.
> REFERENCE properties produce duplicate strings in memory
> --------------------------------------------------------
>
> Key: JCR-1663
> URL: https://issues.apache.org/jira/browse/JCR-1663
> Project: Jackrabbit
> Issue Type: Improvement
> Components: jackrabbit-core, jackrabbit-spi-commons
> Affects Versions: 1.4, core 1.4.5
> Reporter: Roman Puchkovskiy
> Attachments: JCR-1663.patch
>
>
> When reference property is loaded from PM, Serializer.deserialize(NodeReferences, InputStream) is called, which calls PropertyId.valueOf(String), which in turn calls NameFactoryImpl.create(String) which finally splits a full property name to namespace and local name. Namespace is internalized, but local name is not (comments say that this is done to avoid perm space overfilling).
> So, in the end, a new String instance is created for local name. This leads to considerable memory waste when repository has a lot of nodes with REFERENCE properties.
> It seems that local name part could be internalized here too because in the most repositories it's not allowed to create properties with arbitrary names, so the danger of perm space exhaust does not seem to be an argument.
> As for ways to resolve this, maybe a new NameFactory implementation could be created which would be used for properties only (and, possibly, mainly in the PropertyId.valueOf(String)) which would extend an existing NameFactoryImpl overriding its create(String) method.
> What do you think about all this?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JCR-1663) REFERENCE properties produce duplicate
strings in memory
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jukka Zitting updated JCR-1663:
-------------------------------
Attachment: JCR-1663.patch
Do you have a chance of trying out the attached patch (JCR-1663.patch)? It's a rather simple implementation of the flyweight pattern with the added twist that it uses a fixed-size hash table to avoid blowing up too much memory in the worst case.
The change seems to work fine (i.e. no regressions), but I don't have a good performance/memory test suite at hand to verify whether this is worth the added complexity.
> REFERENCE properties produce duplicate strings in memory
> --------------------------------------------------------
>
> Key: JCR-1663
> URL: https://issues.apache.org/jira/browse/JCR-1663
> Project: Jackrabbit
> Issue Type: Improvement
> Components: jackrabbit-core, jackrabbit-spi-commons
> Affects Versions: 1.4, core 1.4.5
> Reporter: Roman Puchkovskiy
> Attachments: JCR-1663.patch
>
>
> When reference property is loaded from PM, Serializer.deserialize(NodeReferences, InputStream) is called, which calls PropertyId.valueOf(String), which in turn calls NameFactoryImpl.create(String) which finally splits a full property name to namespace and local name. Namespace is internalized, but local name is not (comments say that this is done to avoid perm space overfilling).
> So, in the end, a new String instance is created for local name. This leads to considerable memory waste when repository has a lot of nodes with REFERENCE properties.
> It seems that local name part could be internalized here too because in the most repositories it's not allowed to create properties with arbitrary names, so the danger of perm space exhaust does not seem to be an argument.
> As for ways to resolve this, maybe a new NameFactory implementation could be created which would be used for properties only (and, possibly, mainly in the PropertyId.valueOf(String)) which would extend an existing NameFactoryImpl overriding its create(String) method.
> What do you think about all this?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1663) REFERENCE properties produce duplicate
strings in memory
Posted by "Roman Puchkovskiy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12609160#action_12609160 ]
Roman Puchkovskiy commented on JCR-1663:
----------------------------------------
The patch works great, now we have only a little number of NameImpl instances in memory, and number of strings is less too. As for NameImpl instances (and corresponding strings) which are created before checking the cache, it seems that they are garbage-collected early.
> REFERENCE properties produce duplicate strings in memory
> --------------------------------------------------------
>
> Key: JCR-1663
> URL: https://issues.apache.org/jira/browse/JCR-1663
> Project: Jackrabbit
> Issue Type: Improvement
> Components: jackrabbit-core, jackrabbit-spi-commons
> Affects Versions: 1.4, core 1.4.5
> Reporter: Roman Puchkovskiy
> Attachments: JCR-1663.patch
>
>
> When reference property is loaded from PM, Serializer.deserialize(NodeReferences, InputStream) is called, which calls PropertyId.valueOf(String), which in turn calls NameFactoryImpl.create(String) which finally splits a full property name to namespace and local name. Namespace is internalized, but local name is not (comments say that this is done to avoid perm space overfilling).
> So, in the end, a new String instance is created for local name. This leads to considerable memory waste when repository has a lot of nodes with REFERENCE properties.
> It seems that local name part could be internalized here too because in the most repositories it's not allowed to create properties with arbitrary names, so the danger of perm space exhaust does not seem to be an argument.
> As for ways to resolve this, maybe a new NameFactory implementation could be created which would be used for properties only (and, possibly, mainly in the PropertyId.valueOf(String)) which would extend an existing NameFactoryImpl overriding its create(String) method.
> What do you think about all this?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1663) REFERENCE properties produce duplicate
strings in memory
Posted by "Antonio Carballo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12612577#action_12612577 ]
Antonio Carballo commented on JCR-1663:
---------------------------------------
Hmmm...Looks like everyone is on vacation this week. Oh well. I'd appreciate anybody sending this file to my e-mail (see my profile). We've some testing tools that we'd like to run against this patch. Like I said earlier. We applied the patch to the latest trunk and now the build is broken because of this particular source file ("NameFactoryImpl.java"). Thanks.
> REFERENCE properties produce duplicate strings in memory
> --------------------------------------------------------
>
> Key: JCR-1663
> URL: https://issues.apache.org/jira/browse/JCR-1663
> Project: Jackrabbit
> Issue Type: Improvement
> Components: jackrabbit-core, jackrabbit-spi-commons
> Affects Versions: 1.4, core 1.4.5
> Reporter: Roman Puchkovskiy
> Attachments: JCR-1663.patch
>
>
> When reference property is loaded from PM, Serializer.deserialize(NodeReferences, InputStream) is called, which calls PropertyId.valueOf(String), which in turn calls NameFactoryImpl.create(String) which finally splits a full property name to namespace and local name. Namespace is internalized, but local name is not (comments say that this is done to avoid perm space overfilling).
> So, in the end, a new String instance is created for local name. This leads to considerable memory waste when repository has a lot of nodes with REFERENCE properties.
> It seems that local name part could be internalized here too because in the most repositories it's not allowed to create properties with arbitrary names, so the danger of perm space exhaust does not seem to be an argument.
> As for ways to resolve this, maybe a new NameFactory implementation could be created which would be used for properties only (and, possibly, mainly in the PropertyId.valueOf(String)) which would extend an existing NameFactoryImpl overriding its create(String) method.
> What do you think about all this?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1663) REFERENCE properties produce duplicate
strings in memory
Posted by "Antonio Carballo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12611589#action_12611589 ]
Antonio Carballo commented on JCR-1663:
---------------------------------------
Jukka,
We have a series of memory tests with our application and would like to test the latest changes to this module. Will it be possible for you to upload the source file for "NameFactoryImpl.java"? We are running the latest build but we are now getting a compile errors (localName & namespaceURI "var. not found") after the latest changes were incorporated into this module. Thanks.
/Antonio C.
> REFERENCE properties produce duplicate strings in memory
> --------------------------------------------------------
>
> Key: JCR-1663
> URL: https://issues.apache.org/jira/browse/JCR-1663
> Project: Jackrabbit
> Issue Type: Improvement
> Components: jackrabbit-core, jackrabbit-spi-commons
> Affects Versions: 1.4, core 1.4.5
> Reporter: Roman Puchkovskiy
> Attachments: JCR-1663.patch
>
>
> When reference property is loaded from PM, Serializer.deserialize(NodeReferences, InputStream) is called, which calls PropertyId.valueOf(String), which in turn calls NameFactoryImpl.create(String) which finally splits a full property name to namespace and local name. Namespace is internalized, but local name is not (comments say that this is done to avoid perm space overfilling).
> So, in the end, a new String instance is created for local name. This leads to considerable memory waste when repository has a lot of nodes with REFERENCE properties.
> It seems that local name part could be internalized here too because in the most repositories it's not allowed to create properties with arbitrary names, so the danger of perm space exhaust does not seem to be an argument.
> As for ways to resolve this, maybe a new NameFactory implementation could be created which would be used for properties only (and, possibly, mainly in the PropertyId.valueOf(String)) which would extend an existing NameFactoryImpl overriding its create(String) method.
> What do you think about all this?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (JCR-1663) REFERENCE properties produce duplicate
strings in memory
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jukka Zitting resolved JCR-1663.
--------------------------------
Resolution: Fixed
Back form vacation... Thanks for the reviews, tests and suggestions!
I committed an improved version of the proposed solution in revision 678719.
Resolving as fixed.
> REFERENCE properties produce duplicate strings in memory
> --------------------------------------------------------
>
> Key: JCR-1663
> URL: https://issues.apache.org/jira/browse/JCR-1663
> Project: Jackrabbit
> Issue Type: Improvement
> Components: jackrabbit-spi-commons
> Reporter: Roman Puchkovskiy
> Assignee: Jukka Zitting
> Fix For: 1.5
>
> Attachments: JCR-1663.patch
>
>
> When reference property is loaded from PM, Serializer.deserialize(NodeReferences, InputStream) is called, which calls PropertyId.valueOf(String), which in turn calls NameFactoryImpl.create(String) which finally splits a full property name to namespace and local name. Namespace is internalized, but local name is not (comments say that this is done to avoid perm space overfilling).
> So, in the end, a new String instance is created for local name. This leads to considerable memory waste when repository has a lot of nodes with REFERENCE properties.
> It seems that local name part could be internalized here too because in the most repositories it's not allowed to create properties with arbitrary names, so the danger of perm space exhaust does not seem to be an argument.
> As for ways to resolve this, maybe a new NameFactory implementation could be created which would be used for properties only (and, possibly, mainly in the PropertyId.valueOf(String)) which would extend an existing NameFactoryImpl overriding its create(String) method.
> What do you think about all this?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1663) REFERENCE properties produce duplicate
strings in memory
Posted by "Stefan Guggisberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12608329#action_12608329 ]
Stefan Guggisberg commented on JCR-1663:
----------------------------------------
at one point we actually did intern the local Name part. we ran into the said perm space issues at some installations, that's why we only intern the namespace uri. that helped to considerable improve performance (equals() method) and reduce memory footprint.
therefore, -1 for interning local name strings
i agree with jukka that NameFactory maintaining a chache of commonly used names would be a good alternative.
however, unless there's a real issue, i'd prefer to stick with interning namespace uri's.
> REFERENCE properties produce duplicate strings in memory
> --------------------------------------------------------
>
> Key: JCR-1663
> URL: https://issues.apache.org/jira/browse/JCR-1663
> Project: Jackrabbit
> Issue Type: Improvement
> Components: jackrabbit-core, jackrabbit-spi-commons
> Affects Versions: 1.4, core 1.4.5
> Reporter: Roman Puchkovskiy
>
> When reference property is loaded from PM, Serializer.deserialize(NodeReferences, InputStream) is called, which calls PropertyId.valueOf(String), which in turn calls NameFactoryImpl.create(String) which finally splits a full property name to namespace and local name. Namespace is internalized, but local name is not (comments say that this is done to avoid perm space overfilling).
> So, in the end, a new String instance is created for local name. This leads to considerable memory waste when repository has a lot of nodes with REFERENCE properties.
> It seems that local name part could be internalized here too because in the most repositories it's not allowed to create properties with arbitrary names, so the danger of perm space exhaust does not seem to be an argument.
> As for ways to resolve this, maybe a new NameFactory implementation could be created which would be used for properties only (and, possibly, mainly in the PropertyId.valueOf(String)) which would extend an existing NameFactoryImpl overriding its create(String) method.
> What do you think about all this?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JCR-1663) REFERENCE properties produce duplicate
strings in memory
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jukka Zitting updated JCR-1663:
-------------------------------
Component/s: (was: jackrabbit-core)
Affects Version/s: (was: core 1.4.5)
(was: 1.4)
Fix Version/s: 1.5
Assignee: Jukka Zitting
> REFERENCE properties produce duplicate strings in memory
> --------------------------------------------------------
>
> Key: JCR-1663
> URL: https://issues.apache.org/jira/browse/JCR-1663
> Project: Jackrabbit
> Issue Type: Improvement
> Components: jackrabbit-spi-commons
> Reporter: Roman Puchkovskiy
> Assignee: Jukka Zitting
> Fix For: 1.5
>
> Attachments: JCR-1663.patch
>
>
> When reference property is loaded from PM, Serializer.deserialize(NodeReferences, InputStream) is called, which calls PropertyId.valueOf(String), which in turn calls NameFactoryImpl.create(String) which finally splits a full property name to namespace and local name. Namespace is internalized, but local name is not (comments say that this is done to avoid perm space overfilling).
> So, in the end, a new String instance is created for local name. This leads to considerable memory waste when repository has a lot of nodes with REFERENCE properties.
> It seems that local name part could be internalized here too because in the most repositories it's not allowed to create properties with arbitrary names, so the danger of perm space exhaust does not seem to be an argument.
> As for ways to resolve this, maybe a new NameFactory implementation could be created which would be used for properties only (and, possibly, mainly in the PropertyId.valueOf(String)) which would extend an existing NameFactoryImpl overriding its create(String) method.
> What do you think about all this?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1663) REFERENCE properties produce duplicate
strings in memory
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12608289#action_12608289 ]
Jukka Zitting commented on JCR-1663:
------------------------------------
Instead of interning the local part of the name but still creating new Name instances I'd rather see the NameFactory maintain a set of flywheel Name instances for commonly used names. Then we perhaps wouldn't even need to intern the namespace strings!
> REFERENCE properties produce duplicate strings in memory
> --------------------------------------------------------
>
> Key: JCR-1663
> URL: https://issues.apache.org/jira/browse/JCR-1663
> Project: Jackrabbit
> Issue Type: Improvement
> Components: jackrabbit-core, jackrabbit-spi-commons
> Affects Versions: 1.4, core 1.4.5
> Reporter: Roman Puchkovskiy
>
> When reference property is loaded from PM, Serializer.deserialize(NodeReferences, InputStream) is called, which calls PropertyId.valueOf(String), which in turn calls NameFactoryImpl.create(String) which finally splits a full property name to namespace and local name. Namespace is internalized, but local name is not (comments say that this is done to avoid perm space overfilling).
> So, in the end, a new String instance is created for local name. This leads to considerable memory waste when repository has a lot of nodes with REFERENCE properties.
> It seems that local name part could be internalized here too because in the most repositories it's not allowed to create properties with arbitrary names, so the danger of perm space exhaust does not seem to be an argument.
> As for ways to resolve this, maybe a new NameFactory implementation could be created which would be used for properties only (and, possibly, mainly in the PropertyId.valueOf(String)) which would extend an existing NameFactoryImpl overriding its create(String) method.
> What do you think about all this?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1663) REFERENCE properties produce duplicate
strings in memory
Posted by "Roman Puchkovskiy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12608658#action_12608658 ]
Roman Puchkovskiy commented on JCR-1663:
----------------------------------------
Thanks for the patch, we will try it.
As a little improvement, can we avoid creation of NameImpl and splitting a full name string at all in cases where the cache is hit? I.e. can we calculate array index using nameString.hashCode() and then compare nameString with name's toString() result instead of using name.hashCode() and comparing current name with cached name?
> REFERENCE properties produce duplicate strings in memory
> --------------------------------------------------------
>
> Key: JCR-1663
> URL: https://issues.apache.org/jira/browse/JCR-1663
> Project: Jackrabbit
> Issue Type: Improvement
> Components: jackrabbit-core, jackrabbit-spi-commons
> Affects Versions: 1.4, core 1.4.5
> Reporter: Roman Puchkovskiy
> Attachments: JCR-1663.patch
>
>
> When reference property is loaded from PM, Serializer.deserialize(NodeReferences, InputStream) is called, which calls PropertyId.valueOf(String), which in turn calls NameFactoryImpl.create(String) which finally splits a full property name to namespace and local name. Namespace is internalized, but local name is not (comments say that this is done to avoid perm space overfilling).
> So, in the end, a new String instance is created for local name. This leads to considerable memory waste when repository has a lot of nodes with REFERENCE properties.
> It seems that local name part could be internalized here too because in the most repositories it's not allowed to create properties with arbitrary names, so the danger of perm space exhaust does not seem to be an argument.
> As for ways to resolve this, maybe a new NameFactory implementation could be created which would be used for properties only (and, possibly, mainly in the PropertyId.valueOf(String)) which would extend an existing NameFactoryImpl overriding its create(String) method.
> What do you think about all this?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1663) REFERENCE properties produce duplicate
strings in memory
Posted by "Antonio Carballo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614369#action_12614369 ]
Antonio Carballo commented on JCR-1663:
---------------------------------------
Thank you Roman for the java file and comments.
==
We ran more tests (2gb, 30,000+ documents, 10 different doc types) over the weekend and concluded that the patch does alleviate this particular memory issue. However, the JCR engine continues to hang on to memory. Eventually it hits the 'wall' and slows down the response time. Thanks for the patch.
> REFERENCE properties produce duplicate strings in memory
> --------------------------------------------------------
>
> Key: JCR-1663
> URL: https://issues.apache.org/jira/browse/JCR-1663
> Project: Jackrabbit
> Issue Type: Improvement
> Components: jackrabbit-core, jackrabbit-spi-commons
> Affects Versions: 1.4, core 1.4.5
> Reporter: Roman Puchkovskiy
> Attachments: JCR-1663.patch
>
>
> When reference property is loaded from PM, Serializer.deserialize(NodeReferences, InputStream) is called, which calls PropertyId.valueOf(String), which in turn calls NameFactoryImpl.create(String) which finally splits a full property name to namespace and local name. Namespace is internalized, but local name is not (comments say that this is done to avoid perm space overfilling).
> So, in the end, a new String instance is created for local name. This leads to considerable memory waste when repository has a lot of nodes with REFERENCE properties.
> It seems that local name part could be internalized here too because in the most repositories it's not allowed to create properties with arbitrary names, so the danger of perm space exhaust does not seem to be an argument.
> As for ways to resolve this, maybe a new NameFactory implementation could be created which would be used for properties only (and, possibly, mainly in the PropertyId.valueOf(String)) which would extend an existing NameFactoryImpl overriding its create(String) method.
> What do you think about all this?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JCR-1663) REFERENCE properties produce duplicate
strings in memory
Posted by "Roman Puchkovskiy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12608457#action_12608457 ]
Roman Puchkovskiy commented on JCR-1663:
----------------------------------------
We have a big repository with several tenths of thousands of nodes (which have versions). Each node has 4 reference properties.
I'm now looking at our application heap snapshot. Its overall size is 400+ mb and almost 200 mb are strings (nearly 1.4 million of them). Most of those strings are in NameImpl instances (more than 1.3 million of instances). And nearly 1.3 million of strings are actually duplicates of 8 different strings which are local names and full names of our 4 reference properties.
So I think it's a real issue that needs to be resolved :) Our use case is an example when Jackrabbit uses memory inefficiently.
Jukka's suggestion looks much better than mine: consider 1.3 million of NameImpl instances in the example.
> REFERENCE properties produce duplicate strings in memory
> --------------------------------------------------------
>
> Key: JCR-1663
> URL: https://issues.apache.org/jira/browse/JCR-1663
> Project: Jackrabbit
> Issue Type: Improvement
> Components: jackrabbit-core, jackrabbit-spi-commons
> Affects Versions: 1.4, core 1.4.5
> Reporter: Roman Puchkovskiy
>
> When reference property is loaded from PM, Serializer.deserialize(NodeReferences, InputStream) is called, which calls PropertyId.valueOf(String), which in turn calls NameFactoryImpl.create(String) which finally splits a full property name to namespace and local name. Namespace is internalized, but local name is not (comments say that this is done to avoid perm space overfilling).
> So, in the end, a new String instance is created for local name. This leads to considerable memory waste when repository has a lot of nodes with REFERENCE properties.
> It seems that local name part could be internalized here too because in the most repositories it's not allowed to create properties with arbitrary names, so the danger of perm space exhaust does not seem to be an argument.
> As for ways to resolve this, maybe a new NameFactory implementation could be created which would be used for properties only (and, possibly, mainly in the PropertyId.valueOf(String)) which would extend an existing NameFactoryImpl overriding its create(String) method.
> What do you think about all this?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.