You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Erik Holstad (JIRA)" <ji...@apache.org> on 2009/07/15 01:46:15 UTC
[jira] Created: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Making DataNode not instantiate a HashMap when the node is ephmeral
-------------------------------------------------------------------
Key: ZOOKEEPER-472
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
Project: Zookeeper
Issue Type: Improvement
Reporter: Erik Holstad
Priority: Minor
Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754001#action_12754001 ]
Erik Holstad commented on ZOOKEEPER-472:
----------------------------------------
@Mahadev
Yes the Jira was originally about not instantiating the HashSet before any children where added to the node, so in that case there is only a small reason for not having the persistent and the ephemeral nodes the same. For the sorted node I'm just using a sortedSet instead of the hashset, so all the set methods are still valid to use.
Not really sure how many types of nodes there could be, but we have a use case for HBase where this new sorted node would be good.
And in the case of more nodes coming into play it might make sense to have an interface of a base class.
Sorry about not being clear.
Erik
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-472:
-----------------------------------
Component/s: server
Affects Version/s: 3.1.1
3.2.0
Fix Version/s: 3.3.0
Assignee: Erik Holstad
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-472:
-----------------------------------
Status: Patch Available (was: Open)
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.2.0, 3.1.1
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776549#action_12776549 ]
Erik Holstad commented on ZOOKEEPER-472:
----------------------------------------
Not really sure what to do about the no test -1. It is kinda hard to include new test for something like this and make it portable in a good way.
Since I'm not adding any new functionality, the old tests should be enough, or what is the general opinion on this matter?
Erik
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766595#action_12766595 ]
Erik Holstad commented on ZOOKEEPER-472:
----------------------------------------
@ Patrik, sorry for not doing this yesterday, had some other stuff that I needed to take care of. Yeah, that patch was created against an older branch, but I updated to trunk
in Eclipse and tested it before submitting it, but there seems to have been some issues with that.
This patch applied clean on trunk as of this morning using regular patch -p0
Erik
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766655#action_12766655 ]
Erik Holstad commented on ZOOKEEPER-472:
----------------------------------------
@ Mahadev, no reason for it, just a part of the mess with me using another indention scheme for the original patch.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-472:
-----------------------------------
Status: Patch Available (was: Open)
Is this ready? I see some updates, throwing into the patch queue, reviewer please be sure all the comments are addressed.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.2.0, 3.1.1
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731605#action_12731605 ]
Patrick Hunt commented on ZOOKEEPER-472:
----------------------------------------
good point ben, +1
good catch Erik, this is shaping up to be a decent savings. :-)
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12765770#action_12765770 ]
Patrick Hunt commented on ZOOKEEPER-472:
----------------------------------------
Hi Erik, I tried patching the current trunk, but the patch fails (below). Perhaps it's against an older version of zk source? Would you mind
creating a patch against the current source HEAD in svn? (I took a look but it seemed more than just a simple mismatch, easiest if you could address)
Also it's best to have a single patch file as attachment, all you need to do is upload a file with the same name and jira will take care of it:
from howtocontribute:
"In some cases a patch may need to be updated based on review comments. In this case the updated patch should be re-attached to the Jira with the same name. Jira will archive the older version of the patch and make the new patch the active patch. This will enable a history of patches on the Jira. As stated above patch naming is generally ZOOKEEPER-#.patch where ZOOKEEPER-# is the id of the Jira."
you could use "manage attachments" to address (ie remove the v2.patch)
-------------------
patch -p0 < /home/phunt/Desktop/zookeeper-472-v2.patch
patching file src/java/main/org/apache/zookeeper/server/DataNode.java
patching file src/java/main/org/apache/zookeeper/server/DataTree.java
Hunk #35 FAILED at 1029.
1 out of 35 hunks FAILED -- saving rejects to file src/java/main/org/apache/zookeeper/server/DataTree.java.rej
patching file src/java/main/org/apache/zookeeper/server/PrepRequestProcessor.java
Hunk #6 FAILED at 208.
1 out of 9 hunks FAILED -- saving rejects to file src/java/main/org/apache/zookeeper/server/PrepRequestProcessor.java.rej
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472-v2.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766626#action_12766626 ]
Erik Holstad commented on ZOOKEEPER-472:
----------------------------------------
@ Mahadev, yeah that was my bad. Not sure why my environment didn't show that as an error.
Erik
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766611#action_12766611 ]
Mahadev konar commented on ZOOKEEPER-472:
-----------------------------------------
erik,
I just tried compiling the code with ant jar and it fails to compile with the folowing error -
{code}
compile:
[javac] Compiling 104 source files to /home/mahadev/workspace/zookeeper-commit-trunk/build/classes
[javac] /home/mahadev/workspace/zookeeper-commit-trunk/src/java/main/org/apache/zookeeper/server/DataTree.java:1055: children has private access in org.apache.zookeeper.server.DataNode
[javac] node.parent.children.add(path.substring(lastSlash + 1));
[javac] ^
[javac] 1 error
{code}
I havent ahd a chance to review the code yet, but I couldnt help but notice changes to license headers on src/java/main/org/apache/zookeeper/server/DataNode.java. Is there something you changed in the licence?
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-472:
-----------------------------------
Status: Patch Available (was: Open)
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.2.0, 3.1.1
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731595#action_12731595 ]
Benjamin Reed commented on ZOOKEEPER-472:
-----------------------------------------
i think we should expand this to not instantiate a hashmap for all zondes if there aren't any children. it creates a fixed size overhead for all leaf nodes and since there will always be more leaves than inner nodes, it is a none trivial space saving. i think it could also speed serialization/deserialization since it is faster to process a null then an empty hashmap. plus i think it keeps the code simpler to not have a new class.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776572#action_12776572 ]
Patrick Hunt commented on ZOOKEEPER-472:
----------------------------------------
In general we should have tests but I'm fine with the no-test in this case, this is an optimization not a bug fix
and I can't think of any test we could add that would benefit, we already have good test coverage on this area.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-472:
-----------------------------------
Status: Open (was: Patch Available)
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.2.0, 3.1.1
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Holstad updated ZOOKEEPER-472:
-----------------------------------
Attachment: zookeeper-472.patch
Fixed the findbug warning, hopefully, by moving the synchronization away from the node object into the different caller methods.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12765697#action_12765697 ]
Patrick Hunt commented on ZOOKEEPER-472:
----------------------------------------
No yelling here, maybe a stern talking to though. ;-)
Update the patch and we'll take a look. If you release the "in progress" that will allow us to submit for you if we think it's ready.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12777286#action_12777286 ]
Erik Holstad commented on ZOOKEEPER-472:
----------------------------------------
@Mahadev
I totally agree that it is hard to follow a patch when there is a lot of "crap" changes in there. This was not my intention but had to do with the settings
that I used, from the beginning and some un clearness on the website about intentations that has since been fixed.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Holstad updated ZOOKEEPER-472:
-----------------------------------
Attachment: zookeeper-472.patch
I decided to take a lesser approach and just do what this Jira originally said, so basically creating two new methods, addChild() and removeChild() and making the variable children in DataNode private. The children set in DataNode is not instantiated at node creation but instead the first time addChild() is being called.
No synchronization is done one the calls addChild and removeChild since it looks like most other calls are synchronized on the node itself.
Not really sure why getChildren is synchronized though, since as far as I can see it all the uses of it are also synchronized on the node.
According to the docs it says that 2 spaces should be used instead of 4. Is that a miss in the docs or is that what is really being used?
I'm using 2 for this patch.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741689#action_12741689 ]
Erik Holstad commented on ZOOKEEPER-472:
----------------------------------------
After the HBase meeting this weekend we might have a use case for a new node type which keeps the children sorted, so including that we would have 3 different types of nodes, if this new node would be interesting for ZooKeeper in general. Tying that to this issue I came up with the following:
1. Make a Node interface that only have the following methods
copyStat(...)
deserialize(...)
serialize(...)
2. For the ephemeral nodes this would be it, they wouldn't have to have any notion about children.
3, For the regular nodes and the sorted ones would also have:
setChildren(...)
addChild(...)
getChildren(...)
where getChildren(...) could have different return types depending on the node.
4. For the nodes that can have children we also keep the structure uninstantiated until children are actually set or added to the parent. This part I have already done, jsut need to do some more testing on it.
Erik
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753939#action_12753939 ]
Mahadev konar commented on ZOOKEEPER-472:
-----------------------------------------
I htought the jira was just about not instantiating hashmap.. no? Why does PersistentNode and Ephemeral node need to be different? They can be the same but with the property that until and unless a addchild is done, the hashset maps to null... wouldnt that be sufficient?
Even with the above you wont be able to do sorted nodes since the chidlren are still sets... no? The children member should map to some other datastructure that maintains order.. no?
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Holstad updated ZOOKEEPER-472:
-----------------------------------
Attachment: zookeeper-472.patch
Fixed issues, so that patch applies to trunk
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753929#action_12753929 ]
Erik Holstad commented on ZOOKEEPER-472:
----------------------------------------
Did some more coding today and created 3 classes and 1 inteface:
Interface Node
Class EphemeralNode
Class PersistentNode
Class PersistentSortedNode
The problem is that a lot of the class variables from the old DataNode class are being used like node.data = {}, instead of getting and setting it, which leads to some weirdness
when the node is no longer a class, but an interface. So the way I see it we can take two approaches.
1. Having the nodes subclass and override methods from a base class or
2. Start using getters and setters for all variables in the nodes.
Just want to check before I do anything more since this might turn out to be a quite big change of how the nodes are being used internally.
Erik
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776580#action_12776580 ]
Erik Holstad commented on ZOOKEEPER-472:
----------------------------------------
Sounds good to me.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-472:
-----------------------------------
Status: Open (was: Patch Available)
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.2.0, 3.1.1
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Holstad updated ZOOKEEPER-472:
-----------------------------------
Attachment: zookeeper-472-v2.patch
Second version with some minor indention fixes.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472-v2.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12777245#action_12777245 ]
Mahadev konar commented on ZOOKEEPER-472:
-----------------------------------------
+1 .. the patch looks good. one minor nit for future patches, it becomes hard to review the patches if it has a lot of indentation changes. This patch had a lot of indentation than code changes and it became quite hard to review. Its always good to file indentation changes in a seperate jira so that reviewer can easily find out what actual code changes happened.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Holstad updated ZOOKEEPER-472:
-----------------------------------
Attachment: zookeeper-472.patch
Fixed compile error.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12765676#action_12765676 ]
Patrick Hunt commented on ZOOKEEPER-472:
----------------------------------------
Hi, what's the status on this JIRA? I'd like to get it reviewed/committed if it's ready. (you need to "submit patch", I would do it, but I can't as you've marked it as "in progress")
we use 4 spaces, for some reason hadoop mr/hdfs uses 2 but we use 4 (you should see this used throughout our code)
Thanks!
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776510#action_12776510 ]
Hadoop QA commented on ZOOKEEPER-472:
-------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12424615/zookeeper-472.patch
against trunk revision 833938.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 1 new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/61/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/61/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/61/console
This message is automatically generated.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731225#action_12731225 ]
Patrick Hunt commented on ZOOKEEPER-472:
----------------------------------------
Sounds good - as long as we can make the change in a b/w compatible way w/o negatively impacting performance.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Holstad updated ZOOKEEPER-472:
-----------------------------------
Attachment: (was: zookeeper-472-v2.patch)
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12765709#action_12765709 ]
Erik Holstad commented on ZOOKEEPER-472:
----------------------------------------
@ Patrik , well good thing you cannot take me in the ear and sit me down :) Might want to change the instructions on http://wiki.apache.org/hadoop/ZooKeeper/HowToContribute though, to not make people confused? Remember taking a look there when I started and saw the code.
Erik
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472-v2.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Work stopped: (ZOOKEEPER-472) Making DataNode not
instantiate a HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on ZOOKEEPER-472 stopped by Erik Holstad.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472-v2.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Holstad updated ZOOKEEPER-472:
-----------------------------------
Attachment: zookeeper-472.patch
Fixed headers
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-472:
-----------------------------------
Attachment: zookeeper-472.patch
Updated patch to compile against latest trunk.
Also cleaned up some finals.
Also reduced the default child hashset size to 8 rather than 16 (let's be conservative
as to the avg number of subnodes).
Small optimization to getchildren in datatree - allocate exactly the right size array list
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766637#action_12766637 ]
Mahadev konar commented on ZOOKEEPER-472:
-----------------------------------------
erik, there are apache header changes in 2 files, any reason for that?
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12765691#action_12765691 ]
Erik Holstad commented on ZOOKEEPER-472:
----------------------------------------
@ Patrik Hunt, been waiting from someone to have a look at the patch for a little bit. The reason that I didn't mark it as patch available, was because
I got yelled at in the past, not here, for marking it like that when it hadn't been reviewed :) Will change the formatting to 4 spaces and upload again.
Erik
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Work started: (ZOOKEEPER-472) Making DataNode not
instantiate a HashMap when the node is ephmeral
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on ZOOKEEPER-472 started by Erik Holstad.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated ZOOKEEPER-472:
------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. thanks erik.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12777292#action_12777292 ]
Mahadev konar commented on ZOOKEEPER-472:
-----------------------------------------
no worries erik!
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12765765#action_12765765 ]
Patrick Hunt commented on ZOOKEEPER-472:
----------------------------------------
thanks for the pointer - I fixed the indentation item and also added note on notabs. also fixed a problem that should have said tests should end in Test, not start.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472-v2.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776582#action_12776582 ]
Hadoop QA commented on ZOOKEEPER-472:
-------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12424625/zookeeper-472.patch
against trunk revision 833938.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/62/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/62/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/62/console
This message is automatically generated.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12777879#action_12777879 ]
Hudson commented on ZOOKEEPER-472:
----------------------------------
Integrated in ZooKeeper-trunk #536 (See [http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/536/])
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-472) Making DataNode not instantiate a
HashMap when the node is ephmeral
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776220#action_12776220 ]
Hadoop QA commented on ZOOKEEPER-472:
-------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12422389/zookeeper-472.patch
against trunk revision 833938.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
-1 patch. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/60/console
This message is automatically generated.
> Making DataNode not instantiate a HashMap when the node is ephmeral
> -------------------------------------------------------------------
>
> Key: ZOOKEEPER-472
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-472
> Project: Zookeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.1.1, 3.2.0
> Reporter: Erik Holstad
> Assignee: Erik Holstad
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch, zookeeper-472.patch
>
>
> Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.