You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2008/11/07 06:44:44 UTC
[jira] Created: (HBASE-986) Cache sockets in DFSClient
Cache sockets in DFSClient
--------------------------
Key: HBASE-986
URL: https://issues.apache.org/jira/browse/HBASE-986
Project: Hadoop HBase
Issue Type: Improvement
Reporter: stack
Priority: Blocker
Fix For: 0.19.0
Talking with Dhruba here at Apachecon, he suggested that setup of sockets is big price payed during random-read. Stefan hacked into DFSClient an ugly patch that would allow us quickly test random-reading on top of a cache of cached sockets. I just tested it. I see 4X improvement.
Making this a blocker on 0.19.0. Looks like we can hack in a real socket pool using a custom SocketFactory under DFSClient. It'll be messy but the promised improvement is big.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-986) Cache sockets in DFSClient
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack reassigned HBASE-986:
---------------------------
Assignee: stack
> Cache sockets in DFSClient
> --------------------------
>
> Key: HBASE-986
> URL: https://issues.apache.org/jira/browse/HBASE-986
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Assignee: stack
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: socketCache.patch
>
>
> Talking with Dhruba here at Apachecon, he suggested that setup of sockets is big price payed during random-read. Stefan hacked into DFSClient an ugly patch that would allow us quickly test random-reading on top of a cache of cached sockets. I just tested it. I see 4X improvement.
> Making this a blocker on 0.19.0. Looks like we can hack in a real socket pool using a custom SocketFactory under DFSClient. It'll be messy but the promised improvement is big.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-986) Cache sockets in DFSClient
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-986:
------------------------
Fix Version/s: (was: 0.19.0)
0.20.0
Moving out of 0.19. Needs changes server-side in DataXceiver*.java. Ain't going to happen for this release.
> Cache sockets in DFSClient
> --------------------------
>
> Key: HBASE-986
> URL: https://issues.apache.org/jira/browse/HBASE-986
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Assignee: stack
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: socketCache.patch
>
>
> Talking with Dhruba here at Apachecon, he suggested that setup of sockets is big price payed during random-read. Stefan hacked into DFSClient an ugly patch that would allow us quickly test random-reading on top of a cache of cached sockets. I just tested it. I see 4X improvement.
> Making this a blocker on 0.19.0. Looks like we can hack in a real socket pool using a custom SocketFactory under DFSClient. It'll be messy but the promised improvement is big.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-986) Cache sockets in DFSClient
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-986.
-------------------------
Resolution: Later
Resolving as Later. Need datanode fixes first.
> Cache sockets in DFSClient
> --------------------------
>
> Key: HBASE-986
> URL: https://issues.apache.org/jira/browse/HBASE-986
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Assignee: stack
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: socketCache.patch
>
>
> Talking with Dhruba here at Apachecon, he suggested that setup of sockets is big price payed during random-read. Stefan hacked into DFSClient an ugly patch that would allow us quickly test random-reading on top of a cache of cached sockets. I just tested it. I see 4X improvement.
> Making this a blocker on 0.19.0. Looks like we can hack in a real socket pool using a custom SocketFactory under DFSClient. It'll be messy but the promised improvement is big.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-986) Cache sockets in DFSClient
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-986:
------------------------
Attachment: socketCache.patch
Hack to DFSClient adding a cache of sockets by Stefan Groschupf.
> Cache sockets in DFSClient
> --------------------------
>
> Key: HBASE-986
> URL: https://issues.apache.org/jira/browse/HBASE-986
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: socketCache.patch
>
>
> Talking with Dhruba here at Apachecon, he suggested that setup of sockets is big price payed during random-read. Stefan hacked into DFSClient an ugly patch that would allow us quickly test random-reading on top of a cache of cached sockets. I just tested it. I see 4X improvement.
> Making this a blocker on 0.19.0. Looks like we can hack in a real socket pool using a custom SocketFactory under DFSClient. It'll be messy but the promised improvement is big.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-986) Cache sockets in DFSClient
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645710#action_12645710 ]
stack commented on HBASE-986:
-----------------------------
The above 4X is a misreporting; I had a big cache configured when running tests. Will be back with proper numbers.
> Cache sockets in DFSClient
> --------------------------
>
> Key: HBASE-986
> URL: https://issues.apache.org/jira/browse/HBASE-986
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Assignee: stack
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: socketCache.patch
>
>
> Talking with Dhruba here at Apachecon, he suggested that setup of sockets is big price payed during random-read. Stefan hacked into DFSClient an ugly patch that would allow us quickly test random-reading on top of a cache of cached sockets. I just tested it. I see 4X improvement.
> Making this a blocker on 0.19.0. Looks like we can hack in a real socket pool using a custom SocketFactory under DFSClient. It'll be messy but the promised improvement is big.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-986) Cache sockets in DFSClient
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645705#action_12645705 ]
stack commented on HBASE-986:
-----------------------------
>From Dhruba, related HADOOP-3672
> Cache sockets in DFSClient
> --------------------------
>
> Key: HBASE-986
> URL: https://issues.apache.org/jira/browse/HBASE-986
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Assignee: stack
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: socketCache.patch
>
>
> Talking with Dhruba here at Apachecon, he suggested that setup of sockets is big price payed during random-read. Stefan hacked into DFSClient an ugly patch that would allow us quickly test random-reading on top of a cache of cached sockets. I just tested it. I see 4X improvement.
> Making this a blocker on 0.19.0. Looks like we can hack in a real socket pool using a custom SocketFactory under DFSClient. It'll be messy but the promised improvement is big.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.