You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Dmytro Molkov (JIRA)" <ji...@apache.org> on 2010/05/14 18:44:42 UTC
[jira] Created: (HADOOP-6764) Add number of reader threads and
queue length as configuration parameters in RPC.getServer
Add number of reader threads and queue length as configuration parameters in RPC.getServer
------------------------------------------------------------------------------------------
Key: HADOOP-6764
URL: https://issues.apache.org/jira/browse/HADOOP-6764
Project: Hadoop Common
Issue Type: Improvement
Reporter: Dmytro Molkov
Assignee: Dmytro Molkov
In HDFS-599 we are introducing multiple RPC servers running inside of the same process on different ports. Since one might want to configure these servers differently we need a good abstraction to pass configuration values to servers as parameters, not through Configuration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6764) Add number of reader threads and
queue length as configuration parameters in RPC.getServer
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12909622#action_12909622 ]
Hadoop QA commented on HADOOP-6764:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12453608/HADOOP-6764.2.patch
against trunk revision 995285.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
-1 system tests framework. The patch failed system tests framework compile.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h9.grid.sp2.yahoo.net/43/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h9.grid.sp2.yahoo.net/43/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h9.grid.sp2.yahoo.net/43/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h9.grid.sp2.yahoo.net/43/console
This message is automatically generated.
> Add number of reader threads and queue length as configuration parameters in RPC.getServer
> ------------------------------------------------------------------------------------------
>
> Key: HADOOP-6764
> URL: https://issues.apache.org/jira/browse/HADOOP-6764
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Dmytro Molkov
> Assignee: Dmytro Molkov
> Fix For: 0.22.0
>
> Attachments: HADOOP-6764.2.patch, HADOOP-6764.patch
>
>
> In HDFS-599 we are introducing multiple RPC servers running inside of the same process on different ports. Since one might want to configure these servers differently we need a good abstraction to pass configuration values to servers as parameters, not through Configuration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6764) Add number of reader threads and
queue length as configuration parameters in RPC.getServer
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904896#action_12904896 ]
Hairong Kuang commented on HADOOP-6764:
---------------------------------------
Param name queueSize/handlerQueueSize is better to be queueSizePerHandler.
The default value of queueSizePerHandler and numReader should not be 0, right?
> Add number of reader threads and queue length as configuration parameters in RPC.getServer
> ------------------------------------------------------------------------------------------
>
> Key: HADOOP-6764
> URL: https://issues.apache.org/jira/browse/HADOOP-6764
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Dmytro Molkov
> Assignee: Dmytro Molkov
> Attachments: HADOOP-6764.patch
>
>
> In HDFS-599 we are introducing multiple RPC servers running inside of the same process on different ports. Since one might want to configure these servers differently we need a good abstraction to pass configuration values to servers as parameters, not through Configuration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6764) Add number of reader threads and
queue length as configuration parameters in RPC.getServer
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12905325#action_12905325 ]
Hairong Kuang commented on HADOOP-6764:
---------------------------------------
+1. The patch looks good to me.
> Add number of reader threads and queue length as configuration parameters in RPC.getServer
> ------------------------------------------------------------------------------------------
>
> Key: HADOOP-6764
> URL: https://issues.apache.org/jira/browse/HADOOP-6764
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Dmytro Molkov
> Assignee: Dmytro Molkov
> Fix For: 0.22.0
>
> Attachments: HADOOP-6764.2.patch, HADOOP-6764.patch
>
>
> In HDFS-599 we are introducing multiple RPC servers running inside of the same process on different ports. Since one might want to configure these servers differently we need a good abstraction to pass configuration values to servers as parameters, not through Configuration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6764) Add number of reader threads and
queue length as configuration parameters in RPC.getServer
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12928512#action_12928512 ]
Hadoop QA commented on HADOOP-6764:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12453608/HADOOP-6764.2.patch
against trunk revision 1031422.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/34//testReport/
Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/34//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/34//console
This message is automatically generated.
> Add number of reader threads and queue length as configuration parameters in RPC.getServer
> ------------------------------------------------------------------------------------------
>
> Key: HADOOP-6764
> URL: https://issues.apache.org/jira/browse/HADOOP-6764
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Dmytro Molkov
> Assignee: Dmytro Molkov
> Fix For: 0.22.0
>
> Attachments: HADOOP-6764.2.patch, HADOOP-6764.patch
>
>
> In HDFS-599 we are introducing multiple RPC servers running inside of the same process on different ports. Since one might want to configure these servers differently we need a good abstraction to pass configuration values to servers as parameters, not through Configuration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6764) Add number of reader threads and
queue length as configuration parameters in RPC.getServer
Posted by "Dmytro Molkov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916658#action_12916658 ]
Dmytro Molkov commented on HADOOP-6764:
---------------------------------------
System tests failed because of the timeout in compilation. Should I resubmit the patch for tests?
> Add number of reader threads and queue length as configuration parameters in RPC.getServer
> ------------------------------------------------------------------------------------------
>
> Key: HADOOP-6764
> URL: https://issues.apache.org/jira/browse/HADOOP-6764
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Dmytro Molkov
> Assignee: Dmytro Molkov
> Fix For: 0.22.0
>
> Attachments: HADOOP-6764.2.patch, HADOOP-6764.patch
>
>
> In HDFS-599 we are introducing multiple RPC servers running inside of the same process on different ports. Since one might want to configure these servers differently we need a good abstraction to pass configuration values to servers as parameters, not through Configuration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6764) Add number of reader threads and
queue length as configuration parameters in RPC.getServer
Posted by "Dmytro Molkov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmytro Molkov updated HADOOP-6764:
----------------------------------
Status: Patch Available (was: Open)
Doing hudson again
> Add number of reader threads and queue length as configuration parameters in RPC.getServer
> ------------------------------------------------------------------------------------------
>
> Key: HADOOP-6764
> URL: https://issues.apache.org/jira/browse/HADOOP-6764
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Dmytro Molkov
> Assignee: Dmytro Molkov
> Fix For: 0.22.0
>
> Attachments: HADOOP-6764.2.patch, HADOOP-6764.patch
>
>
> In HDFS-599 we are introducing multiple RPC servers running inside of the same process on different ports. Since one might want to configure these servers differently we need a good abstraction to pass configuration values to servers as parameters, not through Configuration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6764) Add number of reader threads and
queue length as configuration parameters in RPC.getServer
Posted by "Dmytro Molkov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmytro Molkov updated HADOOP-6764:
----------------------------------
Attachment: HADOOP-6764.2.patch
Changed the name of the parameter.
And I am now using -1 as the default value for clarity. In the implementation of the Server if the parameters are not -1 they will be used for server configuration, otherwise the CommonConfigurationKeys.IPC_SERVER_HANDLER_QUEUE_SIZE_KEY and CommonConfigurationKeys.IPC_SERVER_RPC_READ_THREADS_KEY will be fetched from the conf and used.
> Add number of reader threads and queue length as configuration parameters in RPC.getServer
> ------------------------------------------------------------------------------------------
>
> Key: HADOOP-6764
> URL: https://issues.apache.org/jira/browse/HADOOP-6764
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Dmytro Molkov
> Assignee: Dmytro Molkov
> Attachments: HADOOP-6764.2.patch, HADOOP-6764.patch
>
>
> In HDFS-599 we are introducing multiple RPC servers running inside of the same process on different ports. Since one might want to configure these servers differently we need a good abstraction to pass configuration values to servers as parameters, not through Configuration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6764) Add number of reader threads and
queue length as configuration parameters in RPC.getServer
Posted by "Dmytro Molkov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmytro Molkov updated HADOOP-6764:
----------------------------------
Attachment: HADOOP-6764.patch
Please have a look at the patch.
It is a rather simple one. Just adding the number of readers and the queue size per handler as parameters to the RPC.getServer in case the application wants to provide those in the code rather than through the configuration.
The main reason for this is HDFS-599 where we are introducing two RPC Servers inside the same process and we want to be able to configure them differently.
> Add number of reader threads and queue length as configuration parameters in RPC.getServer
> ------------------------------------------------------------------------------------------
>
> Key: HADOOP-6764
> URL: https://issues.apache.org/jira/browse/HADOOP-6764
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Dmytro Molkov
> Assignee: Dmytro Molkov
> Attachments: HADOOP-6764.patch
>
>
> In HDFS-599 we are introducing multiple RPC servers running inside of the same process on different ports. Since one might want to configure these servers differently we need a good abstraction to pass configuration values to servers as parameters, not through Configuration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6764) Add number of reader threads and
queue length as configuration parameters in RPC.getServer
Posted by "Dmytro Molkov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmytro Molkov updated HADOOP-6764:
----------------------------------
Status: Open (was: Patch Available)
> Add number of reader threads and queue length as configuration parameters in RPC.getServer
> ------------------------------------------------------------------------------------------
>
> Key: HADOOP-6764
> URL: https://issues.apache.org/jira/browse/HADOOP-6764
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Dmytro Molkov
> Assignee: Dmytro Molkov
> Fix For: 0.22.0
>
> Attachments: HADOOP-6764.2.patch, HADOOP-6764.patch
>
>
> In HDFS-599 we are introducing multiple RPC servers running inside of the same process on different ports. Since one might want to configure these servers differently we need a good abstraction to pass configuration values to servers as parameters, not through Configuration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6764) Add number of reader threads and
queue length as configuration parameters in RPC.getServer
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-6764:
----------------------------------
Status: Patch Available (was: Open)
Hadoop Flags: [Reviewed]
Fix Version/s: 0.22.0
> Add number of reader threads and queue length as configuration parameters in RPC.getServer
> ------------------------------------------------------------------------------------------
>
> Key: HADOOP-6764
> URL: https://issues.apache.org/jira/browse/HADOOP-6764
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Dmytro Molkov
> Assignee: Dmytro Molkov
> Fix For: 0.22.0
>
> Attachments: HADOOP-6764.2.patch, HADOOP-6764.patch
>
>
> In HDFS-599 we are introducing multiple RPC servers running inside of the same process on different ports. Since one might want to configure these servers differently we need a good abstraction to pass configuration values to servers as parameters, not through Configuration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.