You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Suresh Srinivas (JIRA)" <ji...@apache.org> on 2010/02/19 02:52:28 UTC
[jira] Created: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
IPC server response buffer reset threshold should be configurable
-----------------------------------------------------------------
Key: HADOOP-6577
URL: https://issues.apache.org/jira/browse/HADOOP-6577
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 0.22.0
Reporter: Suresh Srinivas
Assignee: Suresh Srinivas
Fix For: 0.22.0
In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836033#action_12836033 ]
Hadoop QA commented on HADOOP-6577:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12436400/hadoop-6577.2.patch
against trunk revision 911748.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/16/console
This message is automatically generated.
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.2.patch, hadoop-6577.2.rel20.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-6577:
------------------------------------
Status: Open (was: Patch Available)
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-6577:
------------------------------------
Attachment: hadoop-6577.2.patch
Attaching the patch again for hudson to pick up instead of branch 0.20 version.
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.2.patch, hadoop-6577.2.rel20.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835567#action_12835567 ]
Hadoop QA commented on HADOOP-6577:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12436280/hadoop-6577.patch
against trunk revision 911646.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/14/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/14/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/14/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/14/console
This message is automatically generated.
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836090#action_12836090 ]
Hudson commented on HADOOP-6577:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #178 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/178/])
. Add hidden configuration option "ipc.server.max.response.size" to change the default 1 MB, the maximum size when large IPC handler response buffer is reset. Contributed by Suresh Srinivas.
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.2.patch, hadoop-6577.2.rel20.patch, hadoop-6577.3.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835620#action_12835620 ]
Arun C Murthy commented on HADOOP-6577:
---------------------------------------
Minor nit, please add a variable to track the config knob:
{code}
static final String HADOOP_RPC_RESPONSE_SIZE = "ipc.server.max.response.size";
{code}
or some such.
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-6577:
------------------------------------
Resolution: Fixed
Release Note:
Add hidden configuration option "ipc.server.max.response.size" to change the default 1 MB, the maximum size when large IPC handler response buffer is reset.
Hadoop Flags: [Incompatible change, Reviewed]
Status: Resolved (was: Patch Available)
I committed the patch.
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.2.patch, hadoop-6577.2.rel20.patch, hadoop-6577.3.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-6577:
------------------------------------
Status: Open (was: Patch Available)
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-6577:
------------------------------------
Attachment: hadoop-6577.2.rel20.patch
Attaching patch for 0.20 branch.
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.2.rel20.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835552#action_12835552 ]
Suresh Srinivas commented on HADOOP-6577:
-----------------------------------------
When a high frequency of requests are made to namenode, each resulting in a response of size greater than 1MB, a lot of garbage is created on the heap. This could result in tenured heap getting filled up very fast and trigger full GC. Full GC results in long stop the world pauses, affecting the applications using HDFS.
In one of the instances observed on the production cluster, an application repeatedly made list status calls, each with response sizes ranging from 3 to 5MB. This resulted in full GC, which could have been avoided by setting the max response size to 10MB.
More permanent solution for this problem is to ensure an operation that results in large response (listStatus) are broken in to multi-step smaller operations. This will be addressed in a separate jira. In the interim I propose adding a hidden config param that could be used for setting the max resp buffer size.
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-6577:
------------------------------------
Attachment: hadoop-6577.3.patch
Attaching a new patch that applies with the latest trunk changes.
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.2.patch, hadoop-6577.2.rel20.patch, hadoop-6577.3.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-6577:
------------------------------------
Status: Patch Available (was: Open)
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836531#action_12836531 ]
dhruba borthakur commented on HADOOP-6577:
------------------------------------------
In your setup, are you going to be setting ipc.server.max.response.size to 10MB?
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.2.patch, hadoop-6577.2.rel20.patch, hadoop-6577.3.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836862#action_12836862 ]
Suresh Srinivas commented on HADOOP-6577:
-----------------------------------------
@dhruba - In your setup, are you going to be setting ipc.server.max.response.size to 10MB?
depending on the production cluster response sizes and patterns, we may have to set the resp buffer size to 10MB. With 100 handlers, this is 1GB hit on the heap.
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.2.patch, hadoop-6577.2.rel20.patch, hadoop-6577.3.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-6577:
------------------------------------
Status: Patch Available (was: Open)
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836154#action_12836154 ]
Hudson commented on HADOOP-6577:
--------------------------------
Integrated in Hadoop-Common-trunk #255 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/255/])
. Add hidden configuration option "ipc.server.max.response.size" to change the default 1 MB, the maximum size when large IPC handler response buffer is reset. Contributed by Suresh Srinivas.
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.2.patch, hadoop-6577.2.rel20.patch, hadoop-6577.3.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-6577:
------------------------------------
Attachment: hadoop-6577.2.patch
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-6577:
------------------------------------
Attachment: hadoop-6577.1.patch
New patch incorporating comments from Arun.
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836058#action_12836058 ]
Arun C Murthy commented on HADOOP-6577:
---------------------------------------
+1
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.2.patch, hadoop-6577.2.rel20.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836069#action_12836069 ]
Hadoop QA commented on HADOOP-6577:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12436407/hadoop-6577.3.patch
against trunk revision 911748.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/17/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/17/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/17/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/17/console
This message is automatically generated.
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.2.patch, hadoop-6577.2.rel20.patch, hadoop-6577.3.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-6577:
------------------------------------
Status: Patch Available (was: Open)
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.2.patch, hadoop-6577.2.rel20.patch, hadoop-6577.3.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-6577:
------------------------------------
Attachment: hadoop-6577.patch
Patch introduces new parameter "ipc.server.max.response.size" used for configuring the max response buffer size.
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-6577:
------------------------------------
Status: Open (was: Patch Available)
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.1.patch, hadoop-6577.2.patch, hadoop-6577.2.patch, hadoop-6577.2.rel20.patch, hadoop-6577.3.patch, hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6577) IPC server response buffer reset
threshold should be configurable
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-6577:
------------------------------------
Status: Patch Available (was: Open)
> IPC server response buffer reset threshold should be configurable
> -----------------------------------------------------------------
>
> Key: HADOOP-6577
> URL: https://issues.apache.org/jira/browse/HADOOP-6577
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hadoop-6577.patch
>
>
> In HDFS-6460, the response buffers in o.a.h.ipc.Server.Handler was reset when the buffer grows beyond max size of 1MB. This frees heap from large responses occupying it. This max response size limit should be configurable. Details in subsequent comment.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.