You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Owen O'Malley (JIRA)" <ji...@apache.org> on 2007/09/14 20:44:32 UTC
[jira] Created: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
locking for the ReflectionUtils.logThreadInfo is too conservative
-----------------------------------------------------------------
Key: HADOOP-1898
URL: https://issues.apache.org/jira/browse/HADOOP-1898
Project: Hadoop
Issue Type: Bug
Components: util
Reporter: Owen O'Malley
Assignee: Owen O'Malley
Fix For: 0.15.0
When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12535281 ]
Hadoop QA commented on HADOOP-1898:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12367802/patch-1898.txt
against trunk revision r584939.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests -1. The patch failed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/956/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/956/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/956/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/956/console
This message is automatically generated.
> locking for the ReflectionUtils.logThreadInfo is too conservative
> -----------------------------------------------------------------
>
> Key: HADOOP-1898
> URL: https://issues.apache.org/jira/browse/HADOOP-1898
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: patch-1898.txt, patch-1898.txt, patch-1898.txt
>
>
> When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-1898:
----------------------------------
Fix Version/s: 0.16.0
Assignee: Amareshwari Sri Ramadasu (was: Owen O'Malley)
Affects Version/s: 0.15.0
> locking for the ReflectionUtils.logThreadInfo is too conservative
> -----------------------------------------------------------------
>
> Key: HADOOP-1898
> URL: https://issues.apache.org/jira/browse/HADOOP-1898
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.16.0
>
> Attachments: patch-1898.txt, patch-1898.txt, patch-1898.txt
>
>
> When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-1898:
----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Amareshwari!
> locking for the ReflectionUtils.logThreadInfo is too conservative
> -----------------------------------------------------------------
>
> Key: HADOOP-1898
> URL: https://issues.apache.org/jira/browse/HADOOP-1898
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.16.0
>
> Attachments: patch-1898.txt, patch-1898.txt, patch-1898.txt, patch-1898.txt
>
>
> When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
Posted by "Amareshwari Sri Ramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sri Ramadasu updated HADOOP-1898:
---------------------------------------------
Attachment: patch-1898.txt
> locking for the ReflectionUtils.logThreadInfo is too conservative
> -----------------------------------------------------------------
>
> Key: HADOOP-1898
> URL: https://issues.apache.org/jira/browse/HADOOP-1898
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: patch-1898.txt, patch-1898.txt
>
>
> When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
Posted by "Amareshwari Sri Ramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sri Ramadasu updated HADOOP-1898:
---------------------------------------------
Attachment: patch-1898.txt
> locking for the ReflectionUtils.logThreadInfo is too conservative
> -----------------------------------------------------------------
>
> Key: HADOOP-1898
> URL: https://issues.apache.org/jira/browse/HADOOP-1898
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.16.0
>
> Attachments: patch-1898.txt, patch-1898.txt, patch-1898.txt, patch-1898.txt
>
>
> When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-1898:
----------------------------------
Status: Open (was: Patch Available)
> locking for the ReflectionUtils.logThreadInfo is too conservative
> -----------------------------------------------------------------
>
> Key: HADOOP-1898
> URL: https://issues.apache.org/jira/browse/HADOOP-1898
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.16.0
>
> Attachments: patch-1898.txt, patch-1898.txt, patch-1898.txt, patch-1898.txt
>
>
> When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
Posted by "Amareshwari Sri Ramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sri Ramadasu updated HADOOP-1898:
---------------------------------------------
Attachment: patch-1898.txt
I did the changes for locking ReflectionUtils. Here static synchronized method is replaced by a method in a class object. This would not be conservative locking now.
> locking for the ReflectionUtils.logThreadInfo is too conservative
> -----------------------------------------------------------------
>
> Key: HADOOP-1898
> URL: https://issues.apache.org/jira/browse/HADOOP-1898
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: patch-1898.txt
>
>
> When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
Posted by "Amareshwari Sri Ramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sri Ramadasu updated HADOOP-1898:
---------------------------------------------
Status: Open (was: Patch Available)
> locking for the ReflectionUtils.logThreadInfo is too conservative
> -----------------------------------------------------------------
>
> Key: HADOOP-1898
> URL: https://issues.apache.org/jira/browse/HADOOP-1898
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.16.0
>
> Attachments: patch-1898.txt, patch-1898.txt, patch-1898.txt, patch-1898.txt
>
>
> When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12537680 ]
Hudson commented on HADOOP-1898:
--------------------------------
Integrated in Hadoop-Nightly #282 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/282/])
> locking for the ReflectionUtils.logThreadInfo is too conservative
> -----------------------------------------------------------------
>
> Key: HADOOP-1898
> URL: https://issues.apache.org/jira/browse/HADOOP-1898
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.16.0
>
> Attachments: patch-1898.txt, patch-1898.txt, patch-1898.txt, patch-1898.txt
>
>
> When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
Posted by "Amareshwari Sri Ramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sri Ramadasu updated HADOOP-1898:
---------------------------------------------
Attachment: patch-1898.txt
Removed Synchronized for the method and introduced synchronized block in the method.
> locking for the ReflectionUtils.logThreadInfo is too conservative
> -----------------------------------------------------------------
>
> Key: HADOOP-1898
> URL: https://issues.apache.org/jira/browse/HADOOP-1898
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: patch-1898.txt, patch-1898.txt, patch-1898.txt
>
>
> When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12535852 ]
Owen O'Malley commented on HADOOP-1898:
---------------------------------------
This looks right, but I'd suggest a minor style change in the patch:
{code}
boolean dumpStack = false;
synchronized (ReflectionUtils.class) {
long now = System.currentTimeMillis();
if (new - previousLogTime > minInterval * 1000) {
previousLogTime = now;
dumpStack = true;
}
}
if (dumpStack) {
..
}
{code}
> locking for the ReflectionUtils.logThreadInfo is too conservative
> -----------------------------------------------------------------
>
> Key: HADOOP-1898
> URL: https://issues.apache.org/jira/browse/HADOOP-1898
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.16.0
>
> Attachments: patch-1898.txt, patch-1898.txt, patch-1898.txt
>
>
> When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
Posted by "Amareshwari Sri Ramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sri Ramadasu updated HADOOP-1898:
---------------------------------------------
Status: Patch Available (was: Open)
Instead of previous patch changes of introducing a new class, now we do it by introducing a synchronized block in the method.
Any comments on the patch?
> locking for the ReflectionUtils.logThreadInfo is too conservative
> -----------------------------------------------------------------
>
> Key: HADOOP-1898
> URL: https://issues.apache.org/jira/browse/HADOOP-1898
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: patch-1898.txt, patch-1898.txt, patch-1898.txt
>
>
> When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12535892 ]
Hadoop QA commented on HADOOP-1898:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12367930/patch-1898.txt
against trunk revision r585648.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/966/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/966/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/966/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/966/console
This message is automatically generated.
> locking for the ReflectionUtils.logThreadInfo is too conservative
> -----------------------------------------------------------------
>
> Key: HADOOP-1898
> URL: https://issues.apache.org/jira/browse/HADOOP-1898
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.16.0
>
> Attachments: patch-1898.txt, patch-1898.txt, patch-1898.txt, patch-1898.txt
>
>
> When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
Posted by "Amareshwari Sri Ramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sri Ramadasu updated HADOOP-1898:
---------------------------------------------
Status: Patch Available (was: Open)
Submitting patch again with suggested change.
> locking for the ReflectionUtils.logThreadInfo is too conservative
> -----------------------------------------------------------------
>
> Key: HADOOP-1898
> URL: https://issues.apache.org/jira/browse/HADOOP-1898
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.16.0
>
> Attachments: patch-1898.txt, patch-1898.txt, patch-1898.txt, patch-1898.txt
>
>
> When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1898) locking for the
ReflectionUtils.logThreadInfo is too conservative
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-1898:
----------------------------------
Status: Patch Available (was: Open)
> locking for the ReflectionUtils.logThreadInfo is too conservative
> -----------------------------------------------------------------
>
> Key: HADOOP-1898
> URL: https://issues.apache.org/jira/browse/HADOOP-1898
> Project: Hadoop
> Issue Type: Bug
> Components: util
> Affects Versions: 0.15.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.16.0
>
> Attachments: patch-1898.txt, patch-1898.txt, patch-1898.txt, patch-1898.txt
>
>
> When the RPC servers get into trouble with their call queues backing up, they occasionally dump the call stacks. These are very useful for identifying hot spots, but the locking is too conservative and so all of the handlers are blocked while the thread call stacks are dumped.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.