You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Christian Kunz (JIRA)" <ji...@apache.org> on 2007/01/19 23:48:29 UTC
[jira] Created: (HADOOP-911) Multithreading issue with libhdfs
library
Multithreading issue with libhdfs library
-----------------------------------------
Key: HADOOP-911
URL: https://issues.apache.org/jira/browse/HADOOP-911
Project: Hadoop
Issue Type: Bug
Components: libhdfs
Affects Versions: 0.9.2
Reporter: Christian Kunz
Multithreaded applications using libhdfs sometimes run into IllegalMonitorStateException or plainly lock up (strace shows a thread being in a loop of calling sched_yield). It probably has to do with the fact that libhdfs does not ensure proper allocation of hashtable entries that map a threadId to JNI interface pointer.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-911) Multithreading issue with libhdfs
library
Posted by "Christian Kunz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12466314 ]
Christian Kunz commented on HADOOP-911:
---------------------------------------
Fixing the storage into the hashtable brings up another issue: thread ids seem to get recycled such that multi-threaded applications using threads with limited lifetime might crash when spawning new threads, i.e. insertion of thread ids into a hashtable is useless. And it is not necessary because AttachCurrentThread can easily be called multiple times for already attached threads without performance penalty (according to documentation basically a no-op).
On the other hand, the library should also contain a function that calls DetachCurrentThread to avoid leaks (of course, no big deal for applications that use a relatively small number of threads or a thread pool).
> Multithreading issue with libhdfs library
> -----------------------------------------
>
> Key: HADOOP-911
> URL: https://issues.apache.org/jira/browse/HADOOP-911
> Project: Hadoop
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 0.9.2
> Reporter: Christian Kunz
>
> Multithreaded applications using libhdfs sometimes run into IllegalMonitorStateException or plainly lock up (strace shows a thread being in a loop of calling sched_yield). It probably has to do with the fact that libhdfs does not ensure proper allocation of hashtable entries that map a threadId to JNI interface pointer.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-911) Multithreading issue with libhdfs
library
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12511507 ]
Arun C Murthy commented on HADOOP-911:
--------------------------------------
+1
> Multithreading issue with libhdfs library
> -----------------------------------------
>
> Key: HADOOP-911
> URL: https://issues.apache.org/jira/browse/HADOOP-911
> Project: Hadoop
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 0.9.2
> Reporter: Christian Kunz
> Priority: Blocker
> Fix For: 0.14.0
>
> Attachments: libhdfs.patch
>
>
> Multithreaded applications using libhdfs sometimes run into IllegalMonitorStateException or plainly lock up (strace shows a thread being in a loop of calling sched_yield). It probably has to do with the fact that libhdfs does not ensure proper allocation of hashtable entries that map a threadId to JNI interface pointer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-911) Multithreading issue with libhdfs
library
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-911:
---------------------------------
Status: Patch Available (was: Open)
> Multithreading issue with libhdfs library
> -----------------------------------------
>
> Key: HADOOP-911
> URL: https://issues.apache.org/jira/browse/HADOOP-911
> Project: Hadoop
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 0.9.2
> Reporter: Christian Kunz
> Priority: Blocker
> Fix For: 0.14.0
>
> Attachments: libhdfs.patch
>
>
> Multithreaded applications using libhdfs sometimes run into IllegalMonitorStateException or plainly lock up (strace shows a thread being in a loop of calling sched_yield). It probably has to do with the fact that libhdfs does not ensure proper allocation of hashtable entries that map a threadId to JNI interface pointer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-911) Multithreading issue with libhdfs
library
Posted by "Christian Kunz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christian Kunz updated HADOOP-911:
----------------------------------
Attachment: libhdfs.patch
I attached the patch.
> Multithreading issue with libhdfs library
> -----------------------------------------
>
> Key: HADOOP-911
> URL: https://issues.apache.org/jira/browse/HADOOP-911
> Project: Hadoop
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 0.9.2
> Reporter: Christian Kunz
> Priority: Blocker
> Fix For: 0.14.0
>
> Attachments: libhdfs.patch
>
>
> Multithreaded applications using libhdfs sometimes run into IllegalMonitorStateException or plainly lock up (strace shows a thread being in a loop of calling sched_yield). It probably has to do with the fact that libhdfs does not ensure proper allocation of hashtable entries that map a threadId to JNI interface pointer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-911) Multithreading issue with libhdfs
library
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-911:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Christian!
> Multithreading issue with libhdfs library
> -----------------------------------------
>
> Key: HADOOP-911
> URL: https://issues.apache.org/jira/browse/HADOOP-911
> Project: Hadoop
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 0.9.2
> Reporter: Christian Kunz
> Assignee: Christian Kunz
> Priority: Blocker
> Fix For: 0.14.0
>
> Attachments: libhdfs.patch
>
>
> Multithreaded applications using libhdfs sometimes run into IllegalMonitorStateException or plainly lock up (strace shows a thread being in a loop of calling sched_yield). It probably has to do with the fact that libhdfs does not ensure proper allocation of hashtable entries that map a threadId to JNI interface pointer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-911) Multithreading issue with libhdfs
library
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12511518 ]
Hadoop QA commented on HADOOP-911:
----------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12361454/libhdfs.patch applied and successfully tested against trunk revision r554811.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/389/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/389/console
> Multithreading issue with libhdfs library
> -----------------------------------------
>
> Key: HADOOP-911
> URL: https://issues.apache.org/jira/browse/HADOOP-911
> Project: Hadoop
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 0.9.2
> Reporter: Christian Kunz
> Priority: Blocker
> Fix For: 0.14.0
>
> Attachments: libhdfs.patch
>
>
> Multithreaded applications using libhdfs sometimes run into IllegalMonitorStateException or plainly lock up (strace shows a thread being in a loop of calling sched_yield). It probably has to do with the fact that libhdfs does not ensure proper allocation of hashtable entries that map a threadId to JNI interface pointer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-911) Multithreading issue with libhdfs
library
Posted by "Sameer Paranjpye (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sameer Paranjpye updated HADOOP-911:
------------------------------------
Assignee: Christian Kunz
> Multithreading issue with libhdfs library
> -----------------------------------------
>
> Key: HADOOP-911
> URL: https://issues.apache.org/jira/browse/HADOOP-911
> Project: Hadoop
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 0.9.2
> Reporter: Christian Kunz
> Assignee: Christian Kunz
> Priority: Blocker
> Fix For: 0.14.0
>
> Attachments: libhdfs.patch
>
>
> Multithreaded applications using libhdfs sometimes run into IllegalMonitorStateException or plainly lock up (strace shows a thread being in a loop of calling sched_yield). It probably has to do with the fact that libhdfs does not ensure proper allocation of hashtable entries that map a threadId to JNI interface pointer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-911) Multithreading issue with libhdfs
library
Posted by "Sameer Paranjpye (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sameer Paranjpye updated HADOOP-911:
------------------------------------
Fix Version/s: 0.14.0
Priority: Blocker (was: Major)
Christian, could you please submit your patch for this issue?
> Multithreading issue with libhdfs library
> -----------------------------------------
>
> Key: HADOOP-911
> URL: https://issues.apache.org/jira/browse/HADOOP-911
> Project: Hadoop
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 0.9.2
> Reporter: Christian Kunz
> Priority: Blocker
> Fix For: 0.14.0
>
>
> Multithreaded applications using libhdfs sometimes run into IllegalMonitorStateException or plainly lock up (strace shows a thread being in a loop of calling sched_yield). It probably has to do with the fact that libhdfs does not ensure proper allocation of hashtable entries that map a threadId to JNI interface pointer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-911) Multithreading issue with libhdfs
library
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12512058 ]
Hudson commented on HADOOP-911:
-------------------------------
Integrated in Hadoop-Nightly #152 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/152/])
> Multithreading issue with libhdfs library
> -----------------------------------------
>
> Key: HADOOP-911
> URL: https://issues.apache.org/jira/browse/HADOOP-911
> Project: Hadoop
> Issue Type: Bug
> Components: libhdfs
> Affects Versions: 0.9.2
> Reporter: Christian Kunz
> Assignee: Christian Kunz
> Priority: Blocker
> Fix For: 0.14.0
>
> Attachments: libhdfs.patch
>
>
> Multithreaded applications using libhdfs sometimes run into IllegalMonitorStateException or plainly lock up (strace shows a thread being in a loop of calling sched_yield). It probably has to do with the fact that libhdfs does not ensure proper allocation of hashtable entries that map a threadId to JNI interface pointer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.