You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Bill Graham (JIRA)" <ji...@apache.org> on 2010/04/28 20:30:48 UTC

[jira] Created: (CHUKWA-483) bin/chukwa stop broken

bin/chukwa stop broken
----------------------

                 Key: CHUKWA-483
                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
             Project: Hadoop Chukwa
          Issue Type: Bug
          Components: scripts
         Environment: Redhat EL 4.x, Java 1.6 
            Reporter: Bill Graham


Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (CHUKWA-483) bin/chukwa stop broken

Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12863999#action_12863999 ] 

Ari Rabkin commented on CHUKWA-483:
-----------------------------------

As per eric's okay, I just committed this.  Thanks Bill for the patch, also Kirk and Eric for testing.

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch, CHUKWA-483.threaddump.txt, chukwa-agent-thread-dump.txt
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (CHUKWA-483) bin/chukwa stop broken

Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862965#action_12862965 ] 

Bill Graham commented on CHUKWA-483:
------------------------------------

The DestroyJavaVM thread is always running, before I even execute a 'kill
-1' command. 'kill -1' just isn't stopping this process, even after I wait 5
minutes.






> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch, CHUKWA-483.threaddump.txt
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (CHUKWA-483) bin/chukwa stop broken

Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862964#action_12862964 ] 

Bill Graham commented on CHUKWA-483:
------------------------------------

The DestroyJavaVM thread is always running, before I even execute a 'kill -1' command. 'kill -1' just isn't stopping this process, even after I wait 5 minutes.

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch, CHUKWA-483.threaddump.txt
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (CHUKWA-483) bin/chukwa stop broken

Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862279#action_12862279 ] 

Ari Rabkin commented on CHUKWA-483:
-----------------------------------

Huhm.  I haven't been particularly paying attention to the script-side of things.  

Eric, can you comment on why the kill -1 isn't killing the process?  Didn't think we were masking that signal.

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (CHUKWA-483) bin/chukwa stop broken

Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eric Yang updated CHUKWA-483:
-----------------------------

    Attachment: chukwa-agent-thread-dump.txt

Thread dump, only UTF8CharFileTailing adaptor and ExecPluginAdaptor are running, before kill -1

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch, CHUKWA-483.threaddump.txt, chukwa-agent-thread-dump.txt
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (CHUKWA-483) bin/chukwa stop broken

Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bill Graham updated CHUKWA-483:
-------------------------------

    Attachment: CHUKWA-483.threaddump.txt

Sure. Here's a thread dump of an agent with no adaptors. I start it, try to stop it with bin/chukwa (using kill -1), the run kill -3.

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch, CHUKWA-483.threaddump.txt
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (CHUKWA-483) bin/chukwa stop broken

Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862928#action_12862928 ] 

Eric Yang commented on CHUKWA-483:
----------------------------------

Looks like DestroyJavaVM thread was running.  This means it should be able to terminate the program.  Maybe it take longer than expected to clean up.

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch, CHUKWA-483.threaddump.txt
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (CHUKWA-483) bin/chukwa stop broken

Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12863896#action_12863896 ] 

Bill Graham commented on CHUKWA-483:
------------------------------------

I do not see core files on my OS when using -TERM.

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch, CHUKWA-483.threaddump.txt, chukwa-agent-thread-dump.txt
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Assigned: (CHUKWA-483) bin/chukwa stop broken

Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bill Graham reassigned CHUKWA-483:
----------------------------------

    Assignee: Bill Graham

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (CHUKWA-483) bin/chukwa stop broken

Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12863431#action_12863431 ] 

Bill Graham commented on CHUKWA-483:
------------------------------------

If I check out the trunk, package and run bin/startAgents.sh without any initial adaptors, then issue kill -3, I see  the DestroyJavaVM thread. It's always there. I see the same thing if I kill -3 a collector process. I don't see it on a DemuxManager process for what it's worth.

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch, CHUKWA-483.threaddump.txt, chukwa-agent-thread-dump.txt
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (CHUKWA-483) bin/chukwa stop broken

Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12863893#action_12863893 ] 

Eric Yang commented on CHUKWA-483:
----------------------------------

Bill and Kirk, do you guys see core.[pid] files pile up in CHUKWA_HOME directory, if the script uses -TERM on your OS?  If you don't see the core file, then I am ok with this patch.

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch, CHUKWA-483.threaddump.txt, chukwa-agent-thread-dump.txt
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (CHUKWA-483) bin/chukwa stop broken

Posted by "Kirk True (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12863843#action_12863843 ] 

Kirk True commented on CHUKWA-483:
----------------------------------

I'm also seeing this behavior starting in 0.4.0 on Fedora. Let me know if there's any useful information I can add.

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch, CHUKWA-483.threaddump.txt, chukwa-agent-thread-dump.txt
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (CHUKWA-483) bin/chukwa stop broken

Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bill Graham updated CHUKWA-483:
-------------------------------

    Status: Patch Available  (was: Open)

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (CHUKWA-483) bin/chukwa stop broken

Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ari Rabkin updated CHUKWA-483:
------------------------------

        Status: Resolved  (was: Patch Available)
    Resolution: Fixed

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch, CHUKWA-483.threaddump.txt, chukwa-agent-thread-dump.txt
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (CHUKWA-483) bin/chukwa stop broken

Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12863159#action_12863159 ] 

Eric Yang commented on CHUKWA-483:
----------------------------------

The DestroyJavaVM thread shouldn't be created unless the main thread is prepared to exit.  Do you have special agent adaptor and collector writer which is preventing the system to exit (i.e. forget to free jni resource, etc)  I will enclose my thread dump for reference.

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch, CHUKWA-483.threaddump.txt
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (CHUKWA-483) bin/chukwa stop broken

Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862288#action_12862288 ] 

Eric Yang commented on CHUKWA-483:
----------------------------------

kill -1 works fine for me with Java 6, and Redhat EL 5.1.  Bill, could you do a kill -3 after kill -1 to see what the process is doing?  Perhaps the existing process was taking a while to terminate?

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-483.1.patch
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (CHUKWA-483) bin/chukwa stop broken

Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CHUKWA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bill Graham updated CHUKWA-483:
-------------------------------

    Attachment: CHUKWA-483.1.patch

> bin/chukwa stop broken
> ----------------------
>
>                 Key: CHUKWA-483
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-483
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: scripts
>         Environment: Redhat EL 4.x, Java 1.6 
>            Reporter: Bill Graham
>         Attachments: CHUKWA-483.1.patch
>
>
> Since JIRA-468, bin/chukwa stop no longer works for collectors and agents. Can we change kill -1 back to kill -TERM?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.