You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Steve Loughran (Created) (JIRA)" <ji...@apache.org> on 2011/12/02 18:17:40 UTC
[jira] [Created] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Review all Service.stop() operations and make sure that they work before a service is started
---------------------------------------------------------------------------------------------
Key: MAPREDUCE-3502
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
Project: Hadoop Map/Reduce
Issue Type: Task
Components: mrv2
Affects Versions: 0.23.0, 0.24.0
Reporter: Steve Loughran
Assignee: Steve Loughran
MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
# not calling other things if the other things are null
# not being re-entrant (i.e. make synchronized if possible),
Maybe
# have a StopService operation that only stops a service if it is live
# factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
my first iteration will skip the sync though it's something to consider.
Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Posted by "Steve Loughran (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167698#comment-13167698 ]
Steve Loughran commented on MAPREDUCE-3502:
-------------------------------------------
Improved patch
# javadoc of {{Service}} interface and {{AbstractService}} so as to more clearly specify the desired behaviour of implementations.
# common methods in {{AbstractService}} to interrupt a non-null thread, stop non-null IPC and webapp servers. These calls return null values to overwrite field variables after the successful operation.
# static methods to stop a service if non null, and one to do it with exceptions caught and logged.
All subclasses have been reviewed and their stop operations
# use the new common interrupt and stop methods where appropriate
# use checks for null fields before trying any stop operation
# nullify all non-final fields after their work.
Some -but not all- of the {{stop()}} methods join onto the thread that was just interrupted; to wait for it to stop. This would be the cleanest option, as it guarantees the worker threads will not invoke a (now-stopped) service. I have not changed the behaviour of any existing services that do not perform {{Thread.join()}} operations. If there was a well defined behaviour for that join (time to wait, exception to throw on failure), it could be injected into the new {{AbstractService.interruptThread()}} method, and used throughout the services. Having a consistent interrupt/join/report failure process would imply only one thing to test, and more trust that new services will follow the examples set in the MRv2 codebase.
This patch does nothing to address the issue of MAPREDUCE-3535 -that the stoppability of a service is not checked before child classes terminate; it is somewhat orthogonal.
This patch is designed to ensure that no matter when {{Service.stop()}} is called (except in the special situation of a re-entrant call on a separate thread), everything is shut down cleanly. This means that even if the failure is triggered halfway through the {{init(Conf)}} or {{start()}} operations the {{stop()}} operation will always clean up what there is.
> Review all Service.stop() operations and make sure that they work before a service is started
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: MAPREDUCE-3502.patch, MAPREDUCE-3502.patch
>
> Original Estimate: 24h
> Time Spent: 0.5h
> Remaining Estimate: 23.5h
>
> MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
> # not calling other things if the other things are null
> # not being re-entrant (i.e. make synchronized if possible),
> Maybe
> # have a StopService operation that only stops a service if it is live
> # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
> my first iteration will skip the sync though it's something to consider.
> Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Posted by "Steve Loughran (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated MAPREDUCE-3502:
--------------------------------------
Attachment: MAPREDUCE-3502.patch
> Review all Service.stop() operations and make sure that they work before a service is started
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: MAPREDUCE-3502.patch, MAPREDUCE-3502.patch
>
> Original Estimate: 24h
> Time Spent: 0.5h
> Remaining Estimate: 23.5h
>
> MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
> # not calling other things if the other things are null
> # not being re-entrant (i.e. make synchronized if possible),
> Maybe
> # have a StopService operation that only stops a service if it is live
> # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
> my first iteration will skip the sync though it's something to consider.
> Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work logged] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Posted by "Steve Loughran (Work logged) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel\#worklog-{worklog.getId()} ]
Steve Loughran logged work on MAPREDUCE-3502:
---------------------------------------------
Author: Steve Loughran
Created on: 12/Dec/11 19:30
Start Date: 12/Dec/11 19:30
Worklog Time Spent: 2h
Issue Time Tracking
-------------------
Worklog Id: (was: 12534)
Time Spent: 2.5h (was: 0.5h)
Remaining Estimate: 21.5h (was: 23.5h)
> Review all Service.stop() operations and make sure that they work before a service is started
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: MAPREDUCE-3502.patch, MAPREDUCE-3502.patch
>
> Original Estimate: 24h
> Time Spent: 2.5h
> Remaining Estimate: 21.5h
>
> MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
> # not calling other things if the other things are null
> # not being re-entrant (i.e. make synchronized if possible),
> Maybe
> # have a StopService operation that only stops a service if it is live
> # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
> my first iteration will skip the sync though it's something to consider.
> Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Posted by "Steve Loughran (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated MAPREDUCE-3502:
--------------------------------------
Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1)
Status: Patch Available (was: In Progress)
> Review all Service.stop() operations and make sure that they work before a service is started
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: MAPREDUCE-3502.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
> # not calling other things if the other things are null
> # not being re-entrant (i.e. make synchronized if possible),
> Maybe
> # have a StopService operation that only stops a service if it is live
> # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
> my first iteration will skip the sync though it's something to consider.
> Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Posted by "Robert Joseph Evans (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-3502:
-------------------------------------------
Target Version/s: 2.0.0, 3.0.0 (was: 0.24.0, 0.23.1)
> Review all Service.stop() operations and make sure that they work before a service is started
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: MAPREDUCE-3502.patch, MAPREDUCE-3502.patch
>
> Original Estimate: 24h
> Time Spent: 2.5h
> Remaining Estimate: 21.5h
>
> MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
> # not calling other things if the other things are null
> # not being re-entrant (i.e. make synchronized if possible),
> Maybe
> # have a StopService operation that only stops a service if it is live
> # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
> my first iteration will skip the sync though it's something to consider.
> Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Posted by "Steve Loughran (Work started) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on MAPREDUCE-3502 started by Steve Loughran.
> Review all Service.stop() operations and make sure that they work before a service is started
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
> # not calling other things if the other things are null
> # not being re-entrant (i.e. make synchronized if possible),
> Maybe
> # have a StopService operation that only stops a service if it is live
> # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
> my first iteration will skip the sync though it's something to consider.
> Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work logged] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Posted by "Steve Loughran (Work logged) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel\#worklog-{worklog.getId()} ]
Steve Loughran logged work on MAPREDUCE-3502:
---------------------------------------------
Author: Steve Loughran
Created on: 02/Dec/11 17:24
Start Date: 02/Dec/11 17:24
Worklog Time Spent: 0.5h
Issue Time Tracking
-------------------
Worklog Id: (was: 12434)
Time Spent: 0.5h
Remaining Estimate: 23.5h (was: 24h)
> Review all Service.stop() operations and make sure that they work before a service is started
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: MAPREDUCE-3502.patch
>
> Original Estimate: 24h
> Time Spent: 0.5h
> Remaining Estimate: 23.5h
>
> MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
> # not calling other things if the other things are null
> # not being re-entrant (i.e. make synchronized if possible),
> Maybe
> # have a StopService operation that only stops a service if it is live
> # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
> my first iteration will skip the sync though it's something to consider.
> Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Posted by "Steve Loughran (Work started) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on MAPREDUCE-3502 started by Steve Loughran.
> Review all Service.stop() operations and make sure that they work before a service is started
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: MAPREDUCE-3502.patch, MAPREDUCE-3502.patch
>
> Original Estimate: 24h
> Time Spent: 2.5h
> Remaining Estimate: 21.5h
>
> MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
> # not calling other things if the other things are null
> # not being re-entrant (i.e. make synchronized if possible),
> Maybe
> # have a StopService operation that only stops a service if it is live
> # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
> my first iteration will skip the sync though it's something to consider.
> Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161765#comment-13161765 ]
Hadoop QA commented on MAPREDUCE-3502:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505908/MAPREDUCE-3502.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 14 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1385//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1385//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1385//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1385//console
This message is automatically generated.
> Review all Service.stop() operations and make sure that they work before a service is started
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: MAPREDUCE-3502.patch
>
> Original Estimate: 24h
> Time Spent: 0.5h
> Remaining Estimate: 23.5h
>
> MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
> # not calling other things if the other things are null
> # not being re-entrant (i.e. make synchronized if possible),
> Maybe
> # have a StopService operation that only stops a service if it is live
> # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
> my first iteration will skip the sync though it's something to consider.
> Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Posted by "Steve Loughran (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202452#comment-13202452 ]
Steve Loughran commented on MAPREDUCE-3502:
-------------------------------------------
Plan
# submit Service w/ better shutdown (and helper) methods first
# submit patches to specific sections of the yarn codebase
> Review all Service.stop() operations and make sure that they work before a service is started
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: MAPREDUCE-3502.patch, MAPREDUCE-3502.patch
>
> Original Estimate: 24h
> Time Spent: 2.5h
> Remaining Estimate: 21.5h
>
> MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
> # not calling other things if the other things are null
> # not being re-entrant (i.e. make synchronized if possible),
> Maybe
> # have a StopService operation that only stops a service if it is live
> # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
> my first iteration will skip the sync though it's something to consider.
> Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Posted by "Steve Loughran (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated MAPREDUCE-3502:
--------------------------------------
Attachment: MAPREDUCE-3502.patch
First walk through; common problems, but hard to stop reliably. Better to check-and-bail-out-early across all subclasses.
I don't want to add a PENDING-SHUTDOWN statehowever, as that just complicates things more
> Review all Service.stop() operations and make sure that they work before a service is started
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: MAPREDUCE-3502.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
> # not calling other things if the other things are null
> # not being re-entrant (i.e. make synchronized if possible),
> Maybe
> # have a StopService operation that only stops a service if it is live
> # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
> my first iteration will skip the sync though it's something to consider.
> Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Posted by "Steve Loughran (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated MAPREDUCE-3502:
--------------------------------------
Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1)
Status: Patch Available (was: Open)
> Review all Service.stop() operations and make sure that they work before a service is started
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: MAPREDUCE-3502.patch, MAPREDUCE-3502.patch
>
> Original Estimate: 24h
> Time Spent: 2.5h
> Remaining Estimate: 21.5h
>
> MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
> # not calling other things if the other things are null
> # not being re-entrant (i.e. make synchronized if possible),
> Maybe
> # have a StopService operation that only stops a service if it is live
> # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
> my first iteration will skip the sync though it's something to consider.
> Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Posted by "Steve Loughran (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated MAPREDUCE-3502:
--------------------------------------
Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1)
Status: Open (was: Patch Available)
> Review all Service.stop() operations and make sure that they work before a service is started
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: MAPREDUCE-3502.patch, MAPREDUCE-3502.patch
>
> Original Estimate: 24h
> Time Spent: 2.5h
> Remaining Estimate: 21.5h
>
> MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
> # not calling other things if the other things are null
> # not being re-entrant (i.e. make synchronized if possible),
> Maybe
> # have a StopService operation that only stops a service if it is live
> # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
> my first iteration will skip the sync though it's something to consider.
> Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167767#comment-13167767 ]
Hadoop QA commented on MAPREDUCE-3502:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12507037/MAPREDUCE-3502.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1422//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1422//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1422//console
This message is automatically generated.
> Review all Service.stop() operations and make sure that they work before a service is started
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: MAPREDUCE-3502.patch, MAPREDUCE-3502.patch
>
> Original Estimate: 24h
> Time Spent: 2.5h
> Remaining Estimate: 21.5h
>
> MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
> # not calling other things if the other things are null
> # not being re-entrant (i.e. make synchronized if possible),
> Maybe
> # have a StopService operation that only stops a service if it is live
> # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
> my first iteration will skip the sync though it's something to consider.
> Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3502) Review all Service.stop()
operations and make sure that they work before a service is started
Posted by "Steve Loughran (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated MAPREDUCE-3502:
--------------------------------------
Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1)
Status: Open (was: Patch Available)
> Review all Service.stop() operations and make sure that they work before a service is started
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: MAPREDUCE-3502.patch
>
> Original Estimate: 24h
> Time Spent: 0.5h
> Remaining Estimate: 23.5h
>
> MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by
> # not calling other things if the other things are null
> # not being re-entrant (i.e. make synchronized if possible),
> Maybe
> # have a StopService operation that only stops a service if it is live
> # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing
> my first iteration will skip the sync though it's something to consider.
> Testing: try to create each instance; call stop() straight after construction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira