You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Jitendra Nath Pandey (Created) (JIRA)" <ji...@apache.org> on 2012/03/26 18:42:28 UTC
[jira] [Created] (MAPREDUCE-4067) Replace YarnRemoteException with
IOException in MRv2 APIs
Replace YarnRemoteException with IOException in MRv2 APIs
---------------------------------------------------------
Key: MAPREDUCE-4067
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4067
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Jitendra Nath Pandey
Assignee: Siddharth Seth
YarnRemoteException is defined as a generic wrapper for all the exceptions in yarn. I think we should instead throw IOExceptions in the API, which can later be extended for more specialized exceptions without breaking compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4067) Replace YarnRemoteException with
IOException in MRv2 APIs
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-4067:
--------------------------------------
Target Version/s: (was: )
Status: Open (was: Patch Available)
Cancelling, the patch will need a rebase.
This is a blocker for a beta release - since it changes APIs. For now leaving it as is.
Craeted YARN-142 to take care of the YARN changes.
> Replace YarnRemoteException with IOException in MRv2 APIs
> ---------------------------------------------------------
>
> Key: MAPREDUCE-4067
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4067
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Jitendra Nath Pandey
> Assignee: Siddharth Seth
> Priority: Critical
> Attachments: MR4067.txt
>
>
> YarnRemoteException is defined as a generic wrapper for all the exceptions in yarn. I think we should instead throw IOExceptions in the API, which can later be extended for more specialized exceptions without breaking compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4067) Replace YarnRemoteException with
IOException in MRv2 APIs
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated MAPREDUCE-4067:
-----------------------------------
Target Version/s: 2.0.0 (was: trunk, 2.0.0)
> Replace YarnRemoteException with IOException in MRv2 APIs
> ---------------------------------------------------------
>
> Key: MAPREDUCE-4067
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4067
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Jitendra Nath Pandey
> Assignee: Siddharth Seth
> Priority: Blocker
> Attachments: MR4067.txt
>
>
> YarnRemoteException is defined as a generic wrapper for all the exceptions in yarn. I think we should instead throw IOExceptions in the API, which can later be extended for more specialized exceptions without breaking compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4067) Replace YarnRemoteException with
IOException in MRv2 APIs
Posted by "Jitendra Nath Pandey (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jitendra Nath Pandey updated MAPREDUCE-4067:
--------------------------------------------
Priority: Blocker (was: Major)
Target Version/s: 0.24.0, 0.23.3
> Replace YarnRemoteException with IOException in MRv2 APIs
> ---------------------------------------------------------
>
> Key: MAPREDUCE-4067
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4067
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Jitendra Nath Pandey
> Assignee: Siddharth Seth
> Priority: Blocker
>
> YarnRemoteException is defined as a generic wrapper for all the exceptions in yarn. I think we should instead throw IOExceptions in the API, which can later be extended for more specialized exceptions without breaking compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4067) Replace YarnRemoteException
with IOException in MRv2 APIs
Posted by "Siddharth Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242773#comment-13242773 ]
Siddharth Seth commented on MAPREDUCE-4067:
-------------------------------------------
YarnRemoteException was serializing the actual exception and sending it over the wire. It was missing an unwind method to get the actual exception. IAC - clients should not be expected to unwind and that should've been handled in the RPC layer itself.
Instead of having the APIs throw an IOException, I'd prefer to define a Yarn / HadoopException which extends Exception and can later be sub-classed for more specific exceptions. Also, possibly add error codes to these exceptions as well as to what is sent over the wire. That may make things easier for clients written in other languages.
> Replace YarnRemoteException with IOException in MRv2 APIs
> ---------------------------------------------------------
>
> Key: MAPREDUCE-4067
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4067
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Jitendra Nath Pandey
> Assignee: Siddharth Seth
> Priority: Blocker
>
> YarnRemoteException is defined as a generic wrapper for all the exceptions in yarn. I think we should instead throw IOExceptions in the API, which can later be extended for more specialized exceptions without breaking compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4067) Replace YarnRemoteException with
IOException in MRv2 APIs
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-4067:
--------------------------------------
Target Version/s: 2.0.0, trunk (was: 0.23.3, 0.24.0)
Status: Patch Available (was: Open)
Fixing the target versions as well - this isn't meant for 0.23.3
> Replace YarnRemoteException with IOException in MRv2 APIs
> ---------------------------------------------------------
>
> Key: MAPREDUCE-4067
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4067
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Jitendra Nath Pandey
> Assignee: Siddharth Seth
> Priority: Blocker
> Attachments: MR4067.txt
>
>
> YarnRemoteException is defined as a generic wrapper for all the exceptions in yarn. I think we should instead throw IOExceptions in the API, which can later be extended for more specialized exceptions without breaking compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4067) Replace YarnRemoteException with
IOException in MRv2 APIs
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-4067:
--------------------------------------
Attachment: MR4067.txt
Turns out the RPC/Security layers don't handle anything other than IOException very well (ugi.doAs will result in UndeclaredThrowableException if not an IOException).
Changes in the patch
- All APIs also throw IOException (for exception originating in the RPC layer)
- YarnRemoteException (continues to extend IOE) can be extended for custom exceptions.
- The old YarnRemoteExceptionPBImpl moved as a record to SerializedException (used by some yarn APIs)
- Added some custom exceptions like UnknownAppId
If folks are ok with this patch, would like to rename YarnRemoteException to YarnException, and the existing YarnException to YarnRuntimeException before this is checked in.
> Replace YarnRemoteException with IOException in MRv2 APIs
> ---------------------------------------------------------
>
> Key: MAPREDUCE-4067
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4067
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Jitendra Nath Pandey
> Assignee: Siddharth Seth
> Priority: Blocker
> Attachments: MR4067.txt
>
>
> YarnRemoteException is defined as a generic wrapper for all the exceptions in yarn. I think we should instead throw IOExceptions in the API, which can later be extended for more specialized exceptions without breaking compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4067) Replace YarnRemoteException with
IOException in MRv2 APIs
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-4067:
-------------------------------
Priority: Critical (was: Blocker)
Target Version/s: 2.0.1-alpha (was: 2.0.0-alpha)
(Don't think this is a blocker, but reset if I'm wrong. An explanation of why its a blocker would be good too, thanks! :))
> Replace YarnRemoteException with IOException in MRv2 APIs
> ---------------------------------------------------------
>
> Key: MAPREDUCE-4067
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4067
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Jitendra Nath Pandey
> Assignee: Siddharth Seth
> Priority: Critical
> Attachments: MR4067.txt
>
>
> YarnRemoteException is defined as a generic wrapper for all the exceptions in yarn. I think we should instead throw IOExceptions in the API, which can later be extended for more specialized exceptions without breaking compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4067) Replace YarnRemoteException
with IOException in MRv2 APIs
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269901#comment-13269901 ]
Hadoop QA commented on MAPREDUCE-4067:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12525902/MR4067.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 19 new or modified test files.
-1 javadoc. The javadoc tool appears to have generated 2 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
-1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2363//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2363//console
This message is automatically generated.
> Replace YarnRemoteException with IOException in MRv2 APIs
> ---------------------------------------------------------
>
> Key: MAPREDUCE-4067
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4067
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Jitendra Nath Pandey
> Assignee: Siddharth Seth
> Priority: Blocker
> Attachments: MR4067.txt
>
>
> YarnRemoteException is defined as a generic wrapper for all the exceptions in yarn. I think we should instead throw IOExceptions in the API, which can later be extended for more specialized exceptions without breaking compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira