You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by "Jakob Homan (JIRA)" <ji...@apache.org> on 2012/07/16 20:46:35 UTC
[jira] [Created] (GIRAPH-258) Check type compatibility before
submitting job
Jakob Homan created GIRAPH-258:
----------------------------------
Summary: Check type compatibility before submitting job
Key: GIRAPH-258
URL: https://issues.apache.org/jira/browse/GIRAPH-258
Project: Giraph
Issue Type: Bug
Reporter: Jakob Homan
Currently if one submits a job with a vertex and inputformat types that are not compatible the job only fails during runtime. Since we know the types at submission time, we can check this before submitting and fail then.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (GIRAPH-258) Check type compatibility before
submitting job
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan reassigned GIRAPH-258:
----------------------------------
Assignee: Eli Reisman
> Check type compatibility before submitting job
> ----------------------------------------------
>
> Key: GIRAPH-258
> URL: https://issues.apache.org/jira/browse/GIRAPH-258
> Project: Giraph
> Issue Type: Bug
> Reporter: Jakob Homan
> Assignee: Eli Reisman
>
> Currently if one submits a job with a vertex and inputformat types that are not compatible the job only fails during runtime. Since we know the types at submission time, we can check this before submitting and fail then.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-258) Check type compatibility before
submitting job
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Reisman updated GIRAPH-258:
-------------------------------
Attachment: GIRAPH-258-3.patch
cleaned up tests a bit more, addded @Test(expected = ...) annotations to tests that should fail etc.
> Check type compatibility before submitting job
> ----------------------------------------------
>
> Key: GIRAPH-258
> URL: https://issues.apache.org/jira/browse/GIRAPH-258
> Project: Giraph
> Issue Type: Bug
> Affects Versions: 0.2.0
> Reporter: Jakob Homan
> Assignee: Eli Reisman
> Fix For: 0.2.0
>
> Attachments: GIRAPH-258-1.patch, GIRAPH-258-2.patch, GIRAPH-258-3.patch
>
>
> Currently if one submits a job with a vertex and inputformat types that are not compatible the job only fails during runtime. Since we know the types at submission time, we can check this before submitting and fail then.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-258) Check type compatibility before
submitting job
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Reisman updated GIRAPH-258:
-------------------------------
Attachment: GIRAPH-258-6.patch
oops. ready now.
> Check type compatibility before submitting job
> ----------------------------------------------
>
> Key: GIRAPH-258
> URL: https://issues.apache.org/jira/browse/GIRAPH-258
> Project: Giraph
> Issue Type: Bug
> Affects Versions: 0.2.0
> Reporter: Jakob Homan
> Assignee: Eli Reisman
> Fix For: 0.2.0
>
> Attachments: GIRAPH-258-1.patch, GIRAPH-258-2.patch, GIRAPH-258-3.patch, GIRAPH-258-4.patch, GIRAPH-258-5.patch, GIRAPH-258-6.patch
>
>
> Currently if one submits a job with a vertex and inputformat types that are not compatible the job only fails during runtime. Since we know the types at submission time, we can check this before submitting and fail then.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-258) Check type compatibility before
submitting job
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Reisman updated GIRAPH-258:
-------------------------------
Attachment: GIRAPH-258-1.patch
passes mvn verify, type tests, etc. seems to work and allows job to fail early in GiraphRunner if command-line specified types are not compatible.
> Check type compatibility before submitting job
> ----------------------------------------------
>
> Key: GIRAPH-258
> URL: https://issues.apache.org/jira/browse/GIRAPH-258
> Project: Giraph
> Issue Type: Bug
> Reporter: Jakob Homan
> Assignee: Eli Reisman
> Attachments: GIRAPH-258-1.patch
>
>
> Currently if one submits a job with a vertex and inputformat types that are not compatible the job only fails during runtime. Since we know the types at submission time, we can check this before submitting and fail then.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-258) Check type compatibility before
submitting job
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Reisman updated GIRAPH-258:
-------------------------------
Attachment: GIRAPH-258-2.patch
fixed stuff mentioned in last comment, passes mvn verify etc.
> Check type compatibility before submitting job
> ----------------------------------------------
>
> Key: GIRAPH-258
> URL: https://issues.apache.org/jira/browse/GIRAPH-258
> Project: Giraph
> Issue Type: Bug
> Affects Versions: 0.2.0
> Reporter: Jakob Homan
> Assignee: Eli Reisman
> Fix For: 0.2.0
>
> Attachments: GIRAPH-258-1.patch, GIRAPH-258-2.patch
>
>
> Currently if one submits a job with a vertex and inputformat types that are not compatible the job only fails during runtime. Since we know the types at submission time, we can check this before submitting and fail then.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-258) Check type compatibility before
submitting job
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13432217#comment-13432217 ]
Jakob Homan commented on GIRAPH-258:
------------------------------------
stale already...
> Check type compatibility before submitting job
> ----------------------------------------------
>
> Key: GIRAPH-258
> URL: https://issues.apache.org/jira/browse/GIRAPH-258
> Project: Giraph
> Issue Type: Bug
> Affects Versions: 0.2.0
> Reporter: Jakob Homan
> Assignee: Eli Reisman
> Fix For: 0.2.0
>
> Attachments: GIRAPH-258-1.patch, GIRAPH-258-2.patch, GIRAPH-258-3.patch, GIRAPH-258-4.patch, GIRAPH-258-5.patch, GIRAPH-258-6.patch
>
>
> Currently if one submits a job with a vertex and inputformat types that are not compatible the job only fails during runtime. Since we know the types at submission time, we can check this before submitting and fail then.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-258) Check type compatibility before
submitting job
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Reisman updated GIRAPH-258:
-------------------------------
Attachment: GIRAPH-258-4.patch
Noticed incorrect comment @throws declaration in code copied over from GiraphMapper, decided to fix that too while i was in there. everything else is the same, passes mvn verify etc.
> Check type compatibility before submitting job
> ----------------------------------------------
>
> Key: GIRAPH-258
> URL: https://issues.apache.org/jira/browse/GIRAPH-258
> Project: Giraph
> Issue Type: Bug
> Affects Versions: 0.2.0
> Reporter: Jakob Homan
> Assignee: Eli Reisman
> Fix For: 0.2.0
>
> Attachments: GIRAPH-258-1.patch, GIRAPH-258-2.patch, GIRAPH-258-3.patch, GIRAPH-258-4.patch
>
>
> Currently if one submits a job with a vertex and inputformat types that are not compatible the job only fails during runtime. Since we know the types at submission time, we can check this before submitting and fail then.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-258) Check type compatibility before
submitting job
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428510#comment-13428510 ]
Eli Reisman commented on GIRAPH-258:
------------------------------------
All of the code is copied over from the GiraphMapper impl where the generics were checked after the job was already running, test code didn't change, just added the validator to it. I will remove check style comments. Those were the pre-existing tests as well, and they work so I left them alone. I can change the unchecked exceptions to fail() calls in the tests (that never made any sense but I figured it was a separate issue) further (very reasonable) fixes to the existing code I stole/moved might deserve its own JIRA.
Fixing now...
> Check type compatibility before submitting job
> ----------------------------------------------
>
> Key: GIRAPH-258
> URL: https://issues.apache.org/jira/browse/GIRAPH-258
> Project: Giraph
> Issue Type: Bug
> Affects Versions: 0.2.0
> Reporter: Jakob Homan
> Assignee: Eli Reisman
> Fix For: 0.2.0
>
> Attachments: GIRAPH-258-1.patch
>
>
> Currently if one submits a job with a vertex and inputformat types that are not compatible the job only fails during runtime. Since we know the types at submission time, we can check this before submitting and fail then.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-258) Check type compatibility before
submitting job
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Reisman updated GIRAPH-258:
-------------------------------
Attachment: GIRAPH-258-5.patch
This passes mvn verify and real run tests. I also refactored the validator class to clean up the validation code and make it more clear. should be good to go.
> Check type compatibility before submitting job
> ----------------------------------------------
>
> Key: GIRAPH-258
> URL: https://issues.apache.org/jira/browse/GIRAPH-258
> Project: Giraph
> Issue Type: Bug
> Affects Versions: 0.2.0
> Reporter: Jakob Homan
> Assignee: Eli Reisman
> Fix For: 0.2.0
>
> Attachments: GIRAPH-258-1.patch, GIRAPH-258-2.patch, GIRAPH-258-3.patch, GIRAPH-258-4.patch, GIRAPH-258-5.patch
>
>
> Currently if one submits a job with a vertex and inputformat types that are not compatible the job only fails during runtime. Since we know the types at submission time, we can check this before submitting and fail then.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-258) Check type compatibility before
submitting job
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Reisman updated GIRAPH-258:
-------------------------------
Attachment: GIRAPH-258-7.patch
rebased, should be working now.
> Check type compatibility before submitting job
> ----------------------------------------------
>
> Key: GIRAPH-258
> URL: https://issues.apache.org/jira/browse/GIRAPH-258
> Project: Giraph
> Issue Type: Bug
> Affects Versions: 0.2.0
> Reporter: Jakob Homan
> Assignee: Eli Reisman
> Fix For: 0.2.0
>
> Attachments: GIRAPH-258-1.patch, GIRAPH-258-2.patch, GIRAPH-258-3.patch, GIRAPH-258-4.patch, GIRAPH-258-5.patch, GIRAPH-258-6.patch, GIRAPH-258-7.patch
>
>
> Currently if one submits a job with a vertex and inputformat types that are not compatible the job only fails during runtime. Since we know the types at submission time, we can check this before submitting and fail then.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-258) Check type compatibility before
submitting job
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428372#comment-13428372 ]
Jakob Homan commented on GIRAPH-258:
------------------------------------
Review:
GiraphRunner.java
+ validator.validateClassTypes(); // to please findbugs
How so? (this comment shows up a bunch. It's probably not helpful in the long term)
GiraphTypeValidator.java
* Constructor comment is a bit confusing.
* Please refactor the if(blah == null) { etc etc} into a utility class so as not to have the same basic code path repeated
* Constants rather than naked numbers.
* It may make sense to subclass IllegalStateException to better encapsulate what we're trying to here.
TestVertexTypes.java
* Is there any reason not call fail rather throwing RTE when a test fails? Alternatively, we can just use the expected annotation, particularly if we subclass ISE
Also, what testing has been done?
> Check type compatibility before submitting job
> ----------------------------------------------
>
> Key: GIRAPH-258
> URL: https://issues.apache.org/jira/browse/GIRAPH-258
> Project: Giraph
> Issue Type: Bug
> Affects Versions: 0.2.0
> Reporter: Jakob Homan
> Assignee: Eli Reisman
> Fix For: 0.2.0
>
> Attachments: GIRAPH-258-1.patch
>
>
> Currently if one submits a job with a vertex and inputformat types that are not compatible the job only fails during runtime. Since we know the types at submission time, we can check this before submitting and fail then.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-258) Check type compatibility before
submitting job
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan resolved GIRAPH-258.
--------------------------------
Resolution: Fixed
+1. Committed this during the JIRA outage. Resolving. Thanks, Eli.
> Check type compatibility before submitting job
> ----------------------------------------------
>
> Key: GIRAPH-258
> URL: https://issues.apache.org/jira/browse/GIRAPH-258
> Project: Giraph
> Issue Type: Bug
> Affects Versions: 0.2.0
> Reporter: Jakob Homan
> Assignee: Eli Reisman
> Fix For: 0.2.0
>
> Attachments: GIRAPH-258-1.patch, GIRAPH-258-2.patch, GIRAPH-258-3.patch, GIRAPH-258-4.patch, GIRAPH-258-5.patch, GIRAPH-258-6.patch, GIRAPH-258-7.patch
>
>
> Currently if one submits a job with a vertex and inputformat types that are not compatible the job only fails during runtime. Since we know the types at submission time, we can check this before submitting and fail then.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira