You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by "Eugene Koontz (JIRA)" <ji...@apache.org> on 2012/08/06 20:38:03 UTC
[jira] [Created] (GIRAPH-282) Make netty the default RPC transport
Eugene Koontz created GIRAPH-282:
------------------------------------
Summary: Make netty the default RPC transport
Key: GIRAPH-282
URL: https://issues.apache.org/jira/browse/GIRAPH-282
Project: Giraph
Issue Type: Improvement
Reporter: Eugene Koontz
Currently we use Hadoop RPC as our means of communication amongst BSP workers, but we would like to transition to Netty for its better performance. This would require making security work by emulating Hadoop RPC's security features. See GIRAPH-211.
The fix for this is present in Avery's GIRAPH-211, so we need to simply break it out into a separate patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-282) Make netty the default RPC
transport
Posted by "Avery Ching (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429620#comment-13429620 ]
Avery Ching commented on GIRAPH-282:
------------------------------------
Are there any examples of anyone here who is still running HadoopRPC instead of Netty solely due to security reasons?
Frankly, there are other security holes mentioned by Eugene in GIRAPH-211 (i.e. ZooKeeper). I expect that we'll work through those issues when someone has time to get to them. Does this really need to block us from moving forward in the meantime?
We are holding up great code improvements (i.e. GIRAPH-200) and piling up cruft (see GIRAPH-45 if blocks on netty vs HadoopRPC). Is there truly a great reason for this?
> Make netty the default RPC transport
> ------------------------------------
>
> Key: GIRAPH-282
> URL: https://issues.apache.org/jira/browse/GIRAPH-282
> Project: Giraph
> Issue Type: Improvement
> Reporter: Eugene Koontz
> Assignee: Eugene Koontz
> Attachments: GIRAPH-282.patch
>
>
> Currently we use Hadoop RPC as our means of communication amongst BSP workers, but we would like to transition to Netty for its better performance. This would require making security work by emulating Hadoop RPC's security features. See GIRAPH-211.
> The fix for this is present in Avery's GIRAPH-262 patch, so we need to simply break it out into a separate patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-282) Make netty the default RPC
transport
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429613#comment-13429613 ]
Jakob Homan commented on GIRAPH-282:
------------------------------------
I'm not comfortable with this at all. At the very least, there needs to be additional documentation and a runtime check that prints out to the user ("your cluster is secure and we're using an insecure RPC").
> Make netty the default RPC transport
> ------------------------------------
>
> Key: GIRAPH-282
> URL: https://issues.apache.org/jira/browse/GIRAPH-282
> Project: Giraph
> Issue Type: Improvement
> Reporter: Eugene Koontz
> Assignee: Eugene Koontz
> Attachments: GIRAPH-282.patch
>
>
> Currently we use Hadoop RPC as our means of communication amongst BSP workers, but we would like to transition to Netty for its better performance. This would require making security work by emulating Hadoop RPC's security features. See GIRAPH-211.
> The fix for this is present in Avery's GIRAPH-262 patch, so we need to simply break it out into a separate patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-282) Make netty the default RPC
transport
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429630#comment-13429630 ]
Jakob Homan commented on GIRAPH-282:
------------------------------------
Security holes should be plugged, not ignored. And introducing new ones just adds additional debt. I'm not -1'ing going to netty as the default; just requiring there be documentation and in-your-face notification of the situation. If we say we support secure Hadoop, we need to be upfront as to the limitations of that support.
> Make netty the default RPC transport
> ------------------------------------
>
> Key: GIRAPH-282
> URL: https://issues.apache.org/jira/browse/GIRAPH-282
> Project: Giraph
> Issue Type: Improvement
> Reporter: Eugene Koontz
> Assignee: Eugene Koontz
> Attachments: GIRAPH-282.patch
>
>
> Currently we use Hadoop RPC as our means of communication amongst BSP workers, but we would like to transition to Netty for its better performance. This would require making security work by emulating Hadoop RPC's security features. See GIRAPH-211.
> The fix for this is present in Avery's GIRAPH-262 patch, so we need to simply break it out into a separate patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-282) Make netty the default RPC
transport
Posted by "Avery Ching (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429608#comment-13429608 ]
Avery Ching commented on GIRAPH-282:
------------------------------------
+1, although I can understand that those running secure Hadoop might want this as a default, they still have the choice and this can be set by default in the configuration if it's important. Let's get this in as most of our users immediately benefit by much better performance.
> Make netty the default RPC transport
> ------------------------------------
>
> Key: GIRAPH-282
> URL: https://issues.apache.org/jira/browse/GIRAPH-282
> Project: Giraph
> Issue Type: Improvement
> Reporter: Eugene Koontz
> Assignee: Eugene Koontz
> Attachments: GIRAPH-282.patch
>
>
> Currently we use Hadoop RPC as our means of communication amongst BSP workers, but we would like to transition to Netty for its better performance. This would require making security work by emulating Hadoop RPC's security features. See GIRAPH-211.
> The fix for this is present in Avery's GIRAPH-262 patch, so we need to simply break it out into a separate patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-282) Make netty the default RPC transport
Posted by "Eugene Koontz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eugene Koontz updated GIRAPH-282:
---------------------------------
Attachment: GIRAPH-282.patch
> Make netty the default RPC transport
> ------------------------------------
>
> Key: GIRAPH-282
> URL: https://issues.apache.org/jira/browse/GIRAPH-282
> Project: Giraph
> Issue Type: Improvement
> Reporter: Eugene Koontz
> Assignee: Eugene Koontz
> Attachments: GIRAPH-282.patch
>
>
> Currently we use Hadoop RPC as our means of communication amongst BSP workers, but we would like to transition to Netty for its better performance. This would require making security work by emulating Hadoop RPC's security features. See GIRAPH-211.
> The fix for this is present in Avery's GIRAPH-262 patch, so we need to simply break it out into a separate patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-282) Make netty the default RPC transport
Posted by "Eugene Koontz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eugene Koontz updated GIRAPH-282:
---------------------------------
Description:
Currently we use Hadoop RPC as our means of communication amongst BSP workers, but we would like to transition to Netty for its better performance. This would require making security work by emulating Hadoop RPC's security features. See GIRAPH-211.
The fix for this is present in Avery's GIRAPH-262 patch, so we need to simply break it out into a separate patch.
was:
Currently we use Hadoop RPC as our means of communication amongst BSP workers, but we would like to transition to Netty for its better performance. This would require making security work by emulating Hadoop RPC's security features. See GIRAPH-211.
The fix for this is present in Avery's GIRAPH-211, so we need to simply break it out into a separate patch.
> Make netty the default RPC transport
> ------------------------------------
>
> Key: GIRAPH-282
> URL: https://issues.apache.org/jira/browse/GIRAPH-282
> Project: Giraph
> Issue Type: Improvement
> Reporter: Eugene Koontz
>
> Currently we use Hadoop RPC as our means of communication amongst BSP workers, but we would like to transition to Netty for its better performance. This would require making security work by emulating Hadoop RPC's security features. See GIRAPH-211.
> The fix for this is present in Avery's GIRAPH-262 patch, so we need to simply break it out into a separate patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira