You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Boris Shkolnik (JIRA)" <ji...@apache.org> on 2009/10/20 21:56:59 UTC
[jira] Created: (HADOOP-6325) need security keys storage solution
need security keys storage solution
-----------------------------------
Key: HADOOP-6325
URL: https://issues.apache.org/jira/browse/HADOOP-6325
Project: Hadoop Common
Issue Type: New Feature
Reporter: Boris Shkolnik
set, get, store, load security keys
key alias - byte[]
key value - byte[]
store/load from DataInput/Output stream
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-6325) need security keys storage solution
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik resolved HADOOP-6325.
------------------------------------
Resolution: Won't Fix
Closing it. We used JobTokens class to store shuffle key.
> need security keys storage solution
> -----------------------------------
>
> Key: HADOOP-6325
> URL: https://issues.apache.org/jira/browse/HADOOP-6325
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: HADOOP-6325.patch
>
>
> set, get, store, load security keys
> key alias - byte[]
> key value - byte[]
> store/load from DataInput/Output stream
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6325) need security keys storage solution
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768168#action_12768168 ]
Tom White commented on HADOOP-6325:
-----------------------------------
Could this use java.security.KeyStore?
> need security keys storage solution
> -----------------------------------
>
> Key: HADOOP-6325
> URL: https://issues.apache.org/jira/browse/HADOOP-6325
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Boris Shkolnik
> Attachments: HADOOP-6325.patch
>
>
> set, get, store, load security keys
> key alias - byte[]
> key value - byte[]
> store/load from DataInput/Output stream
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-6325) need security keys storage solution
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik reassigned HADOOP-6325:
--------------------------------------
Assignee: Boris Shkolnik
> need security keys storage solution
> -----------------------------------
>
> Key: HADOOP-6325
> URL: https://issues.apache.org/jira/browse/HADOOP-6325
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: HADOOP-6325.patch
>
>
> set, get, store, load security keys
> key alias - byte[]
> key value - byte[]
> store/load from DataInput/Output stream
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HADOOP-6325) need security keys storage solution
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das reopened HADOOP-6325:
---------------------------------
I think we should leave this issue open. We will need a solution to support the use case of arbitrary user supplied security credentials...
> need security keys storage solution
> -----------------------------------
>
> Key: HADOOP-6325
> URL: https://issues.apache.org/jira/browse/HADOOP-6325
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: HADOOP-6325.patch
>
>
> set, get, store, load security keys
> key alias - byte[]
> key value - byte[]
> store/load from DataInput/Output stream
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6325) need security keys storage solution
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768338#action_12768338 ]
Allen Wittenauer commented on HADOOP-6325:
------------------------------------------
Are these stores expected to pre-encrypted or do they do the encryption themselves? I sort of echo what Tom says: are we building something custom that we shouldn't be?
> need security keys storage solution
> -----------------------------------
>
> Key: HADOOP-6325
> URL: https://issues.apache.org/jira/browse/HADOOP-6325
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: HADOOP-6325.patch
>
>
> set, get, store, load security keys
> key alias - byte[]
> key value - byte[]
> store/load from DataInput/Output stream
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6325) need security keys storage solution
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768876#action_12768876 ]
Owen O'Malley commented on HADOOP-6325:
---------------------------------------
I think this actually belongs over in map/reduce rather than in common.
The use case is: when the user submits a job, they need to include credentials. Currently this would go into the job conf, but that is visible from the web. We are better off factoring this out into a separate map.
Note that the credentials will be disjoint for different HDFS clusters (or other filesystems). So it will look like:
"hdfs://nn1/" -> binary blob1
"hdfs://nn2/" -> binary blob2
"mapred.job.token" -> binary blob3
This key storage should be included in the call to submitJob.
> need security keys storage solution
> -----------------------------------
>
> Key: HADOOP-6325
> URL: https://issues.apache.org/jira/browse/HADOOP-6325
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: HADOOP-6325.patch
>
>
> set, get, store, load security keys
> key alias - byte[]
> key value - byte[]
> store/load from DataInput/Output stream
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6325) need security keys storage solution
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768867#action_12768867 ]
Owen O'Malley commented on HADOOP-6325:
---------------------------------------
The intent is to have a key-value store for security credentials. It is the equivalent of a secure job configuration. It needs to be passable over RPC and therefore should be implemented as a writable.
I don't see a good way (or justification) to use the KeyStore.
They will only be stored in the JobTracker's system directory and so they don't need to be encrypted themselves.
> need security keys storage solution
> -----------------------------------
>
> Key: HADOOP-6325
> URL: https://issues.apache.org/jira/browse/HADOOP-6325
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: HADOOP-6325.patch
>
>
> set, get, store, load security keys
> key alias - byte[]
> key value - byte[]
> store/load from DataInput/Output stream
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6325) need security keys storage solution
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791110#action_12791110 ]
Boris Shkolnik commented on HADOOP-6325:
----------------------------------------
{quote}
Note that the credentials will be disjoint for different HDFS clusters (or other filesystems). So it will look like:
"hdfs://nn1/" -> binary blob1
"hdfs://nn2/" -> binary blob2
"mapred.job.token" -> binary blob3
This key storage should be included in the call to submitJob.
{quote}
If the keys are given on a command line - how can we pass it to the job. All the command lines arguments are passed thru config, and we want to avoid id.
> need security keys storage solution
> -----------------------------------
>
> Key: HADOOP-6325
> URL: https://issues.apache.org/jira/browse/HADOOP-6325
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Boris Shkolnik
> Assignee: Boris Shkolnik
> Attachments: HADOOP-6325.patch
>
>
> set, get, store, load security keys
> key alias - byte[]
> key value - byte[]
> store/load from DataInput/Output stream
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6325) need security keys storage solution
Posted by "Boris Shkolnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated HADOOP-6325:
-----------------------------------
Attachment: HADOOP-6325.patch
implementation + test
> need security keys storage solution
> -----------------------------------
>
> Key: HADOOP-6325
> URL: https://issues.apache.org/jira/browse/HADOOP-6325
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Boris Shkolnik
> Attachments: HADOOP-6325.patch
>
>
> set, get, store, load security keys
> key alias - byte[]
> key value - byte[]
> store/load from DataInput/Output stream
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.