You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Allen Wittenauer (JIRA)" <ji...@apache.org> on 2015/03/28 00:47:57 UTC
[jira] [Updated] (HADOOP-8545) Filesystem Implementation for
OpenStack Swift
[ https://issues.apache.org/jira/browse/HADOOP-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Allen Wittenauer updated HADOOP-8545:
-------------------------------------
Release Note:
Added file system implementation for OpenStack Swift.
There are two implementation: block and native (similar to Amazon S3 integration).
Data locality issue solved by patch in Swift, commit procedure to OpenStack is in progress.
To use implementation add to core-site.xml following:
```xml
<property>
<name>fs.swift.impl</name>
<value>com.mirantis.fs.SwiftFileSystem</value>
</property>
<property>
<name>fs.swift.block.impl</name>
<value>com.mirantis.fs.block.SwiftBlockFileSystem</value>
</property>
```
In MapReduce job specify following configs for OpenStack Keystone authentication:
```java
conf.set("swift.auth.url", "http://172.18.66.117:5000/v2.0/tokens");
conf.set("swift.tenant", "superuser");
conf.set("swift.username", "admin1");
conf.set("swift.password", "password");
conf.setInt("swift.http.port", 8080);
conf.setInt("swift.https.port", 443);
```java
Additional information specified on github: https://github.com/DmitryMezhensky/Hadoop-and-Swift-integration
was:
Added file system implementation for OpenStack Swift.
There are two implementation: block and native (similar to Amazon S3 integration).
Data locality issue solved by patch in Swift, commit procedure to OpenStack is in progress.
To use implementation add to core-site.xml following:
...
<property>
<name>fs.swift.impl</name>
<value>com.mirantis.fs.SwiftFileSystem</value>
</property>
<property>
<name>fs.swift.block.impl</name>
<value>com.mirantis.fs.block.SwiftBlockFileSystem</value>
</property>
...
In MapReduce job specify following configs for OpenStack Keystone authentication:
conf.set("swift.auth.url", "http://172.18.66.117:5000/v2.0/tokens");
conf.set("swift.tenant", "superuser");
conf.set("swift.username", "admin1");
conf.set("swift.password", "password");
conf.setInt("swift.http.port", 8080);
conf.setInt("swift.https.port", 443);
Additional information specified on github: https://github.com/DmitryMezhensky/Hadoop-and-Swift-integration
> Filesystem Implementation for OpenStack Swift
> ---------------------------------------------
>
> Key: HADOOP-8545
> URL: https://issues.apache.org/jira/browse/HADOOP-8545
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Affects Versions: 1.2.0, 2.0.3-alpha
> Reporter: Tim Miller
> Assignee: Dmitry Mezhensky
> Labels: hadoop, patch
> Fix For: 2.3.0
>
> Attachments: HADOOP-8545-026.patch, HADOOP-8545-027.patch, HADOOP-8545-028.patch, HADOOP-8545-029.patch, HADOOP-8545-030.patch, HADOOP-8545-031.patch, HADOOP-8545-032.patch, HADOOP-8545-033.patch, HADOOP-8545-034.patch, HADOOP-8545-035.patch, HADOOP-8545-035.patch, HADOOP-8545-036.patch, HADOOP-8545-037.patch, HADOOP-8545-1.patch, HADOOP-8545-10.patch, HADOOP-8545-11.patch, HADOOP-8545-12.patch, HADOOP-8545-13.patch, HADOOP-8545-14.patch, HADOOP-8545-15.patch, HADOOP-8545-16.patch, HADOOP-8545-17.patch, HADOOP-8545-18.patch, HADOOP-8545-19.patch, HADOOP-8545-2.patch, HADOOP-8545-20.patch, HADOOP-8545-21.patch, HADOOP-8545-22.patch, HADOOP-8545-23.patch, HADOOP-8545-24.patch, HADOOP-8545-25.patch, HADOOP-8545-3.patch, HADOOP-8545-4.patch, HADOOP-8545-5.patch, HADOOP-8545-6.patch, HADOOP-8545-7.patch, HADOOP-8545-8.patch, HADOOP-8545-9.patch, HADOOP-8545-javaclouds-2.patch, HADOOP-8545.patch, HADOOP-8545.patch, HADOOP-8545.suresh.patch
>
>
> ,Add a filesystem implementation for OpenStack Swift object store, similar to the one which exists today for S3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)