You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2013/07/02 14:36:25 UTC
[jira] [Commented] (HADOOP-8545) Filesystem Implementation for
OpenStack Swift
[ https://issues.apache.org/jira/browse/HADOOP-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13697728#comment-13697728 ]
Steve Loughran commented on HADOOP-8545:
----------------------------------------
The auth package classes are all designed for Jackson binding and MUST NOT be final, else you see a stack trace. Test to replicate this is {{TestSwiftRestClient}}
{code}
at org.apache.hadoop.fs.swift.util.JSONUtil.toObject(JSONUtil.java:81)
at org.apache.hadoop.fs.swift.http.SwiftRestClient$AuthenticationPost.extractResult(SwiftRestClient.java:1127)
at org.apache.hadoop.fs.swift.http.SwiftRestClient$AuthenticationPost.extractResult(SwiftRestClient.java:1085)
at org.apache.hadoop.fs.swift.http.SwiftRestClient.perform(SwiftRestClient.java:1407)
at org.apache.hadoop.fs.swift.http.SwiftRestClient.authenticate(SwiftRestClient.java:1080)
at org.apache.hadoop.fs.swift.http.TestSwiftRestClient.testPutAndDelete(TestSwiftRestClient.java:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
Caused by: org.codehaus.jackson.map.JsonMappingException: No suitable constructor found for type [simple type, class org.apache.hadoop.fs.swift.auth.AuthenticationWrapper]: can not instantiate from JSON object (need to add/enable type information?)
at [Source: java.io.StringReader@7d0c3a08; line: 1, column: 2]
at org.codehaus.jackson.map.JsonMappingException.from(JsonMappingException.java:163)
at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:483)
at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350)
at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2402)
at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1602)
at org.apache.hadoop.fs.swift.util.JSONUtil.toObject(JSONUtil.java:75)
... 14 more
{code}
Same for {{SwiftObjectFileStatus}}
{code}
Running org.apache.hadoop.fs.swift.TestSwiftFileSystemBasicOps
Tests run: 16, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 130.317 sec <<< FAILURE!
testLsRoot(org.apache.hadoop.fs.swift.TestSwiftFileSystemBasicOps) Time elapsed: 3267 sec <<< ERROR!
org.apache.hadoop.fs.swift.exceptions.SwiftJsonMarshallingException: org.codehaus.jackson.map.JsonMappingException: No suitable constructor found for type [simple type, class org.apache.hadoop.fs.swift.snative.SwiftObjectFileStatus]: can not instantiate from JSON object (need to add/enable type information?)
at [Source: java.io.StringReader@6179d854; line: 1, column: 3] source: [{"hash": "d41d8cd98f00b204e9800998ecf8427e", "last_modified": "2013-05-13T21:05:26.066260", "bytes": 0, "name": "user", "content_type": "application/octet-stream"}, {"subdir": "user/"}]
at org.apache.hadoop.fs.swift.util.JSONUtil.toObject(JSONUtil.java:121)
at org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.listDirectory(SwiftNativeFileSystemStore.java:354)
at org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.listSubPaths(SwiftNativeFileSystemStore.java:399)
at org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem.listStatus(SwiftNativeFileSystem.java:436)
at org.apache.hadoop.fs.swift.TestSwiftFileSystemBasicOps.testLsRoot(TestSwiftFileSystemBasicOps.java:51)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
Caused by: org.codehaus.jackson.map.JsonMappingException: No suitable constructor found for type [simple type, class org.apache.hadoop.fs.swift.snative.SwiftObjectFileStatus]: can not instantiate from JSON object (need to add/enable type information?)
at [Source: java.io.StringReader@6179d854; line: 1, column: 3]
at org.codehaus.jackson.map.JsonMappingException.from(JsonMappingException.java:163)
at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:483)
at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350)
at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:120)
at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:97)
at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:26)
at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2402)
at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1616)
at org.apache.hadoop.fs.swift.util.JSONUtil.toObject(JSONUtil.java:115)
... 13 more
{code}
The variable names are fixed to match those in the Keystone and Swift protocols, hence the use of underscores.
I've reverted those files and pulled in taken the other changes: the switch to {{isDirectory()}}, the javadocs on the exceptions and the various minor formatting patches; added some more javadocs on the tests. Tested against public OpenStack implementations
> Filesystem Implementation for OpenStack Swift
> ---------------------------------------------
>
> Key: HADOOP-8545
> URL: https://issues.apache.org/jira/browse/HADOOP-8545
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Affects Versions: 1.2.0, 2.0.3-alpha
> Reporter: Tim Miller
> Assignee: Dmitry Mezhensky
> Labels: hadoop, patch
> Attachments: HADOOP-8545-026.patch, HADOOP-8545-027.patch, HADOOP-8545-028.patch, HADOOP-8545-029.patch, HADOOP-8545-030.patch, HADOOP-8545-10.patch, HADOOP-8545-11.patch, HADOOP-8545-12.patch, HADOOP-8545-13.patch, HADOOP-8545-14.patch, HADOOP-8545-15.patch, HADOOP-8545-16.patch, HADOOP-8545-17.patch, HADOOP-8545-18.patch, HADOOP-8545-19.patch, HADOOP-8545-1.patch, HADOOP-8545-20.patch, HADOOP-8545-21.patch, HADOOP-8545-22.patch, HADOOP-8545-23.patch, HADOOP-8545-24.patch, HADOOP-8545-25.patch, HADOOP-8545-2.patch, HADOOP-8545-3.patch, HADOOP-8545-4.patch, HADOOP-8545-5.patch, HADOOP-8545-6.patch, HADOOP-8545-7.patch, HADOOP-8545-8.patch, HADOOP-8545-9.patch, HADOOP-8545-javaclouds-2.patch, HADOOP-8545.patch, HADOOP-8545.patch, HADOOP-8545.suresh.patch
>
>
> ,Add a filesystem implementation for OpenStack Swift object store, similar to the one which exists today for S3.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira