You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Chetan Kothari <ch...@oracle.com> on 2017/01/12 15:53:37 UTC
Drill with object storage using Stocator
I am trying to integrate Drill with Oracle Storage Cloud Service using Stocator.
But I get following error when I try to use stocator schema. Any input on this will be helpful.
Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment initialization: String index out of range: -37
... 4 common frames omitted
Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: -37
at java.lang.String.substring(Unknown Source) ~[na:1.8.0_101]
at com.ibm.stocator.fs.swift.SwiftAPIClient.list(SwiftAPIClient.java:393) ~[stocator-1.0.4.jar:na]
at com.ibm.stocator.fs.ObjectStoreFileSystem.listStatus(ObjectStoreFileSystem.java:267) ~[stocator-1.0.4.jar:na]
at org.apache.drill.exec.store.dfs.DrillFileSystem.listStatus(DrillFileSystem.java:523) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.accessible(WorkspaceSchemaFactory.java:159) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.store.dfs.FileSystemSchemaFactory$FileSystemSchema.<init>(FileSystemSchemaFactory.java:78) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.store.dfs.FileSystemSchemaFactory.registerSchemas(FileSystemSchemaFactory.java:65) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.store.dfs.FileSystemPlugin.registerSchemas(FileSystemPlugin.java:150) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:365) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:72) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:61) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.ops.QueryContext.getRootSchema(QueryContext.java:155) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.ops.QueryContext.getRootSchema(QueryContext.java:145) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.ops.QueryContext.getNewDefaultSchema(QueryContext.java:131) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:62) ~[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1008) [drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264) [drill-java-exec-1.9.0.jar:1.9.0]
... 3 common frames omitted
I use following configuration in drill
{
"type": "file",
"enabled": true,
"connection": "swift2d://omniturefilecontainer.stocator ",
"config": {
"fs.swift2d.impl": "com.ibm.stocator.fs.ObjectStoreFileSystem",
"fs.swift2d.service.stocator.username": "HYPERLINK "mailto:chetan.kothari@oracle.com"chetan.kothari@oracle.com",
"fs.swift2d.service.stocator.password": "welcome1",
"fs.swift2d.service.stocator.auth.url": "https://storage.oraclecloud.com/auth/v2.0/tokens",
"fs.swif2d.service.stocator.public": "true",
"fs.swif2d.service.stocator.region": "em2",
"fs.swif2d.service.stocator.CONTAINER-NAME": "omniturefilecontainer",
"fs.swift2d.service.stocator.tenant": "Storage-orcldfmlcsr1",
"fs.swift2d.service.stocator.auth.method": "keystone"
},
"workspaces": {
"root": {
"location": "swift2d://omniturefilecontainer.stocator",
"writable": false,
"defaultInputFormat": null
},
"tmp": {
"location": "/tmp",
"writable": true,
"defaultInputFormat": null
}
},
"formats": {
"tsv": {
"type": "text",
"extensions": [
"tsv"
],
"delimiter": "\t"
},
"csv": {
"type": "text",
"extensions": [
"csv"
],
"delimiter": ","
},
"parquet": {
"type": "parquet"
},
"json": {
"type": "json",
"extensions": [
"json"
]
},
"avro": {
"type": "avro"
},
"sequencefile": {
"type": "sequencefile",
"extensions": [
"seq"
]
},
"csvh": {
"type": "text",
"extensions": [
"csvh"
],
"extractHeader": true,
"delimiter": ","
}
}
}