You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Jacques Nadeau (JIRA)" <ji...@apache.org> on 2015/01/04 22:27:36 UTC
[jira] [Updated] (DRILL-1290) Document Configuration Steps for
Different Hadoop Distributions
[ https://issues.apache.org/jira/browse/DRILL-1290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jacques Nadeau updated DRILL-1290:
----------------------------------
Fix Version/s: (was: 0.8.0)
Future
Summary: Document Configuration Steps for Different Hadoop Distributions (was: Creating storage plugin for "hdfs:///" failed with "Unable to create/ update plugin: myhdfs")
> Document Configuration Steps for Different Hadoop Distributions
> ---------------------------------------------------------------
>
> Key: DRILL-1290
> URL: https://issues.apache.org/jira/browse/DRILL-1290
> Project: Apache Drill
> Issue Type: New Feature
> Components: Metadata
> Environment: OS: Centos 6.4
> HDFS: CDH5.1
> Drill: 0.4.0
> Reporter: Hao Zhu
> Assignee: Neeraja
> Fix For: Future
>
>
> In web GUI, I can successfully create a new storage plugin named "myhdfs" using "file:///":
> {code}
> {
> "type": "file",
> "enabled": true,
> "connection": "file:///",
> "workspaces": {
> "root": {
> "location": "/",
> "writable": false,
> "storageformat": null
> },
> "tmp": {
> "location": "/tmp",
> "writable": true,
> "storageformat": "csv"
> }
> },
> "formats": {
> "psv": {
> "type": "text",
> "extensions": [
> "tbl"
> ],
> "delimiter": "|"
> },
> "csv": {
> "type": "text",
> "extensions": [
> "csv"
> ],
> "delimiter": ","
> },
> "tsv": {
> "type": "text",
> "extensions": [
> "tsv"
> ],
> "delimiter": "\t"
> },
> "parquet": {
> "type": "parquet"
> },
> "json": {
> "type": "json"
> }
> }
> }
> {code}
> However if I try to change "file:///" to "hdfs:///" to point to HDFS other than local file system, drill log errors out "[qtp416200645-67] DEBUG o.a.d.e.server.rest.StorageResources - Unable to create/ update plugin: myhdfs".
> {code}
> {
> "type": "file",
> "enabled": true,
> "connection": "hdfs:///",
> "workspaces": {
> "root": {
> "location": "/",
> "writable": false,
> "storageformat": null
> },
> "tmp": {
> "location": "/tmp",
> "writable": true,
> "storageformat": "csv"
> }
> },
> "formats": {
> "psv": {
> "type": "text",
> "extensions": [
> "tbl"
> ],
> "delimiter": "|"
> },
> "csv": {
> "type": "text",
> "extensions": [
> "csv"
> ],
> "delimiter": ","
> },
> "tsv": {
> "type": "text",
> "extensions": [
> "tsv"
> ],
> "delimiter": "\t"
> },
> "parquet": {
> "type": "parquet"
> },
> "json": {
> "type": "json"
> }
> }
> }
> {code}
> On my cluster, I am using CDH5 hdfs, and it all client configurations are valid. For example, on the drillbit server:
> {code}
> [root@hdm ~]# hdfs dfs -ls /
> Found 3 items
> drwxr-xr-x - hbase hbase 0 2014-08-04 22:55 /hbase
> drwxrwxrwt - hdfs supergroup 0 2014-07-31 16:31 /tmp
> drwxr-xr-x - hdfs supergroup 0 2014-07-11 12:06 /user
> {code}
> Is there anything wrong with the storage plugin syntax for HDFS?
> If so, can drill log prints more debug info to show the reason why it failed?
> Thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)