JSON storage configuration error(invalid json mapping)

[SiS <ne...@cht3.com>]

Hi,

I try to change the storage configuration in apache drill in embedded mode to identify headers and to change the delimiter of csv files. I also renamed the new format category from csv to sap. 

I tried to use the information from the documentation and created the following json storage information:

{
 "type": "file",
 "enabled": true,
 "connection": "file:///",
 "workspaces": {
  "root": {
    "location": "/",
    "writable": false,
    "defaultInputFormat": null
  },
  "tmp": {
  "location": "/tmp",
  "writable": true,
  "defaultInputFormat": null
  }
},
  "formats": {
    "sap": {
     "type": "text",
     "extensions": [
       "sap"
     ],
     "skipFirstLine": false,
     "extractHeader": true,
     "delimiter": "|"
   },
   "psv": {
    "type": "text",
    "extensions": [
       "tbl"
    ],
    "delimiter": "|"
   },
   "csv": {
     "type": "text",
     "extensions": [
       "csv"
     ],
   "delimiter": ","
   },
   "tsv": {
     "type": "text",
     "extensions": [
       "tsv"
      ],
     "delimiter": "\t"
   },
   "parquet": {
      "type": "parquet"
   },
   "json": {
     "type": "json"
   },
   "avro": {
      "type": "avro"
   }
}
}
But always when I try to save it in the web-ui I got the message: error (invalid json mapping).

The exec.storage.enable_new_text_reader is set true. 

Could somebody help my how I can add the two config items: skipFirstLine and extractHeader?

BR

Re: JSON storage configuration error(invalid json mapping)

[Kristine Hahn <kh...@maprtech.com>]

I can save the storage plugin config from your email to the web-ui in Drill
1.3. http://jsonlint.com/ confirms that your config is valid json. Please
try again.

Could somebody help my how I can add the two config items: skipFirstLine
> and extractHeader?


As you've discovered, using the default settings, when save the config,
Drill removes skipFirstLine=false when extractHeader=true. I think the docs
need to say "Ensure skipFirstLine does not = true when extractHeader=true"
instead of "Ensure skipFirstLine=false when extractHeader=true".


Kristine Hahn
Sr. Technical Writer
415-497-8107 @krishahn skype:krishahn


On Wed, Dec 9, 2015 at 5:50 AM, SiS <ne...@cht3.com> wrote:

> Hi,
>
> I try to change the storage configuration in apache drill in embedded mode
> to identify headers and to change the delimiter of csv files. I also
> renamed the new format category from csv to sap.
>
> I tried to use the information from the documentation and created the
> following json storage information:
>
> {
>  "type": "file",
>  "enabled": true,
>  "connection": "file:///",
>  "workspaces": {
>   "root": {
>     "location": "/",
>     "writable": false,
>     "defaultInputFormat": null
>   },
>   "tmp": {
>   "location": "/tmp",
>   "writable": true,
>   "defaultInputFormat": null
>   }
> },
>   "formats": {
>     "sap": {
>      "type": "text",
>      "extensions": [
>        "sap"
>      ],
>      "skipFirstLine": false,
>      "extractHeader": true,
>      "delimiter": "|"
>    },
>    "psv": {
>     "type": "text",
>     "extensions": [
>        "tbl"
>     ],
>     "delimiter": "|"
>    },
>    "csv": {
>      "type": "text",
>      "extensions": [
>        "csv"
>      ],
>    "delimiter": ","
>    },
>    "tsv": {
>      "type": "text",
>      "extensions": [
>        "tsv"
>       ],
>      "delimiter": "\t"
>    },
>    "parquet": {
>       "type": "parquet"
>    },
>    "json": {
>      "type": "json"
>    },
>    "avro": {
>       "type": "avro"
>    }
> }
> }
> But always when I try to save it in the web-ui I got the message: error
> (invalid json mapping).
>
> The exec.storage.enable_new_text_reader is set true.
>
> Could somebody help my how I can add the two config items: skipFirstLine
> and extractHeader?
>
> BR