You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by abdelhakim deneche <ad...@gmail.com> on 2014/11/18 17:48:04 UTC
Review Request 28173: DRILL-1545 - Json files can only be read when
they have a .json extension
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28173/
-----------------------------------------------------------
Review request for drill.
Bugs: DRILL-1545
https://issues.apache.org/jira/browse/DRILL-1545
Repository: drill-git
Description
-------
updated JSONFormatPlugin, adding an extensions array into JSONFormatConfig. I used TextFormatPlugin as a reference.
Now the user can set the following format in Web UI:
"json": {
"type": "json",
"extensions": [
"json",
"log"
]
}
The user can also write a separate config for json and log like this:
"json": {
"type": "json",
"extensions": [
"json"
]
},
"log": {
"type": "json",
"extensions": [
"log"
]
}
Diffs
-----
exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONFormatPlugin.java e1165a2
exec/java-exec/src/main/resources/bootstrap-storage-plugins.json 4a20bea
Diff: https://reviews.apache.org/r/28173/diff/
Testing
-------
created two copies of the same .json data file, one with .json extension and another with .log extension.
updated "json" format like this:
"json": {
"type": "json",
"extensions": [
"json",
"log"
]
}
once the change was done, I was able to query both files from sqlline.
Thanks,
abdelhakim deneche
Re: Review Request 28173: DRILL-1545 - Json files can only be read
when they have a .json extension
Posted by Jason Altekruse <al...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28173/#review62940
-----------------------------------------------------------
Ship it!
Ship It!
- Jason Altekruse
On Nov. 21, 2014, 11:40 p.m., abdelhakim deneche wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28173/
> -----------------------------------------------------------
>
> (Updated Nov. 21, 2014, 11:40 p.m.)
>
>
> Review request for drill.
>
>
> Bugs: DRILL-1545
> https://issues.apache.org/jira/browse/DRILL-1545
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> updated JSONFormatPlugin, adding an extensions array into JSONFormatConfig. I used TextFormatPlugin as a reference.
> Now the user can set the following format in Web UI:
> "json": {
> "type": "json",
> "extensions": [
> "json",
> "log"
> ]
> }
>
> The user can also write a separate config for json and log like this:
> "json": {
> "type": "json",
> "extensions": [
> "json"
> ]
> },
> "log": {
> "type": "json",
> "extensions": [
> "log"
> ]
> }
>
>
> Diffs
> -----
>
> exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONFormatPlugin.java e1165a2
> exec/java-exec/src/main/resources/bootstrap-storage-plugins.json 4a20bea
>
> Diff: https://reviews.apache.org/r/28173/diff/
>
>
> Testing
> -------
>
> created two copies of the same .json data file, one with .json extension and another with .log extension.
>
> updated "json" format like this:
>
> "json": {
> "type": "json",
> "extensions": [
> "json",
> "log"
> ]
> }
>
> once the change was done, I was able to query both files from sqlline.
>
>
> Thanks,
>
> abdelhakim deneche
>
>
Re: Review Request 28173: DRILL-1545 - Json files can only be read
when they have a .json extension
Posted by abdelhakim deneche <ad...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28173/
-----------------------------------------------------------
(Updated Nov. 21, 2014, 11:40 p.m.)
Review request for drill.
Changes
-------
JSONFormatConfig has been updated to better handle existing json configurations without "extensions" attribute. When a user updates Drill, it doesn't need to update json format config
Bugs: DRILL-1545
https://issues.apache.org/jira/browse/DRILL-1545
Repository: drill-git
Description
-------
updated JSONFormatPlugin, adding an extensions array into JSONFormatConfig. I used TextFormatPlugin as a reference.
Now the user can set the following format in Web UI:
"json": {
"type": "json",
"extensions": [
"json",
"log"
]
}
The user can also write a separate config for json and log like this:
"json": {
"type": "json",
"extensions": [
"json"
]
},
"log": {
"type": "json",
"extensions": [
"log"
]
}
Diffs (updated)
-----
exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONFormatPlugin.java e1165a2
exec/java-exec/src/main/resources/bootstrap-storage-plugins.json 4a20bea
Diff: https://reviews.apache.org/r/28173/diff/
Testing
-------
created two copies of the same .json data file, one with .json extension and another with .log extension.
updated "json" format like this:
"json": {
"type": "json",
"extensions": [
"json",
"log"
]
}
once the change was done, I was able to query both files from sqlline.
Thanks,
abdelhakim deneche
Re: Review Request 28173: DRILL-1545 - Json files can only be read
when they have a .json extension
Posted by abdelhakim deneche <ad...@gmail.com>.
> On nov. 18, 2014, 4:52 après-midi, Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONFormatPlugin.java, line 75
> > <https://reviews.apache.org/r/28173/diff/1/?file=767087#file767087line75>
> >
> > What is the purpose of this and why is it only get(0) than the entire list?
I wondered the same actually, but I basically copy/pasted this code from TextFormatPlugin.
Should I change it to take the entire list instead ?
- abdelhakim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28173/#review61932
-----------------------------------------------------------
On nov. 18, 2014, 4:48 après-midi, abdelhakim deneche wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28173/
> -----------------------------------------------------------
>
> (Updated nov. 18, 2014, 4:48 après-midi)
>
>
> Review request for drill.
>
>
> Bugs: DRILL-1545
> https://issues.apache.org/jira/browse/DRILL-1545
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> updated JSONFormatPlugin, adding an extensions array into JSONFormatConfig. I used TextFormatPlugin as a reference.
> Now the user can set the following format in Web UI:
> "json": {
> "type": "json",
> "extensions": [
> "json",
> "log"
> ]
> }
>
> The user can also write a separate config for json and log like this:
> "json": {
> "type": "json",
> "extensions": [
> "json"
> ]
> },
> "log": {
> "type": "json",
> "extensions": [
> "log"
> ]
> }
>
>
> Diffs
> -----
>
> exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONFormatPlugin.java e1165a2
> exec/java-exec/src/main/resources/bootstrap-storage-plugins.json 4a20bea
>
> Diff: https://reviews.apache.org/r/28173/diff/
>
>
> Testing
> -------
>
> created two copies of the same .json data file, one with .json extension and another with .log extension.
>
> updated "json" format like this:
>
> "json": {
> "type": "json",
> "extensions": [
> "json",
> "log"
> ]
> }
>
> once the change was done, I was able to query both files from sqlline.
>
>
> Thanks,
>
> abdelhakim deneche
>
>
Re: Review Request 28173: DRILL-1545 - Json files can only be read
when they have a .json extension
Posted by abdelhakim deneche <ad...@gmail.com>.
> On nov. 18, 2014, 4:52 après-midi, Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONFormatPlugin.java, line 75
> > <https://reviews.apache.org/r/28173/diff/1/?file=767087#file767087line75>
> >
> > What is the purpose of this and why is it only get(0) than the entire list?
>
> abdelhakim deneche wrote:
> I wondered the same actually, but I basically copy/pasted this code from TextFormatPlugin.
>
> Should I change it to take the entire list instead ?
I just tested the following format config:
"json": {
"type": "json",
"extensions": [
"json",
"log"
]
}
and I was able to query both .json and .log (with json data inside) fine.
- abdelhakim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28173/#review61932
-----------------------------------------------------------
On nov. 18, 2014, 4:48 après-midi, abdelhakim deneche wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28173/
> -----------------------------------------------------------
>
> (Updated nov. 18, 2014, 4:48 après-midi)
>
>
> Review request for drill.
>
>
> Bugs: DRILL-1545
> https://issues.apache.org/jira/browse/DRILL-1545
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> updated JSONFormatPlugin, adding an extensions array into JSONFormatConfig. I used TextFormatPlugin as a reference.
> Now the user can set the following format in Web UI:
> "json": {
> "type": "json",
> "extensions": [
> "json",
> "log"
> ]
> }
>
> The user can also write a separate config for json and log like this:
> "json": {
> "type": "json",
> "extensions": [
> "json"
> ]
> },
> "log": {
> "type": "json",
> "extensions": [
> "log"
> ]
> }
>
>
> Diffs
> -----
>
> exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONFormatPlugin.java e1165a2
> exec/java-exec/src/main/resources/bootstrap-storage-plugins.json 4a20bea
>
> Diff: https://reviews.apache.org/r/28173/diff/
>
>
> Testing
> -------
>
> created two copies of the same .json data file, one with .json extension and another with .log extension.
>
> updated "json" format like this:
>
> "json": {
> "type": "json",
> "extensions": [
> "json",
> "log"
> ]
> }
>
> once the change was done, I was able to query both files from sqlline.
>
>
> Thanks,
>
> abdelhakim deneche
>
>
Re: Review Request 28173: DRILL-1545 - Json files can only be read
when they have a .json extension
Posted by Steven Phillips <sp...@maprtech.com>.
> On Nov. 18, 2014, 4:52 p.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONFormatPlugin.java, line 75
> > <https://reviews.apache.org/r/28173/diff/1/?file=767087#file767087line75>
> >
> > What is the purpose of this and why is it only get(0) than the entire list?
>
> abdelhakim deneche wrote:
> I wondered the same actually, but I basically copy/pasted this code from TextFormatPlugin.
>
> Should I change it to take the entire list instead ?
>
> abdelhakim deneche wrote:
> I just tested the following format config:
> "json": {
> "type": "json",
> "extensions": [
> "json",
> "log"
> ]
> }
>
> and I was able to query both .json and .log (with json data inside) fine.
That line is setting the extension for the output file of the writer. It only needs or can use one value.
- Steven
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28173/#review61932
-----------------------------------------------------------
On Nov. 18, 2014, 4:48 p.m., abdelhakim deneche wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28173/
> -----------------------------------------------------------
>
> (Updated Nov. 18, 2014, 4:48 p.m.)
>
>
> Review request for drill.
>
>
> Bugs: DRILL-1545
> https://issues.apache.org/jira/browse/DRILL-1545
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> updated JSONFormatPlugin, adding an extensions array into JSONFormatConfig. I used TextFormatPlugin as a reference.
> Now the user can set the following format in Web UI:
> "json": {
> "type": "json",
> "extensions": [
> "json",
> "log"
> ]
> }
>
> The user can also write a separate config for json and log like this:
> "json": {
> "type": "json",
> "extensions": [
> "json"
> ]
> },
> "log": {
> "type": "json",
> "extensions": [
> "log"
> ]
> }
>
>
> Diffs
> -----
>
> exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONFormatPlugin.java e1165a2
> exec/java-exec/src/main/resources/bootstrap-storage-plugins.json 4a20bea
>
> Diff: https://reviews.apache.org/r/28173/diff/
>
>
> Testing
> -------
>
> created two copies of the same .json data file, one with .json extension and another with .log extension.
>
> updated "json" format like this:
>
> "json": {
> "type": "json",
> "extensions": [
> "json",
> "log"
> ]
> }
>
> once the change was done, I was able to query both files from sqlline.
>
>
> Thanks,
>
> abdelhakim deneche
>
>
Re: Review Request 28173: DRILL-1545 - Json files can only be read
when they have a .json extension
Posted by Jacques Nadeau <ja...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28173/#review61932
-----------------------------------------------------------
exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONFormatPlugin.java
<https://reviews.apache.org/r/28173/#comment103863>
What is the purpose of this and why is it only get(0) than the entire list?
- Jacques Nadeau
On Nov. 18, 2014, 4:48 p.m., abdelhakim deneche wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28173/
> -----------------------------------------------------------
>
> (Updated Nov. 18, 2014, 4:48 p.m.)
>
>
> Review request for drill.
>
>
> Bugs: DRILL-1545
> https://issues.apache.org/jira/browse/DRILL-1545
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> updated JSONFormatPlugin, adding an extensions array into JSONFormatConfig. I used TextFormatPlugin as a reference.
> Now the user can set the following format in Web UI:
> "json": {
> "type": "json",
> "extensions": [
> "json",
> "log"
> ]
> }
>
> The user can also write a separate config for json and log like this:
> "json": {
> "type": "json",
> "extensions": [
> "json"
> ]
> },
> "log": {
> "type": "json",
> "extensions": [
> "log"
> ]
> }
>
>
> Diffs
> -----
>
> exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONFormatPlugin.java e1165a2
> exec/java-exec/src/main/resources/bootstrap-storage-plugins.json 4a20bea
>
> Diff: https://reviews.apache.org/r/28173/diff/
>
>
> Testing
> -------
>
> created two copies of the same .json data file, one with .json extension and another with .log extension.
>
> updated "json" format like this:
>
> "json": {
> "type": "json",
> "extensions": [
> "json",
> "log"
> ]
> }
>
> once the change was done, I was able to query both files from sqlline.
>
>
> Thanks,
>
> abdelhakim deneche
>
>