You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by "Du, David (Contractor)" <Da...@noblis.org> on 2018/11/13 19:45:49 UTC
LogFile Plugin Configuration does not work
I installed drill-logfile-plugin-1.0.0 JAR file to <drill_install>/jars/3rdParty/ directory, and configured dfs as the following, but I got error: "Please retry: error (invalid JSON mapping)", in the sqlline.log file, it shows an error: Unable to find constructor for storage config named 'log' of type 'org.apache.drill.exec.store.log.LogFormatPlugin$LogFormatConfig, but I double checked the drill-logfile-plugin-1.0.0.jar file is in the jars/3rdParty folder:
My config for dfs with log plugin suport is:
{
"type": "file",
"connection": "file:///",
"config": null,
"workspaces": {
"root": {
"location": "/",
"writable": false,
"defaultInputFormat": null,
"allowAccessOutsideWorkspace": false
},
"test": {
"location": "/Users/tsd",
"writable": false,
"defaultInputFormat": null,
"allowAccessOutsideWorkspace": false
},
"tmp": {
"location": "/tmp",
"writable": true,
"defaultInputFormat": null,
"allowAccessOutsideWorkspace": false
}
},
"formats": {
"log" : {
"type" : "log",
"extensions" : [ "log" ],
"fieldNames" : [ "date", "time", "pid", "action", "query" ],
"dataTypes" : [ "DATE", "TIME", "INT", "VARCHAR", "VARCHAR" ],
"dateFormat" : "yyMMdd",
"timeFormat" : "HH:mm:ss",
"pattern" : "(\\d{6})\\s(\\d{2}:\\d{2}:\\d{2})\\s+(\\d+)\\s(\\w+)\\s+(.+)",
"errorOnMismatch" : false
}
},
"enabled": true
}
If I configured the log section as this to remove some fields, the error will disappear, but some fields will be missing:
"log": {
"type": "log",
"extensions": [
"log"
],
"fieldNames": [
"date",
"time",
"pid",
"action",
"query"
],
"pattern": "(\\d{6})\\s(\\d{2}:\\d{2}:\\d{2})\\s+(\\d+)\\s(\\w+)\\s+(.+)"
}
Re: LogFile Plugin Configuration does not work
Posted by Charles Givre <cg...@gmail.com>.
Hi David,
Please delete the JAR file and follow the instructions here: https://drill.apache.org/docs/logfile-plugin/ <https://drill.apache.org/docs/logfile-plugin/>. Your config is for an older version of the log reader.
—C
> On Nov 13, 2018, at 14:45, Du, David (Contractor) <Da...@noblis.org> wrote:
>
> I installed drill-logfile-plugin-1.0.0 JAR file to <drill_install>/jars/3rdParty/ directory, and configured dfs as the following, but I got error: "Please retry: error (invalid JSON mapping)", in the sqlline.log file, it shows an error: Unable to find constructor for storage config named 'log' of type 'org.apache.drill.exec.store.log.LogFormatPlugin$LogFormatConfig, but I double checked the drill-logfile-plugin-1.0.0.jar file is in the jars/3rdParty folder:
>
> My config for dfs with log plugin suport is:
> {
> "type": "file",
> "connection": "file:///",
> "config": null,
> "workspaces": {
> "root": {
> "location": "/",
> "writable": false,
> "defaultInputFormat": null,
> "allowAccessOutsideWorkspace": false
> },
> "test": {
> "location": "/Users/tsd",
> "writable": false,
> "defaultInputFormat": null,
> "allowAccessOutsideWorkspace": false
> },
> "tmp": {
> "location": "/tmp",
> "writable": true,
> "defaultInputFormat": null,
> "allowAccessOutsideWorkspace": false
> }
> },
> "formats": {
> "log" : {
> "type" : "log",
> "extensions" : [ "log" ],
> "fieldNames" : [ "date", "time", "pid", "action", "query" ],
> "dataTypes" : [ "DATE", "TIME", "INT", "VARCHAR", "VARCHAR" ],
> "dateFormat" : "yyMMdd",
> "timeFormat" : "HH:mm:ss",
> "pattern" : "(\\d{6})\\s(\\d{2}:\\d{2}:\\d{2})\\s+(\\d+)\\s(\\w+)\\s+(.+)",
> "errorOnMismatch" : false
> }
> },
> "enabled": true
> }
>
> If I configured the log section as this to remove some fields, the error will disappear, but some fields will be missing:
> "log": {
> "type": "log",
> "extensions": [
> "log"
> ],
> "fieldNames": [
> "date",
> "time",
> "pid",
> "action",
> "query"
> ],
> "pattern": "(\\d{6})\\s(\\d{2}:\\d{2}:\\d{2})\\s+(\\d+)\\s(\\w+)\\s+(.+)"
> }
>
Re: LogFile Plugin Configuration does not work
Posted by Charles Givre <cg...@gmail.com>.
Hi David,
The docs you’re following reflect an older version of the logfile reader. Here is a link to the correct documentation:
https://github.com/cgivre/drill/blob/24556d857cbbe7aa2baa1fc6cbd85fb614b5d975/exec/java-exec/src/main/java/org/apache/drill/exec/store/log/README.md <https://github.com/cgivre/drill/blob/24556d857cbbe7aa2baa1fc6cbd85fb614b5d975/exec/java-exec/src/main/java/org/apache/drill/exec/store/log/README.md>
Best,
—C
> On Nov 14, 2018, at 10:36, Du, David (Contractor) <Da...@noblis.org> wrote:
>
>
>
> I installed drill-logfile-plugin-1.0.0 JAR file to <drill_install>/jars/3rdParty/ directory, and configured dfs as the following, but I got error: "Please retry: error (invalid JSON mapping)", in the sqlline.log file, it shows an error: Unable to find constructor for storage config named 'log' of type 'org.apache.drill.exec.store.log.LogFormatPlugin$LogFormatConfig, but I double checked the drill-logfile-plugin-1.0.0.jar file is in the jars/3rdParty folder:
>
> My config for dfs with log plugin suport is:
> {
> "type": "file",
> "connection": "file:///",
> "config": null,
> "workspaces": {
> "root": {
> "location": "/",
> "writable": false,
> "defaultInputFormat": null,
> "allowAccessOutsideWorkspace": false
> },
> "test": {
> "location": "/Users/tsd",
> "writable": false,
> "defaultInputFormat": null,
> "allowAccessOutsideWorkspace": false
> },
> "tmp": {
> "location": "/tmp",
> "writable": true,
> "defaultInputFormat": null,
> "allowAccessOutsideWorkspace": false
> }
> },
> "formats": {
> "log" : {
> "type" : "log",
> "extensions" : [ "log" ],
> "fieldNames" : [ "date", "time", "pid", "action", "query" ],
> "dataTypes" : [ "DATE", "TIME", "INT", "VARCHAR", "VARCHAR" ],
> "dateFormat" : "yyMMdd",
> "timeFormat" : "HH:mm:ss",
> "pattern" : "(\\d{6})\\s(\\d{2}:\\d{2}:\\d{2})\\s+(\\d+)\\s(\\w+)\\s+(.+)",
> "errorOnMismatch" : false
> }
> },
> "enabled": true
> }
>
> If I configured the log section as this to remove some fields, the error will disappear, but some fields will be missing:
> "log": {
> "type": "log",
> "extensions": [
> "log"
> ],
> "fieldNames": [
> "date",
> "time",
> "pid",
> "action",
> "query"
> ],
> "pattern": "(\\d{6})\\s(\\d{2}:\\d{2}:\\d{2})\\s+(\\d+)\\s(\\w+)\\s+(.+)"
> }
>
LogFile Plugin Configuration does not work
Posted by "Du, David (Contractor)" <Da...@noblis.org>.
I installed drill-logfile-plugin-1.0.0 JAR file to <drill_install>/jars/3rdParty/ directory, and configured dfs as the following, but I got error: "Please retry: error (invalid JSON mapping)", in the sqlline.log file, it shows an error: Unable to find constructor for storage config named 'log' of type 'org.apache.drill.exec.store.log.LogFormatPlugin$LogFormatConfig, but I double checked the drill-logfile-plugin-1.0.0.jar file is in the jars/3rdParty folder:
My config for dfs with log plugin suport is:
{
"type": "file",
"connection": "file:///",
"config": null,
"workspaces": {
"root": {
"location": "/",
"writable": false,
"defaultInputFormat": null,
"allowAccessOutsideWorkspace": false
},
"test": {
"location": "/Users/tsd",
"writable": false,
"defaultInputFormat": null,
"allowAccessOutsideWorkspace": false
},
"tmp": {
"location": "/tmp",
"writable": true,
"defaultInputFormat": null,
"allowAccessOutsideWorkspace": false
}
},
"formats": {
"log" : {
"type" : "log",
"extensions" : [ "log" ],
"fieldNames" : [ "date", "time", "pid", "action", "query" ],
"dataTypes" : [ "DATE", "TIME", "INT", "VARCHAR", "VARCHAR" ],
"dateFormat" : "yyMMdd",
"timeFormat" : "HH:mm:ss",
"pattern" : "(\\d{6})\\s(\\d{2}:\\d{2}:\\d{2})\\s+(\\d+)\\s(\\w+)\\s+(.+)",
"errorOnMismatch" : false
}
},
"enabled": true
}
If I configured the log section as this to remove some fields, the error will disappear, but some fields will be missing:
"log": {
"type": "log",
"extensions": [
"log"
],
"fieldNames": [
"date",
"time",
"pid",
"action",
"query"
],
"pattern": "(\\d{6})\\s(\\d{2}:\\d{2}:\\d{2})\\s+(\\d+)\\s(\\w+)\\s+(.+)"
}