You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by "shfshihuafeng (via GitHub)" <gi...@apache.org> on 2023/11/27 07:11:44 UTC
[I] it cause an exception when query http plugin data with fullName (drill)
shfshihuafeng opened a new issue, #2851:
URL: https://github.com/apache/drill/issues/2851
Before submitting a bug report, please verify that you are using the most current version of Drill.
**Describe the bug**
i create a http plugin named "http" and then use "use http" command to switch schema.
i query http Data with "select * from http.sunrise"，It report except
**To Reproduce**
Steps to reproduce the behavior:
1. create plugin with name "http"
2. use http
3. select * from http.sunrise;
**Expected behavior**
apache drill> select * from http.sunrise;
+----------------------------------------------------------------------------------+--------+
| results | status |
+----------------------------------------------------------------------------------+--------+
| {"sunrise":"5:42:47 AM","sunset":"5:52:15 PM","solar_noon":"11:47:31 AM","day_length":"12:09:28","civil_twilight_begin":"5:21:47 AM","civil_twilight_end":"6:13:15 PM","nautical_twilight_begin":"4:56:00 AM","nautical_twilight_end":"6:39:02 PM","astronomical_twilight_begin":"4:30:08 AM","astronomical_twilight_end":"7:04:54 PM"} | OK |
+----------------------------------------------------------------------------------+--------+
1 row selected (2.967 seconds)
**Error detail, log output or screenshots**
Error: CONNECTION ERROR: API 'http' does not exist in HTTP storage plugin 'http'
**Drill version**
1.22.0
**Additional context**
http plugin config
{
"type": "http",
"connections": {
"sunrise": {
"url": "https://api.sunrise-sunset.org/json",
"requireTail": false,
"method": "GET",
"params": [
"lat",
"lng",
"date"
],
"authType": "none",
"inputType": "json",
"xmlDataLevel": 1,
"postParameterLocation": "QUERY_STRING",
"verifySSLCert": true
}
},
"timeout": 5,
"retryDelay": 1000,
"proxyType": "direct",
"authMode": "SHARED_USER",
"enabled": true
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] it cause an exception when query http plugin data with fullName (drill)
Posted by "cgivre (via GitHub)" <gi...@apache.org>.
cgivre commented on issue #2851:
URL: https://github.com/apache/drill/issues/2851#issuecomment-1827794423
@shfshihuafeng I don't think this is a bug. You are first running the query `USE http` which sets the root at `http`. Then you're running a `SELECT ... FROM http.sunrise`. So at that point, the plugin is looking for a path: `http.http.sunrise` which does not exist.
I'd bet that if you either skipped the `USE` query or ran a `SELECT ... FROM sunrise` it would work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] it cause an exception when query http plugin data with fullName (drill)
Posted by "Javelin2007 (via GitHub)" <gi...@apache.org>.
Javelin2007 commented on issue #2851:
URL: https://github.com/apache/drill/issues/2851#issuecomment-1828871115
@cgivre You are right, but when use http, the default schema is http, when we use another data source schema, will report error.
It think this is not right.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] it cause an exception when query http plugin data with fullName (drill)
Posted by "shfshihuafeng (via GitHub)" <gi...@apache.org>.
shfshihuafeng commented on issue #2851:
URL: https://github.com/apache/drill/issues/2851#issuecomment-1828922228
@cgivre
It is correct to query data with full schema Name from other data sources
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] it cause an exception when query http plugin data with fullName (drill)
Posted by "shfshihuafeng (via GitHub)" <gi...@apache.org>.
shfshihuafeng commented on issue #2851:
URL: https://github.com/apache/drill/issues/2851#issuecomment-1828919544
@cgivre
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] it cause an exception when query http plugin data with fullName (drill)
Posted by "shfshihuafeng (via GitHub)" <gi...@apache.org>.
shfshihuafeng commented on issue #2851:
URL: https://github.com/apache/drill/issues/2851#issuecomment-1849470023
@cgivre
There are two problems with this
1. behavior with USE is inconsistent
2. when i enter the scehma with 'use',data can not be queried
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] it cause an exception when query http plugin data with fullName (drill)
Posted by "cgivre (via GitHub)" <gi...@apache.org>.
cgivre commented on issue #2851:
URL: https://github.com/apache/drill/issues/2851#issuecomment-1843610920
@shfshihuafeng
So are you saying the behavior with `USE` is inconsistent? I want to make sure I understand the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] it cause an exception when query http plugin data with fullName (drill)
Posted by "shfshihuafeng (via GitHub)" <gi...@apache.org>.
shfshihuafeng commented on issue #2851:
URL: https://github.com/apache/drill/issues/2851#issuecomment-1827601211
index ef6368d63..de764db2e 100644
--- a/contrib/storage-http/src/main/java/org/apache/drill/exec/store/http/HttpSchemaFactory.java
+++ b/contrib/storage-http/src/main/java/org/apache/drill/exec/store/http/HttpSchemaFactory.java
@@ -85,7 +85,7 @@ public class HttpSchemaFactory extends AbstractSchemaFactory {
HttpAPIConnectionSchema subSchema = subSchemas.get(name);
if (subSchema != null) {
return subSchema;
- } else if (tables.containsKey(name)) {
+ } else if (tables.containsKey(name) || plugin.getName().equals(name)) {
return null;
} else {
throw UserException
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] it cause an exception when query http plugin data with fullName (drill)
Posted by "niemipt (via GitHub)" <gi...@apache.org>.
niemipt commented on issue #2851:
URL: https://github.com/apache/drill/issues/2851#issuecomment-2434619914
I think that I have exactly the same problem. It seems that after you change schema to http storage schema you cannot run any SQL.
> ➜ tmp cat bootstrap-storage-plugins.json
> {
> "storage": {
> "http": {
> "type": "http",
> "cacheResults": false,
> "enabled": true,
> "timeout": 5,
> "connections": {
> "sunrise": {
> "url": "https://api.sunrise-sunset.org/json",
> "requireTail": false,
> "method": "GET",
> "dataPath": "results",
> "headers": null,
> "params": [
> "lat",
> "lng",
> "date"
> ],
> "authType": "none",
> "userName": null,
> "password": null,
> "postBody": null
> }
> }
> }
> }
> }
> ➜ tmp docker run -it --rm --name drill -v "${HOME}"/tmp/bootstrap-storage-plugins.json:/opt/drill/conf/bootstrap-storage-plugins.json apache/drill
> WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
> Apache Drill 1.21.1
> "Drill must go on."
> apache drill> !schemas
> +--------------------+---------------+
> | TABLE_SCHEM | TABLE_CATALOG |
> +--------------------+---------------+
> | cp.default | DRILL |
> | dfs.default | DRILL |
> | dfs.root | DRILL |
> | dfs.tmp | DRILL |
> | http | DRILL |
> | information_schema | DRILL |
> | sys | DRILL |
> +--------------------+---------------+
> apache drill> SELECT version FROM sys.version;
> +---------+
> | version |
> +---------+
> | 1.21.1 |
> +---------+
> 1 row selected (2.638 seconds)
> apache drill> SELECT sunrise, sunset FROM http.sunrise WHERE `lat` = 36.7201600 AND `lng` = -4.4203400 AND `date` = 'today';
> +------------+------------+
> | sunrise | sunset |
> +------------+------------+
> | 6:32:42 AM | 5:30:55 PM |
> +------------+------------+
> 1 row selected (2.506 seconds)
> apache drill> use http;
> +------+----------------------------------+
> | ok | summary |
> +------+----------------------------------+
> | true | Default schema changed to [http] |
> +------+----------------------------------+
> 1 row selected (0.219 seconds)
> apache drill (http)> SELECT sunrise, sunset FROM sunrise WHERE `lat` = 36.7201600 AND `lng` = -4.4203400 AND `date` = 'today';
> Error: CONNECTION ERROR: API 'http' does not exist in HTTP storage plugin 'http'
>
>
> [Error Id: 574dff2e-53ac-47c8-a4dc-f4368f5d60ea ] (state=,code=0)
> apache drill (http)> SELECT sunrise, sunset FROM http.sunrise WHERE `lat` = 36.7201600 AND `lng` = -4.4203400 AND `date` = 'today';
> Error: CONNECTION ERROR: API 'http' does not exist in HTTP storage plugin 'http'
>
>
> [Error Id: ff623dec-3ecc-4eed-9caa-bad804694ea9 ] (state=,code=0)
> apache drill (http)> SELECT version FROM sys.version;
> Error: CONNECTION ERROR: API 'sys' does not exist in HTTP storage plugin 'http'
>
>
> [Error Id: fd4309e5-8883-4219-b0b7-3beec7147b96 ] (state=,code=0)
> apache drill (http)> use sys;
> Error: CONNECTION ERROR: API 'sys' does not exist in HTTP storage plugin 'http'
>
>
> [Error Id: 08a3b91a-a216-4088-8fc2-e0bc0d79bfc1 ] (state=,code=0)
> apache drill (http)> !schemas
> +--------------------+---------------+
> | TABLE_SCHEM | TABLE_CATALOG |
> +--------------------+---------------+
> | cp.default | DRILL |
> | dfs.default | DRILL |
> | dfs.root | DRILL |
> | dfs.tmp | DRILL |
> | http | DRILL |
> | information_schema | DRILL |
> | sys | DRILL |
> +--------------------+---------------+
> apache drill (http)> !reconnect
> Reconnecting to "jdbc:drill:zk=local"...
> apache drill> SELECT version FROM sys.version;
> +---------+
> | version |
> +---------+
> | 1.21.1 |
> +---------+
> 1 row selected (0.24 seconds)
> apache drill> !quit
> ➜ tmp
Only exception is that if you name HTTP storage and connection with same name (eg. schema sunrise and connection sunrise) you can query data using both schema.connection and connection but you still can't query any other schema using anotherschema.connection.
> ➜ tmp cat bootstrap-storage-plugins.json
> {
> "storage": {
> "sunrise": {
> "type": "http",
> "cacheResults": false,
> "enabled": true,
> "timeout": 5,
> "connections": {
> "sunrise": {
> "url": "https://api.sunrise-sunset.org/json",
> "requireTail": false,
> "method": "GET",
> "dataPath": "results",
> "headers": null,
> "params": [
> "lat",
> "lng",
> "date"
> ],
> "authType": "none",
> "userName": null,
> "password": null,
> "postBody": null
> }
> }
> }
> }
> }
> ➜ tmp docker run -it --rm --name drill -v "${HOME}"/tmp/bootstrap-storage-plugins.json:/opt/drill/conf/bootstrap-storage-plugins.json apache/drill
> WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
> Apache Drill 1.21.1
> "Say hello to my little Drill."
> apache drill> !schemas
> +--------------------+---------------+
> | TABLE_SCHEM | TABLE_CATALOG |
> +--------------------+---------------+
> | cp.default | DRILL |
> | dfs.default | DRILL |
> | dfs.root | DRILL |
> | dfs.tmp | DRILL |
> | information_schema | DRILL |
> | sunrise | DRILL |
> | sys | DRILL |
> +--------------------+---------------+
> apache drill> SELECT version FROM sys.version;
> +---------+
> | version |
> +---------+
> | 1.21.1 |
> +---------+
> 1 row selected (2.235 seconds)
> apache drill> SELECT sunrise, sunset FROM sunrise.sunrise WHERE `lat` = 36.7201600 AND `lng` = -4.4203400 AND `date` = 'today';
> +------------+------------+
> | sunrise | sunset |
> +------------+------------+
> | 6:32:42 AM | 5:30:55 PM |
> +------------+------------+
> 1 row selected (2.545 seconds)
> apache drill> use sunrise;
> +------+-------------------------------------+
> | ok | summary |
> +------+-------------------------------------+
> | true | Default schema changed to [sunrise] |
> +------+-------------------------------------+
> 1 row selected (0.188 seconds)
> apache drill (sunrise)> SELECT sunrise, sunset FROM sunrise.sunrise WHERE `lat` = 36.7201600 AND `lng` = -4.4203400 AND `date` = 'today';
> +------------+------------+
> | sunrise | sunset |
> +------------+------------+
> | 6:32:42 AM | 5:30:55 PM |
> +------------+------------+
> 1 row selected (1.146 seconds)
> apache drill (sunrise)> SELECT sunrise, sunset FROM sunrise WHERE `lat` = 36.7201600 AND `lng` = -4.4203400 AND `date` = 'today';
> +------------+------------+
> | sunrise | sunset |
> +------------+------------+
> | 6:32:42 AM | 5:30:55 PM |
> +------------+------------+
> 1 row selected (1.143 seconds)
> apache drill (sunrise)> SELECT version FROM sys.version;
> Error: CONNECTION ERROR: API 'sys' does not exist in HTTP storage plugin 'sunrise'
>
>
> [Error Id: 8cb31758-252b-4949-a30e-100b469d2476 ] (state=,code=0)
> apache drill (sunrise)> use sys;
> Error: CONNECTION ERROR: API 'sys' does not exist in HTTP storage plugin 'sunrise'
>
>
> [Error Id: 3d1e62b3-f395-4509-9001-5a47376e6063 ] (state=,code=0)
> apache drill (sunrise)> !schemas
> +--------------------+---------------+
> | TABLE_SCHEM | TABLE_CATALOG |
> +--------------------+---------------+
> | cp.default | DRILL |
> | dfs.default | DRILL |
> | dfs.root | DRILL |
> | dfs.tmp | DRILL |
> | information_schema | DRILL |
> | sunrise | DRILL |
> | sys | DRILL |
> +--------------------+---------------+
> apache drill (sunrise)> !quit
> ➜ tmp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org