You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/03/04 23:22:45 UTC
[GitHub] [druid] EwanValentine opened a new issue #10945: Scan query throws and error when using smile encoding for response
EwanValentine opened a new issue #10945:
URL: https://github.com/apache/druid/issues/10945
Smile encoding throws an error, using the demo set-up and demo data:
Example cURL command:
```bash
$ curl -X POST 'http://localhost:8082/druid/v2/?pretty' \
-H 'Content-Type:application/json' \
-H 'Accept:application/x-jackson-smile' \
-d @query.json
:)
errorPUnknown exceptionerrorMessageMissing type id when trying to resolve subtype of [simple type, class org.apache.druid.query.Query]: missing type id property 'queryType'
at [Source: (org.eclipse.jetty.server.HttpInputOverHTTP); line: -1, column: 0]errorClassxcom.fasterxml.jackson.databind.exc.InvalidTypeIdExceptionhost!⏎
```
### Affected Version
`0.20.0`
### Description
Please include as much detailed information about the problem as possible.
- Cluster size - local docker-compose example set-up
- Configurations in use - defaults
- Steps to reproduce the problem - Run the cURL command above, use the following query:
```javascript
{
"queryType": "scan",
"dataSource": "wikipedia",
"resultFormat": "list",
"columns":[],
"intervals": [
"2013-01-01/2019-01-02"
],
"batchSize":20480,
"limit":3
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on issue #10945: Scan query throws and error when using smile encoding for response
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on issue #10945:
URL: https://github.com/apache/druid/issues/10945#issuecomment-791523646
I didn't dive into JDBC deeply, but according to the code, you are right.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on issue #10945: Scan query throws and error when using smile encoding for response
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on issue #10945:
URL: https://github.com/apache/druid/issues/10945#issuecomment-791495231
Druid also supports [JDBC](https://druid.apache.org/docs/latest/querying/sql.html#jdbc) and [GZIP over HTTP](https://druid.apache.org/docs/latest/operations/http-compression.html) . I think both of these two ways help reduce network traffic load as you want.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] EwanValentine commented on issue #10945: Scan query throws and error when using smile encoding for response
Posted by GitBox <gi...@apache.org>.
EwanValentine commented on issue #10945:
URL: https://github.com/apache/druid/issues/10945#issuecomment-791475750
@FrankChen021 Yes that's what I was expecting. From reading the docs and looking through the code, I expected to be able to send a query in JSON (`Content-Type: application/json`), then get the data back in smile format, but setting `Accept: application/x-jackson-smile`, as per the docs.
In terms of why we wanted to use Smile: because the docs say it's possible to use it, and there's apparently performance/efficiency benefits for using it over JSON. We're going to be requesting large amounts of data back from Druid. Given the only way to do this (as there's no encoded connection to Druid, similar to say Postgres), is via the HTTP API, we wanted to see how much of a benefit Smile would be over JSON. Hope that makes sense. Advice is very much welcome!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on issue #10945: Scan query throws and error when using smile encoding for response
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on issue #10945:
URL: https://github.com/apache/druid/issues/10945#issuecomment-791351416
It's due to the implementation takes 'Accept' parameter which indicates smile encoding to decode the HTTP body which is actually a JSON.
https://github.com/apache/druid/blob/16acd6686a337fc9a12e9aeda443fca973bdc625/server/src/main/java/org/apache/druid/server/QueryResource.java#L192
https://github.com/apache/druid/blob/16acd6686a337fc9a12e9aeda443fca973bdc625/server/src/main/java/org/apache/druid/server/QueryResource.java#L196
The input reader should alway decode the body according to 'Content-Type' header, while the output writer should encode the response by 'Accept' header. This problem might be introduced by #4033
I'm wondering why you want to use smile ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] EwanValentine commented on issue #10945: Scan query throws and error when using smile encoding for response
Posted by GitBox <gi...@apache.org>.
EwanValentine commented on issue #10945:
URL: https://github.com/apache/druid/issues/10945#issuecomment-791517089
@FrankChen021 thanks for that! We looked into JDBC, we're using Go, but thought we could use Java/Scala for the JDBC connector, but it looks as though the JDBC driver is also utilising HTTP under the hood, see: https://github.com/apache/calcite/blob/master/druid/src/main/java/org/apache/calcite/adapter/druid/DruidConnectionImpl.java#L100
GZIP could work though!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jihoonson closed issue #10945: Scan query throws and error when using smile encoding for response
Posted by GitBox <gi...@apache.org>.
jihoonson closed issue #10945:
URL: https://github.com/apache/druid/issues/10945
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org