You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by GitBox <gi...@apache.org> on 2021/11/28 11:42:50 UTC
[GitHub] [drill] Z0ltrix opened a new pull request #2387: DRILL-8055: Druid Plugin support for V0.17 and up
Z0ltrix opened a new pull request #2387:
URL: https://github.com/apache/drill/pull/2387
# [DRILL-8055](https://issues.apache.org/jira/browse/DRILL-8055): Druid Plugin support for V0.17 and up
## Description
Switched from deprecated Select API to Scan API to support Druid >= 0.17
## Documentation
Todo...
## Testing
Automated Testing in Storage Plugin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] Z0ltrix commented on pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
Z0ltrix commented on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-991803420
> > > @Z0ltrix Thank you very much for this PR! I don't know how much time you have to work on this, but I wanted to suggest that it might be worth adding some additional pushdowns to the Druid plugin. This got a lot easier since DRILL-7985 (#2289) when @vvysotskyi added the `implementor` class. I don't really know what is possible in the new Druid interface, but maybe take a look and see what you think.
> >
> >
> > Hi @cgivre what do you think about creating a new Ticket for this Change? If everything goes well we could include this PR in the next Release 1.20 so that users are able to use Druid in newer Versions. Regards Christian
>
> I'm fine with that approach. I was a bit concerned about scope creep but I thought I'd mention it since this is the area of the plugin that is affected. Is this ready for review? I'd like to see this get into Drill 1.20 as well.
yes... ready for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] cgivre commented on pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
cgivre commented on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-991801401
> > @Z0ltrix Thank you very much for this PR! I don't know how much time you have to work on this, but I wanted to suggest that it might be worth adding some additional pushdowns to the Druid plugin. This got a lot easier since DRILL-7985 (#2289) when @vvysotskyi added the `implementor` class. I don't really know what is possible in the new Druid interface, but maybe take a look and see what you think.
>
> Hi @cgivre what do you think about creating a new Ticket for this Change? If everything goes well we could include this PR in the next Release 1.20 so that users are able to use Druid in newer Versions. Regards Christian
I'm fine with that approach. I was a bit concerned about scope creep but I thought I'd mention it since this is the area of the plugin that is affected. Is this ready for review?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] Z0ltrix commented on pull request #2387: DRILL-8055: Druid Plugin support for V0.17 and up
Posted by GitBox <gi...@apache.org>.
Z0ltrix commented on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-991626929
> @Z0ltrix Thank you very much for this PR! I don't know how much time you have to work on this, but I wanted to suggest that it might be worth adding some additional pushdowns to the Druid plugin. This got a lot easier since DRILL-7985 (#2289) when @vvysotskyi added the `implementor` class. I don't really know what is possible in the new Druid interface, but maybe take a look and see what you think.
Hi @cgivre
what do you think about creating a new Ticket for this Change?
If everything goes well we could include this PR in the next Release 1.20 so that users are able to use Drill in newer Versions.
Regards
Christian
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] cgivre edited a comment on pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
cgivre edited a comment on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-991801401
> > @Z0ltrix Thank you very much for this PR! I don't know how much time you have to work on this, but I wanted to suggest that it might be worth adding some additional pushdowns to the Druid plugin. This got a lot easier since DRILL-7985 (#2289) when @vvysotskyi added the `implementor` class. I don't really know what is possible in the new Druid interface, but maybe take a look and see what you think.
>
> Hi @cgivre what do you think about creating a new Ticket for this Change? If everything goes well we could include this PR in the next Release 1.20 so that users are able to use Druid in newer Versions. Regards Christian
I'm fine with that approach. I was a bit concerned about scope creep but I thought I'd mention it since this is the area of the plugin that is affected. Is this ready for review? I'd like to see this get into Drill 1.20 as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] cgivre commented on pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
cgivre commented on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-991801401
> > @Z0ltrix Thank you very much for this PR! I don't know how much time you have to work on this, but I wanted to suggest that it might be worth adding some additional pushdowns to the Druid plugin. This got a lot easier since DRILL-7985 (#2289) when @vvysotskyi added the `implementor` class. I don't really know what is possible in the new Druid interface, but maybe take a look and see what you think.
>
> Hi @cgivre what do you think about creating a new Ticket for this Change? If everything goes well we could include this PR in the next Release 1.20 so that users are able to use Druid in newer Versions. Regards Christian
I'm fine with that approach. I was a bit concerned about scope creep but I thought I'd mention it since this is the area of the plugin that is affected. Is this ready for review?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] lgtm-com[bot] commented on pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
lgtm-com[bot] commented on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-991812914
This pull request **fixes 1 alert** when merging 75b7d6a84131e31f9810df1602ba39a5af6e9e9b into b3ffa80cf0412b99ccf40fb4ba4edff452ffc6d3 - [view on LGTM.com](https://lgtm.com/projects/g/apache/drill/rev/pr-2ea412d5adf8ba133032567648332151efffaeaa)
**fixed alerts:**
* 1 for Result of multiplication cast to wider type
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] cgivre commented on a change in pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
cgivre commented on a change in pull request #2387:
URL: https://github.com/apache/drill/pull/2387#discussion_r767206461
##########
File path: contrib/storage-druid/README.md
##########
@@ -4,13 +4,13 @@ Drill druid storage plugin allows you to perform SQL queries against Druid datas
This storage plugin is part of [Apache Drill](https://github.com/apache/drill)
### Tested with Druid version
-[0.16.0-incubating](https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating)
+[0.22.0](https://github.com/apache/druid/releases/tag/druid-0.22.0)
Review comment:
Perfect. Thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] lgtm-com[bot] commented on pull request #2387: DRILL-8055: Druid Plugin support for V0.17 and up
Posted by GitBox <gi...@apache.org>.
lgtm-com[bot] commented on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-991360159
This pull request **fixes 1 alert** when merging b96bafcca072b6f7e3c448a0f5a70f83399caf30 into fe1d15f1941334792e3f1c1dd53d932dd1b66279 - [view on LGTM.com](https://lgtm.com/projects/g/apache/drill/rev/pr-90a02bd9b9822c6e5c0242df22c9e0aa221f0eeb)
**fixed alerts:**
* 1 for Result of multiplication cast to wider type
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] Z0ltrix commented on a change in pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
Z0ltrix commented on a change in pull request #2387:
URL: https://github.com/apache/drill/pull/2387#discussion_r767203915
##########
File path: contrib/storage-druid/src/test/java/org/apache/drill/exec/store/druid/rest/DruidQueryClientTest.java
##########
@@ -90,23 +90,23 @@ public void executeQueryCalledNoResponsesFoundReturnsEmptyEventList()
new ByteArrayInputStream("[]".getBytes(StandardCharsets.UTF_8.name()));
when(httpEntity.getContent()).thenReturn(inputStream);
- DruidSelectResponse response = druidQueryClient.executeQuery(QUERY);
+ DruidScanResponse response = druidQueryClient.executeQuery(QUERY);
assertThat(response.getEvents()).isEmpty();
- assertThat(response.getPagingIdentifiers()).isEmpty();
+ //assertThat(response.getPagingIdentifiers()).isEmpty();
Review comment:
oh... yes definitely... missed that during coding
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] cgivre edited a comment on pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
cgivre edited a comment on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-991801401
> > @Z0ltrix Thank you very much for this PR! I don't know how much time you have to work on this, but I wanted to suggest that it might be worth adding some additional pushdowns to the Druid plugin. This got a lot easier since DRILL-7985 (#2289) when @vvysotskyi added the `implementor` class. I don't really know what is possible in the new Druid interface, but maybe take a look and see what you think.
>
> Hi @cgivre what do you think about creating a new Ticket for this Change? If everything goes well we could include this PR in the next Release 1.20 so that users are able to use Druid in newer Versions. Regards Christian
I'm fine with that approach. I was a bit concerned about scope creep but I thought I'd mention it since this is the area of the plugin that is affected. Is this ready for review? I'd like to see this get into Drill 1.20 as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] Z0ltrix commented on pull request #2387: DRILL-8055: Druid Plugin support for V0.17 and up
Posted by GitBox <gi...@apache.org>.
Z0ltrix commented on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-991626929
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] Z0ltrix edited a comment on pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
Z0ltrix edited a comment on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-991626929
> @Z0ltrix Thank you very much for this PR! I don't know how much time you have to work on this, but I wanted to suggest that it might be worth adding some additional pushdowns to the Druid plugin. This got a lot easier since DRILL-7985 (#2289) when @vvysotskyi added the `implementor` class. I don't really know what is possible in the new Druid interface, but maybe take a look and see what you think.
Hi @cgivre
what do you think about creating a new Ticket for this Change?
If everything goes well we could include this PR in the next Release 1.20 so that users are able to use Druid in newer Versions.
Regards
Christian
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] lgtm-com[bot] commented on pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
lgtm-com[bot] commented on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-991647891
This pull request **fixes 1 alert** when merging 0a4ce507e7c0c5f7250f0087dba646554adab71a into b3ffa80cf0412b99ccf40fb4ba4edff452ffc6d3 - [view on LGTM.com](https://lgtm.com/projects/g/apache/drill/rev/pr-706570ea0844289e813d5f36cfb23e4ee19e18ef)
**fixed alerts:**
* 1 for Result of multiplication cast to wider type
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] lgtm-com[bot] commented on pull request #2387: DRILL-8055: Druid Plugin support for V0.17 and up
Posted by GitBox <gi...@apache.org>.
lgtm-com[bot] commented on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-991360159
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] cgivre commented on a change in pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
cgivre commented on a change in pull request #2387:
URL: https://github.com/apache/drill/pull/2387#discussion_r767202665
##########
File path: contrib/storage-druid/README.md
##########
@@ -4,13 +4,13 @@ Drill druid storage plugin allows you to perform SQL queries against Druid datas
This storage plugin is part of [Apache Drill](https://github.com/apache/drill)
### Tested with Druid version
-[0.16.0-incubating](https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating)
+[0.22.0](https://github.com/apache/druid/releases/tag/druid-0.22.0)
Review comment:
Are there any versions of Druid that this will NOT work with? If so, maybe say that this will work with Druid > n or whatever.... I'm not very knowledgable of Druid, so I don't really know if that is even relevant.
##########
File path: contrib/storage-druid/src/test/java/org/apache/drill/exec/store/druid/rest/DruidQueryClientTest.java
##########
@@ -90,23 +90,23 @@ public void executeQueryCalledNoResponsesFoundReturnsEmptyEventList()
new ByteArrayInputStream("[]".getBytes(StandardCharsets.UTF_8.name()));
when(httpEntity.getContent()).thenReturn(inputStream);
- DruidSelectResponse response = druidQueryClient.executeQuery(QUERY);
+ DruidScanResponse response = druidQueryClient.executeQuery(QUERY);
assertThat(response.getEvents()).isEmpty();
- assertThat(response.getPagingIdentifiers()).isEmpty();
+ //assertThat(response.getPagingIdentifiers()).isEmpty();
Review comment:
Please remove commented out code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] cgivre commented on pull request #2387: DRILL-8055: Druid Plugin support for V0.17 and up
Posted by GitBox <gi...@apache.org>.
cgivre commented on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-985143617
@Z0ltrix Thank you very much for this PR! I don't know how much time you have to work on this, but I wanted to suggest that it might be worth adding some additional pushdowns to the Druid plugin. This got a lot easier since DRILL-7985 (https://github.com/apache/drill/pull/2289) when @vvysotskyi added the `implementor` class. I don't really know what is possible in the new Druid interface, but maybe take a look and see what you think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] cgivre merged pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
cgivre merged pull request #2387:
URL: https://github.com/apache/drill/pull/2387
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] Z0ltrix commented on pull request #2387: DRILL-8055: Druid Plugin support for V0.17 and up
Posted by GitBox <gi...@apache.org>.
Z0ltrix commented on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-981070620
compile and normal test works, but TestDrillQueries has problems in compose file. I need to address them first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] cgivre commented on a change in pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
cgivre commented on a change in pull request #2387:
URL: https://github.com/apache/drill/pull/2387#discussion_r767206469
##########
File path: contrib/storage-druid/src/test/java/org/apache/drill/exec/store/druid/rest/DruidQueryClientTest.java
##########
@@ -90,23 +90,23 @@ public void executeQueryCalledNoResponsesFoundReturnsEmptyEventList()
new ByteArrayInputStream("[]".getBytes(StandardCharsets.UTF_8.name()));
when(httpEntity.getContent()).thenReturn(inputStream);
- DruidSelectResponse response = druidQueryClient.executeQuery(QUERY);
+ DruidScanResponse response = druidQueryClient.executeQuery(QUERY);
assertThat(response.getEvents()).isEmpty();
- assertThat(response.getPagingIdentifiers()).isEmpty();
+ //assertThat(response.getPagingIdentifiers()).isEmpty();
Review comment:
Thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] Z0ltrix commented on a change in pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
Z0ltrix commented on a change in pull request #2387:
URL: https://github.com/apache/drill/pull/2387#discussion_r767203793
##########
File path: contrib/storage-druid/README.md
##########
@@ -4,13 +4,13 @@ Drill druid storage plugin allows you to perform SQL queries against Druid datas
This storage plugin is part of [Apache Drill](https://github.com/apache/drill)
### Tested with Druid version
-[0.16.0-incubating](https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating)
+[0.22.0](https://github.com/apache/druid/releases/tag/druid-0.22.0)
Review comment:
Druid 0.17.x to 0.19.x does not have the "offset" parametser in the scan query api, so paging is not possible. I thought it would be better to avoid all Druid Versions < 0.20.x for scan api instead of sacrifice pagination and reading all rows at once.
This means:
- Drill < 1.20 works well with Druid <= 0.16 (select api)
- Drill >= 1.20 will work with Druid >= 0.20 (scan api)
- Druid 0.17 to 0.19 will not be supported by Drill because of missing offset parameter in scan api
I mentioned this in the docs https://github.com/apache/drill-site/pull/18/files#diff-c47da4191a5cd392dabb624042159b63be990b216242e3da3491896378025e24R45
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] Z0ltrix edited a comment on pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
Z0ltrix edited a comment on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-991626929
> @Z0ltrix Thank you very much for this PR! I don't know how much time you have to work on this, but I wanted to suggest that it might be worth adding some additional pushdowns to the Druid plugin. This got a lot easier since DRILL-7985 (#2289) when @vvysotskyi added the `implementor` class. I don't really know what is possible in the new Druid interface, but maybe take a look and see what you think.
Hi @cgivre
what do you think about creating a new Ticket for this Change?
If everything goes well we could include this PR in the next Release 1.20 so that users are able to use Druid in newer Versions.
Regards
Christian
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] Z0ltrix commented on a change in pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
Z0ltrix commented on a change in pull request #2387:
URL: https://github.com/apache/drill/pull/2387#discussion_r767203793
##########
File path: contrib/storage-druid/README.md
##########
@@ -4,13 +4,13 @@ Drill druid storage plugin allows you to perform SQL queries against Druid datas
This storage plugin is part of [Apache Drill](https://github.com/apache/drill)
### Tested with Druid version
-[0.16.0-incubating](https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating)
+[0.22.0](https://github.com/apache/druid/releases/tag/druid-0.22.0)
Review comment:
Druid 0.17.x to 0.19.x does not have the "offset" parametser in the scan query api, so paging is not possible. I thought it would be better to avoid all Druid Versions < 0.20.x for scan api instead of sacrifice pagination and reading all rows at once.
This means:
- Drill < 1.20 works well with Druid <= 0.16 (select api)
- Drill >= 1.20 will work with Druid >= 0.20 (scan api)
- Druid 0.17 to 0.19 will not be supported by Drill because of missing offset parameter in scan api
I mentioned this in the docs https://github.com/apache/drill-site/pull/18/files#diff-c47da4191a5cd392dabb624042159b63be990b216242e3da3491896378025e24R45
##########
File path: contrib/storage-druid/src/test/java/org/apache/drill/exec/store/druid/rest/DruidQueryClientTest.java
##########
@@ -90,23 +90,23 @@ public void executeQueryCalledNoResponsesFoundReturnsEmptyEventList()
new ByteArrayInputStream("[]".getBytes(StandardCharsets.UTF_8.name()));
when(httpEntity.getContent()).thenReturn(inputStream);
- DruidSelectResponse response = druidQueryClient.executeQuery(QUERY);
+ DruidScanResponse response = druidQueryClient.executeQuery(QUERY);
assertThat(response.getEvents()).isEmpty();
- assertThat(response.getPagingIdentifiers()).isEmpty();
+ //assertThat(response.getPagingIdentifiers()).isEmpty();
Review comment:
oh... yes deffinetly... missed that during coding
##########
File path: contrib/storage-druid/src/test/java/org/apache/drill/exec/store/druid/rest/DruidQueryClientTest.java
##########
@@ -90,23 +90,23 @@ public void executeQueryCalledNoResponsesFoundReturnsEmptyEventList()
new ByteArrayInputStream("[]".getBytes(StandardCharsets.UTF_8.name()));
when(httpEntity.getContent()).thenReturn(inputStream);
- DruidSelectResponse response = druidQueryClient.executeQuery(QUERY);
+ DruidScanResponse response = druidQueryClient.executeQuery(QUERY);
assertThat(response.getEvents()).isEmpty();
- assertThat(response.getPagingIdentifiers()).isEmpty();
+ //assertThat(response.getPagingIdentifiers()).isEmpty();
Review comment:
oh... yes definitely... missed that during coding
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] Z0ltrix commented on a change in pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
Z0ltrix commented on a change in pull request #2387:
URL: https://github.com/apache/drill/pull/2387#discussion_r767203915
##########
File path: contrib/storage-druid/src/test/java/org/apache/drill/exec/store/druid/rest/DruidQueryClientTest.java
##########
@@ -90,23 +90,23 @@ public void executeQueryCalledNoResponsesFoundReturnsEmptyEventList()
new ByteArrayInputStream("[]".getBytes(StandardCharsets.UTF_8.name()));
when(httpEntity.getContent()).thenReturn(inputStream);
- DruidSelectResponse response = druidQueryClient.executeQuery(QUERY);
+ DruidScanResponse response = druidQueryClient.executeQuery(QUERY);
assertThat(response.getEvents()).isEmpty();
- assertThat(response.getPagingIdentifiers()).isEmpty();
+ //assertThat(response.getPagingIdentifiers()).isEmpty();
Review comment:
oh... yes deffinetly... missed that during coding
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] cgivre commented on a change in pull request #2387: DRILL-8055: Druid Plugin support for V0.20 and up
Posted by GitBox <gi...@apache.org>.
cgivre commented on a change in pull request #2387:
URL: https://github.com/apache/drill/pull/2387#discussion_r767202665
##########
File path: contrib/storage-druid/README.md
##########
@@ -4,13 +4,13 @@ Drill druid storage plugin allows you to perform SQL queries against Druid datas
This storage plugin is part of [Apache Drill](https://github.com/apache/drill)
### Tested with Druid version
-[0.16.0-incubating](https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating)
+[0.22.0](https://github.com/apache/druid/releases/tag/druid-0.22.0)
Review comment:
Are there any versions of Druid that this will NOT work with? If so, maybe say that this will work with Druid > n or whatever.... I'm not very knowledgable of Druid, so I don't really know if that is even relevant.
##########
File path: contrib/storage-druid/src/test/java/org/apache/drill/exec/store/druid/rest/DruidQueryClientTest.java
##########
@@ -90,23 +90,23 @@ public void executeQueryCalledNoResponsesFoundReturnsEmptyEventList()
new ByteArrayInputStream("[]".getBytes(StandardCharsets.UTF_8.name()));
when(httpEntity.getContent()).thenReturn(inputStream);
- DruidSelectResponse response = druidQueryClient.executeQuery(QUERY);
+ DruidScanResponse response = druidQueryClient.executeQuery(QUERY);
assertThat(response.getEvents()).isEmpty();
- assertThat(response.getPagingIdentifiers()).isEmpty();
+ //assertThat(response.getPagingIdentifiers()).isEmpty();
Review comment:
Please remove commented out code.
##########
File path: contrib/storage-druid/README.md
##########
@@ -4,13 +4,13 @@ Drill druid storage plugin allows you to perform SQL queries against Druid datas
This storage plugin is part of [Apache Drill](https://github.com/apache/drill)
### Tested with Druid version
-[0.16.0-incubating](https://github.com/apache/incubator-druid/releases/tag/druid-0.16.0-incubating)
+[0.22.0](https://github.com/apache/druid/releases/tag/druid-0.22.0)
Review comment:
Perfect. Thx!
##########
File path: contrib/storage-druid/src/test/java/org/apache/drill/exec/store/druid/rest/DruidQueryClientTest.java
##########
@@ -90,23 +90,23 @@ public void executeQueryCalledNoResponsesFoundReturnsEmptyEventList()
new ByteArrayInputStream("[]".getBytes(StandardCharsets.UTF_8.name()));
when(httpEntity.getContent()).thenReturn(inputStream);
- DruidSelectResponse response = druidQueryClient.executeQuery(QUERY);
+ DruidScanResponse response = druidQueryClient.executeQuery(QUERY);
assertThat(response.getEvents()).isEmpty();
- assertThat(response.getPagingIdentifiers()).isEmpty();
+ //assertThat(response.getPagingIdentifiers()).isEmpty();
Review comment:
Thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] Z0ltrix commented on pull request #2387: DRILL-8055: Druid Plugin support for V0.17 and up
Posted by GitBox <gi...@apache.org>.
Z0ltrix commented on pull request #2387:
URL: https://github.com/apache/drill/pull/2387#issuecomment-985522171
> @Z0ltrix Thank you very much for this PR! I don't know how much time you have to work on this, but I wanted to suggest that it might be worth adding some additional pushdowns to the Druid plugin. This got a lot easier since DRILL-7985 (#2289) when @vvysotskyi added the `implementor` class. I don't really know what is possible in the new Druid interface, but maybe take a look and see what you think.
Hi @cgivre ,
thanks for the hint, i will have a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org