You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/08/13 02:17:06 UTC
[GitHub] [druid] hqx871 opened a new issue #11591: GroupBy query cannot use result cache on broker
hqx871 opened a new issue #11591:
URL: https://github.com/apache/druid/issues/11591
Hi, I want to enable result cache on broker, but I found it does not support group by query. However, more than 50% of our query is group by query. Hope help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduced the Results of the PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following step:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with each no more than 2000 rows.
3. set druid.broker.cache.useCache=true and druid.broker.cache.populateCache=true for broker
4. then take a sql as follow, and I finally got the partialy merged row as {time='2015-09-12T02:00:00.000Z', channel = '#es.wikipedia', namespace = 'Main'}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia_dynamic
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, I think we can enable result-level caching on broker for groupBy v2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduced the result of PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following step:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with each no more than 2000 rows.
3. set druid.broker.cache.useCache and druid.broker.cache.populateCache to be true for broker
4. then take a sql as follow, and I finally got some partialy merged row, i.e {time='2015-09-12T02:00:00.000Z', channel = '#es.wikipedia', namespace = 'Main'}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, I think we can enable result-level caching on broker for groupBy v2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduced the problem described by the PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following step:
1. make GroupByStrategyV2#isCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with each no more than 2000 rows.
3. set druid.broker.cache.useCache and druid.broker.cache.populateCache to be true for broker
4. then take a sql as follow, and I finally got some partialy merged row, i.e {time='2015-09-12T02:00:00.000Z', channel = '#es.wikipedia', namespace = 'Main'}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, I think we can enable result-level caching on broker for groupBy v2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduced the problem of PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following step:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with each no more than 2000 rows.
3. set druid.broker.cache.useCache and druid.broker.cache.populateCache to be true for broker
4. then take a sql as follow, and I finally got some partialy merged row, i.e {time='2015-09-12T02:00:00.000Z', channel = '#es.wikipedia', namespace = 'Main'}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, I think we can enable result-level caching on broker for groupBy v2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduce the Results of the PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following steps:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with each no more than 2000 rows.
3. set druid.broker.cache.useCache=true and druid.broker.cache.populateCache=true for broker
4. then take a sql as follow, and I finally got the partialy merged row as {time='2015-09-12T02:00:00.000Z', channel = '#es.wikipedia', namespace = 'Main'}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia_dynamic
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, I think we can enable result-level caching on broker for groupBy v2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduced the results of the PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following step:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with each no more than 2000 rows.
3. set druid.broker.cache.useCache and druid.broker.cache.populateCache to be true for broker
4. then take a sql as follow, and I finally got some partialy merged row, i.e {time='2015-09-12T02:00:00.000Z', channel = '#es.wikipedia', namespace = 'Main'}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, I think we can enable result-level caching on broker for groupBy v2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 commented on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 commented on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-898400610
I have tried to make GroupByStrategyV2#isCacheable to return true, which will return false on broker before. Then executing some group by case, there is no error and the result is correct as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduce the Results of the PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following steps:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with each no more than 2000 rows.
3. set druid.broker.cache.useCache=true and druid.broker.cache.populateCache=true
4. then take a sql as follow, and I finally got the partialy merged row as {time='2015-09-12T02:00:00.000Z', channel = '#es.wikipedia', namespace = 'Main'}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia_dynamic
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, I think we can enable result-level caching on broker for groupBy v2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 removed a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 removed a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-898400610
I have tried to make GroupByStrategyV2#isCacheable return true, which will return false on broker before. Then executing some group by queries, there is no error and the result is correct as well. Is it right to do this? I have no idea of what effect the change will have.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully to replay the Results #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following steps:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with nearly no more than 2000 rows.
3. then take a sql as follow, and I finally got the partialy merged row as {channel = '#es.wikipedia' and namespace in('Main')}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia_dynamic
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, the result-level cache can be enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduce the Results of the PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following steps:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with nearly no more than 2000 rows.
3. then take a sql as follow, and I finally got the partialy merged row as {channel = '#es.wikipedia' and namespace in('Main')}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia_dynamic
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, the result-level cache can be enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-898783079
I finally found the cause from #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker]. May be we can just disable the segment-level cache on broker and enable result-level cache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduced the problem described by the PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following step:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with each no more than 2000 rows.
3. set druid.broker.cache.useCache and druid.broker.cache.populateCache to be true for broker
4. then take a sql as follow, and I finally got some partialy merged row, i.e {time='2015-09-12T02:00:00.000Z', channel = '#es.wikipedia', namespace = 'Main'}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, I think we can enable result-level caching on broker for groupBy v2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-898174051
cache does not work for groupBy v2 queries. See #11370
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduced the results of the PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following step:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with each no more than 2000 rows.
3. set druid.broker.cache.useCache and druid.broker.cache.populateCache to be true for broker
4. then take a sql as follow, and I finally got the partialy merged row as {time='2015-09-12T02:00:00.000Z', channel = '#es.wikipedia', namespace = 'Main'}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia_dynamic
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, I think we can enable result-level caching on broker for groupBy v2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduce the Results of the PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following steps:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with each no more than 2000 rows.
3. then take a sql as follow, and I finally got the partialy merged row as {time='2015-09-12T02:00:00.000Z', channel = '#es.wikipedia', namespace = 'Main'}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia_dynamic
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, I think the result-level cache can be enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully to reproduce the Results #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following steps:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with nearly no more than 2000 rows.
3. then take a sql as follow, and I finally got the partialy merged row as {channel = '#es.wikipedia' and namespace in('Main')}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia_dynamic
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, the result-level cache can be enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-898400610
I have tried to make GroupByStrategyV2#isCacheable to return true, which will return false on broker before. Then executing some group by queries, there is no error and the result is correct as well. Is it right to do this change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-898400610
I have tried to make GroupByStrategyV2#isCacheable return true, which will return false on broker before. Then executing some group by queries, there is no error and the result is correct as well. Is it right to do this? I have no idea of what effect the change will have.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] abhishekagarwal87 commented on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
abhishekagarwal87 commented on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-898207225
I think we can enable result-level caching on broker for groupBy v2. caching is disabled even before result-level caching came to the fore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduced the results of the PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following step:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with each no more than 2000 rows.
3. set druid.broker.cache.useCache=true and druid.broker.cache.populateCache=true for broker
4. then take a sql as follow, and I finally got the partialy merged row as {time='2015-09-12T02:00:00.000Z', channel = '#es.wikipedia', namespace = 'Main'}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia_dynamic
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, I think we can enable result-level caching on broker for groupBy v2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-898400610
I have tried to make GroupByStrategyV2#isCacheable to return true, which will return false on broker before. Then executing some group by case, there is no error and the result is correct as well. Is it right to change GroupByStrategyV2#isCacheable to return true?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 commented on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 commented on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-898783079
I finally found the cause from issue groupBy v2: Results not fully merged when caching is enabled on the broker #3820. May be we can just disable the segment-level cache on broker and enable result-level cache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 commented on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 commented on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have success to replay the Results #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following steps:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with nearly no more than 2000 rows.
3. then take a sql as follow, and I finally got the partialy merged row as {channel = '#es.wikipedia' and namespace in('Main')}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia_dynamic
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, the result-level cache can be enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduce the Results of the PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following steps:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with each no more than 2000 rows.
3. then take a sql as follow, and I finally got the partialy merged row as {time='2015-09-12T02:00:00.000Z', channel = '#es.wikipedia', namespace = 'Main'}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia_dynamic
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, the result-level cache can be enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduce the Results of the PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following steps:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with each no more than 2000 rows.
3. then take a sql as follow, and I finally got the partialy merged row as {time='2015-09-12T02:00:00.000Z', channel = '#es.wikipedia', namespace = 'Main'}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia_dynamic
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, I think we can enable result-level caching on broker for groupBy v2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduce the Results of the PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following steps:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with each no more than 2000 rows.
3. then take a sql as follow, and I finally got the partialy merged row as {channel = '#es.wikipedia' and namespace in('Main')}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia_dynamic
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, the result-level cache can be enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-898783079
I finally found the cause from issue #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker]. May be we can just disable the segment-level cache on broker and enable result-level cache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-898400610
I have tried to make GroupByStrategyV2#isCacheable to return true, which will return false on broker before. Then executing some group by queries, there is no error and the result is correct as well. Is it right to do this? I have no idea of what effect the change will have.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-898400610
I have tried to make GroupByStrategyV2#isCacheable to return true, which will return false on broker before. Then executing some group by queries, there is no error and the result is correct as well. Is it right to do this change? I have no idea of what effect the change will have.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-899384859
Thanks @abhishekagarwal87 and @FrankChen021. I have successfully reproduced the Results of the PR #3820 [groupBy v2: Results not fully merged when caching is enabled on the broker] by following steps:
1. make GroupByStrategyV2#isQueryCacheable to return true.
2. ingestion the wikiticker-2015-09-12-sampled.json.gz by dynamic partition it into 20 segments with each no more than 2000 rows.
3. set druid.broker.cache.useCache=true and druid.broker.cache.populateCache=true for broker
4. then take a sql as follow, and I finally got the partialy merged row as {time='2015-09-12T02:00:00.000Z', channel = '#es.wikipedia', namespace = 'Main'}
```
select TIME_FLOOR(__time,'PT1H') "time",
channel,
namespace,
sum(added) sum_added
from wikipedia_dynamic
where __time >='2015-09-12T00:00:00.000Z'
and __time <'2015-09-13T00:00:00.000Z'
GROUP BY TIME_FLOOR(__time,'PT1H'),channel,namespace
limit 10000
```
Finally, as @abhishekagarwal87 said, I think we can enable result-level caching on broker for groupBy v2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] hqx871 edited a comment on issue #11591: GroupBy query cannot use result cache on broker
Posted by GitBox <gi...@apache.org>.
hqx871 edited a comment on issue #11591:
URL: https://github.com/apache/druid/issues/11591#issuecomment-898400610
I have tried to make GroupByStrategyV2#isCacheable to return true, which will return false on broker before. Then executing some group by case, there is no error and the result is correct as well. Is it right to change the GroupByStrategyV2#isCacheable to return true?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org