You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/11/11 15:32:13 UTC
[GitHub] [druid] cchanyi opened a new issue #11910: Reindex from Druid Datasource gives 0 values for metrics Quantiles and HLL
cchanyi opened a new issue #11910:
URL: https://github.com/apache/druid/issues/11910
This is similar to https://github.com/apache/druid/issues/9914 and others related. The workaround does not work for Quantiles and HLL in 0.21.1.
### Affected Version
0.21.1
### Description
Using a very simple example in the console. The goal is to take a dimensional column from the source table and create a new datasource that aggregates that column into a Quantile sketch if it's a number or HLL Sketch if it's a string. I can sum the number column as a metric, but the Quantiles or HLL are empty.
The workaround listed is actually populated now from the Load Data in console. The fix was to include the dimensions from the source datasource as metrics in the inputSource. If I have a dimension called `dim_double` - I can sum that dimension, but cannot quantiles from it. With the string dimension called `dim_string` - I cannot get HLL sketch from this.
```
"ioConfig": {
"type": "index_parallel",
"inputSource": {
"type": "druid",
"dataSource": "source_datasource",
"interval": "2021-08-18/2021-08-19",
"metrics": [
"count",
"dim_double",
"dim_string"
]
},
"appendToExisting": false
},
```
Then, the metricSpec looks like:
```
"metricsSpec": [
{
"type": "longSum",
"name": "count",
"fieldName": "count",
"expression": null
},
{
"name": "hll_dim_string",
"type": "HLLSketchBuild",
"fieldName": "dim_string"
},
{
"fieldName": "dim_double",
"type": "doubleSum",
"name": "sum_dim_double"
},
{
"name": "quantiles_doubles_dim_double",
"type": "quantilesDoublesSketch",
"fieldName": "dim_double"
}
],
```
I have not tested on earlier versions. This seems like a common use-case. Note, if I build the spec from the source data it does work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] cchanyi commented on issue #11910: Reindex from Druid Datasource gives 0 values for metrics Quantiles and HLL
Posted by GitBox <gi...@apache.org>.
cchanyi commented on issue #11910:
URL: https://github.com/apache/druid/issues/11910#issuecomment-986964617
This is not a bug. I can get this working on 0.21.1 and 0.22. I've run into some other bugs I will log - this caused some issues with the real data that contributed to this not working. Once worked around this worked. I will log those other issues, but this can be closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] cchanyi commented on issue #11910: Reindex from Druid Datasource gives 0 values for metrics Quantiles and HLL
Posted by GitBox <gi...@apache.org>.
cchanyi commented on issue #11910:
URL: https://github.com/apache/druid/issues/11910#issuecomment-968305281
I tested on newest Druid version and it works. I had to use a different dataset though, so not positive if this is related to the data. But with `0.22.0` I was able to ingest a small table of dimensions, then create a new druid source and add in both HLL and Quantiles sketches with proper metrics. I going to try the same source on 0.21.1 just to compare.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] cchanyi commented on issue #11910: Reindex from Druid Datasource gives 0 values for metrics Quantiles and HLL
Posted by GitBox <gi...@apache.org>.
cchanyi commented on issue #11910:
URL: https://github.com/apache/druid/issues/11910#issuecomment-970628443
So my test case works on `0.21.1` - just not with the real table. Trying to see the difference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] cchanyi closed issue #11910: Reindex from Druid Datasource gives 0 values for metrics Quantiles and HLL
Posted by GitBox <gi...@apache.org>.
cchanyi closed issue #11910:
URL: https://github.com/apache/druid/issues/11910
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org