[GitHub] [druid] techdocsmith commented on a change in pull request #11574: Configurable maxStreamLength for doubles sketches

[GitBox <gi...@apache.org>]

techdocsmith commented on a change in pull request #11574:
URL: https://github.com/apache/druid/pull/11574#discussion_r695936382



##########
File path: docs/development/extensions-core/datasketches-quantiles.md
##########
@@ -56,6 +56,7 @@ The result of the aggregation is a DoublesSketch that is the union of all sketch
 |name|A String for the output (result) name of the calculation.|yes|
 |fieldName|A String for the name of the input field (can contain sketches or raw numeric values).|yes|
 |k|Parameter that determines the accuracy and size of the sketch. Higher k means higher accuracy but more space to store sketches. Must be a power of 2 from 2 to 32768. See [accuracy information](https://datasketches.apache.org/docs/Quantiles/OrigQuantilesSketch) in the DataSketches documentation for details.|no, defaults to 128|
+|maxStreamLength|The max number of items that can be stored in each sketch. Currently, the query can throw `IllegalStateException` when one of the sketch hits this limit. See the [GitHub issue](https://github.com/apache/druid/issues/11544) for more details of the bug. A known workaround is using a higher max stream length. See [accuracy information](https://datasketches.apache.org/docs/Quantiles/OrigQuantilesSketch) in the DataSketches documentation for how many bytes are required per stream length. Since this parameter is added as a temporary solution to avoid the known issue, it can be removed in a future release when the bug is fixed.|no, defaults to 1,000,000,000|

Review comment:
       ```suggestion
   |maxStreamLength|The maximum number of items to store in each sketch. If one of the sketches hits this limit, the query can throw an `IllegalStateException`. See the [GitHub issue 11544](https://github.com/apache/druid/issues/11544) for more details. To workaround this issue, increase the max stream length. See [accuracy information](https://datasketches.apache.org/docs/Quantiles/OrigQuantilesSketch) in the DataSketches documentation for how many bytes are required per stream length. This parameter is added as a temporary solution to avoid the known issue. It may be removed in a future release after the bug is fixed.|no, defaults to 1,000,000,000|
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org