[GitHub] [incubator-pinot] xiaohui-sun commented on a change in pull request #5184: [TE] Validate the number of data points to retrieve while fetching timeseries

[GitBox <gi...@apache.org>]

xiaohui-sun commented on a change in pull request #5184: [TE] Validate the number of data points to retrieve while fetching timeseries
URL: https://github.com/apache/incubator-pinot/pull/5184#discussion_r398661229
 
 

 ##########
 File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dataframe/util/DataFrameUtils.java
 ##########
 @@ -493,14 +518,14 @@ public long apply(long... values) {
    * Helper: Returns a pre-populated ThirdeyeRequestBuilder instance. Removes invalid filter values.
    *
    * @param slice metric data slice
-   * @param metric metric dto
    * @param dataset dataset dto
    * @param expressions metric expressions
-   * @param metricDAO metric config DAO
    * @return ThirdeyeRequestBuilder
    * @throws ExecutionException
    */
-  private static ThirdEyeRequest.ThirdEyeRequestBuilder makeThirdEyeRequestBuilder(MetricSlice slice, MetricConfigDTO metric, DatasetConfigDTO dataset, List<MetricExpression> expressions, MetricConfigManager metricDAO) throws ExecutionException {
+  private static ThirdEyeRequest.ThirdEyeRequestBuilder makeThirdEyeRequestBuilder(MetricSlice slice, DatasetConfigDTO dataset, List<MetricExpression> expressions) throws ExecutionException {
+    validateTimeSeriesSize(dataset, slice.getStart(), slice.getEnd(), slice.getGranularity());
 
 Review comment:
   1. I think we can have this validation earlier when parsing YAML. 
   2. Suggest to split the method into two - estimateTimeSeriesSize and validateTimeSeriesSize.
   3. Can we add a limit when retrieving data? Like using LIMIT 10000. I remember we have this for Pinot.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org