[GitHub] [druid] maytasm edited a comment on pull request #11553: Fix ingestion task failure when no input split to process

[GitBox <gi...@apache.org>]

maytasm edited a comment on pull request #11553:
URL: https://github.com/apache/druid/pull/11553#issuecomment-895290411


   > > To name a few cases:
   > > 
   > > * Reindex with datasource name that doesn’t exist
   > > * Reindex with interval that does not contain any data
   > > * Ingestion with interval in granularitySpec mismatching the inputSpec
   > 
   > The first 2 cases sound like a user doing something they shouldn't be doing - and the re-index is a no-op. Maybe we should surface this as a typo to the end user?
   > 
   > Can you provide an example of the 3rd case?
   > 
   > I'm wondering if all these cases are situations where the end user should be warned of something "unexpected" happening.
   
   For the 3rd case, imagine you have a hash partitioning with the following spec:
   ```
      ...
       "ioConfig": {
         "type": "index_parallel",
         "inputSource": {
           "type": "druid",
           "dataSource": "mydatasource",
           "interval": "2020-07-31T00:00:00.000Z/2020-08-01T00:00:00.000Z",
           "filter": null,
           "dimensions": null,
           "metrics": [
             ...
           ]
         },
         "inputFormat": null,
         "appendToExisting": false
       },
       ....
       "granularitySpec": {
           "type": "uniform",
           "segmentGranularity": "DAY",
           "queryGranularity": "HOUR",
           "rollup": true,
           "intervals": [
             "2020-08-01T00:00:00.000Z/2020-08-02T00:00:00.000Z"
           ]
         },
       ....
       "partitionsSpec": {
           "type": "hashed",
           "numShards": null,
           "partitionDimensions": [],
           "partitionFunction": "murmur3_32_abs",
           "maxRowsPerSegment": 5000000
         },
       ....
       ```
   
   The partial_segment_merge phase tasks will have no input split to process as the interval in the ioConfig does not match with intervals in granularitySpec


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org