[GitHub] [druid] kfaraz commented on pull request #11848: Add support for multi dimension range partitioning

[GitBox <gi...@apache.org>]

kfaraz commented on pull request #11848:
URL: https://github.com/apache/druid/pull/11848#issuecomment-960545965


   >     1. If someone updates to the new version, but doesn't use the new feature (i.e. they keep submitting ingest specs with `single_dim` shard specs), will that lead to segments getting written with `single_dim` or `multi_dim` shard specs? Hopefully it's `single_dim`, since that means people can safely roll back without losing the ability to load their existing segments.
   
   Thanks for pointing this out, @gianm . I had originally intended to add this logic to ensure backward compatibility but I had missed it.
   I have added it now by making `SingleDimensionShardSpec` extend `MultiDimensionShardSpec`.
   Thus, while converting a `BuildingMultiDimensionShardSpec`, a `SingleDimensionShardSpec` is created if there is only one dimension, otherwise a `MultiDimensionShardSpec` is created.
   
   ```java
   @Override
     public MultiDimensionShardSpec convert(int numCorePartitions)
     {
       return dimensions != null && dimensions.size() == 1
              ? new SingleDimensionShardSpec(...) 
              : new MultiDimensionShardSpec(...);
     }
   ```
   
   The only drawback is that if someone tried to partition their data using type `multi_dim` with a single dimension specified in the `partitionDimensions` field, the shard specs created would still have type `single_dim`. But I guess that is to be expected.
   
   >     2. If someone is in the middle of a rolling update to the new version, and has some servers updated but not others, will that be safe? Or will some servers write out `multi_dim` shard specs or other `multi_dim` things that other servers on the older version cannot read?
   
   The above mentioned approach will take care of this as well, because servers will __output `multi_dim` shard specs__ only if a user explicitly chooses `multi_dim` partitioning __with more than one dimensions__.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org