You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/06/03 14:14:53 UTC

[GitHub] [incubator-doris] spaces-X opened a new pull request #3768: Disable Bitmap or Hll type in keys or in values with incorrect agg-type

spaces-X opened a new pull request #3768:
URL: https://github.com/apache/incubator-doris/pull/3768


   Bitmap and Hll type can not be used with incorrect aggregate functions, which will cause to BE crush.
   Add some logical checks in FE's ColumnDef#analyze to avoid creating tables or changing schemas incorrectly.
   
   - Keys never be bitmap or hll type
   - values with bitmap or hll type have to be associated with bitmap_union or hll_union
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] morningman commented on a change in pull request #3768: Disable Bitmap or Hll type in keys or in values with incorrect agg-type

Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #3768:
URL: https://github.com/apache/incubator-doris/pull/3768#discussion_r435125457



##########
File path: fe/src/main/java/org/apache/doris/catalog/AggregateType.java
##########
@@ -85,20 +85,22 @@
         compatibilityMap.put(MAX, EnumSet.copyOf(primitiveTypeList));
 
         primitiveTypeList.clear();
-        compatibilityMap.put(REPLACE, EnumSet.allOf(PrimitiveType.class));
+        // all types except bitmap and hll.
+        EnumSet<PrimitiveType> exc_bitmap_hll = EnumSet.allOf(PrimitiveType.class);
+        exc_bitmap_hll.remove(PrimitiveType.BITMAP);
+        exc_bitmap_hll.remove(PrimitiveType.HLL);
+        compatibilityMap.put(REPLACE, EnumSet.copyOf(exc_bitmap_hll));
+
+        compatibilityMap.put(REPLACE_IF_NOT_NULL, EnumSet.copyOf(exc_bitmap_hll));
 
-        primitiveTypeList.clear();
-        compatibilityMap.put(REPLACE_IF_NOT_NULL, EnumSet.allOf(PrimitiveType.class));
-       
-        primitiveTypeList.clear();
         primitiveTypeList.add(PrimitiveType.HLL);
         compatibilityMap.put(HLL_UNION, EnumSet.copyOf(primitiveTypeList));
 
-        primitiveTypeList.clear();
+
         primitiveTypeList.add(PrimitiveType.BITMAP);

Review comment:
       Why remove the `primitiveTypeList.clear();`.
   Now, the `BITMAP_UNION` is compatible with both `BITMAP` and `HLL`?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] spaces-X commented on pull request #3768: Disable Bitmap or Hll type in keys or in values with incorrect agg-type

Posted by GitBox <gi...@apache.org>.
spaces-X commented on pull request #3768:
URL: https://github.com/apache/incubator-doris/pull/3768#issuecomment-638660609


   > Please add UT for this case.
   > You can reference to `CreateTableStmtTest.java`
   
   Done.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] spaces-X commented on pull request #3768: Disable Bitmap or Hll type in keys or in values with incorrect agg-type

Posted by GitBox <gi...@apache.org>.
spaces-X commented on pull request #3768:
URL: https://github.com/apache/incubator-doris/pull/3768#issuecomment-638236361


   [issue](https://github.com/apache/incubator-doris/issues/3769)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] spaces-X commented on pull request #3768: Disable Bitmap or Hll type in keys or in values with incorrect agg-type

Posted by GitBox <gi...@apache.org>.
spaces-X commented on pull request #3768:
URL: https://github.com/apache/incubator-doris/pull/3768#issuecomment-638659009


   [Fix #3769](https://github.com/apache/incubator-doris/issues/3769)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] morningman commented on pull request #3768: Disable Bitmap or Hll type in keys or in values with incorrect agg-type

Posted by GitBox <gi...@apache.org>.
morningman commented on pull request #3768:
URL: https://github.com/apache/incubator-doris/pull/3768#issuecomment-638564961


   > [issue](https://github.com/apache/incubator-doris/issues/3769)
   
   Link issue like this: `#3769`
   If you want to close the issue automatically after PR being merged, like this: `Fix #3769`


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] imay commented on a change in pull request #3768: Disable Bitmap or Hll type in keys or in values with incorrect agg-type

Posted by GitBox <gi...@apache.org>.
imay commented on a change in pull request #3768:
URL: https://github.com/apache/incubator-doris/pull/3768#discussion_r435656472



##########
File path: fe/src/main/java/org/apache/doris/analysis/ColumnDef.java
##########
@@ -131,6 +131,16 @@ public void analyze(boolean isOlap) throws AnalysisException {
 
         Type type = typeDef.getType();
 
+        // disable Bitmap Hll type in keys, values without aggregate function.
+        if ( type.isBitmapType() || type.isHllType() ){
+            if (isKey){

Review comment:
       ```suggestion
               if (isKey) {
   ```

##########
File path: fe/src/main/java/org/apache/doris/analysis/ColumnDef.java
##########
@@ -131,6 +131,16 @@ public void analyze(boolean isOlap) throws AnalysisException {
 
         Type type = typeDef.getType();
 
+        // disable Bitmap Hll type in keys, values without aggregate function.
+        if ( type.isBitmapType() || type.isHllType() ){

Review comment:
       Should keep code style with original code.
   ```suggestion
           if (type.isBitmapType() || type.isHllType()) {
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] spaces-X commented on a change in pull request #3768: Disable Bitmap or Hll type in keys or in values with incorrect agg-type

Posted by GitBox <gi...@apache.org>.
spaces-X commented on a change in pull request #3768:
URL: https://github.com/apache/incubator-doris/pull/3768#discussion_r435135824



##########
File path: fe/src/main/java/org/apache/doris/catalog/AggregateType.java
##########
@@ -85,20 +85,22 @@
         compatibilityMap.put(MAX, EnumSet.copyOf(primitiveTypeList));
 
         primitiveTypeList.clear();
-        compatibilityMap.put(REPLACE, EnumSet.allOf(PrimitiveType.class));
+        // all types except bitmap and hll.
+        EnumSet<PrimitiveType> exc_bitmap_hll = EnumSet.allOf(PrimitiveType.class);
+        exc_bitmap_hll.remove(PrimitiveType.BITMAP);
+        exc_bitmap_hll.remove(PrimitiveType.HLL);
+        compatibilityMap.put(REPLACE, EnumSet.copyOf(exc_bitmap_hll));
+
+        compatibilityMap.put(REPLACE_IF_NOT_NULL, EnumSet.copyOf(exc_bitmap_hll));
 
-        primitiveTypeList.clear();
-        compatibilityMap.put(REPLACE_IF_NOT_NULL, EnumSet.allOf(PrimitiveType.class));
-       
-        primitiveTypeList.clear();
         primitiveTypeList.add(PrimitiveType.HLL);
         compatibilityMap.put(HLL_UNION, EnumSet.copyOf(primitiveTypeList));
 
-        primitiveTypeList.clear();
+
         primitiveTypeList.add(PrimitiveType.BITMAP);

Review comment:
       By mistake.
   Thanks for reminding.
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] imay merged pull request #3768: Disable Bitmap or Hll type in keys or in values with incorrect agg-type

Posted by GitBox <gi...@apache.org>.
imay merged pull request #3768:
URL: https://github.com/apache/incubator-doris/pull/3768


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org