You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/09/20 20:36:36 UTC

[GitHub] [spark] sunchao commented on a diff in pull request #37952: [SPARK-40508][SQL] Treat unknown partitioning as UnknownPartitioning

sunchao commented on code in PR #37952:
URL: https://github.com/apache/spark/pull/37952#discussion_r975791836


##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanPartitioningAndOrdering.scala:
##########
@@ -54,8 +55,9 @@ object V2ScanPartitioningAndOrdering extends Rule[LogicalPlan] with SQLConfHelpe
             }
           }
         case _: UnknownPartitioning => None
-        case p => throw new IllegalArgumentException("Unsupported data source V2 partitioning " +
-            "type: " + p.getClass.getSimpleName)
+        case p =>
+          logWarning("Spark ignores the partitioning. Please use KeyGroupedPartitioning for better performance")

Review Comment:
   nit: could we also log the class name here? e.g.,
   ```
   Spark ignores partitioning ${p.getClass.getSimpleName}. Please use KeyGroupedPartitioning for better performance
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org