[GitHub] [spark] amaliujia commented on a diff in pull request #38979: [SPARK-41446][CONNECT][PYTHON] Make `createDataFrame` support schema and more input dataset types

[GitBox <gi...@apache.org>]

amaliujia commented on code in PR #38979:
URL: https://github.com/apache/spark/pull/38979#discussion_r1044122515


##########
connector/connect/common/src/main/protobuf/spark/connect/relations.proto:
##########
@@ -304,6 +305,24 @@ message LocalRelation {
   // Local collection data serialized into Arrow IPC streaming format which contains
   // the schema of the data.
   bytes data = 1;
+
+  // (Optional) The user provided schema.
+  //
+  // The Sever side will update the column names and data types according to this schema.
+  oneof schema {
+
+    DataType datatype = 2;
+
+    // Server will use Catalyst parser to parse this string to DataType.
+    string datatype_str = 3;
+
+    // Column names
+    StringList cols = 4;

Review Comment:
   I am a bit confused on this one: what would be the types for such column names? Are the types inferred but the names are provided? Can the behavior be documented here?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org