[GitHub] [spark] ueshin commented on a diff in pull request #40498: [SPARK-42878][CONNECT] The table API in DataFrameReader could also accept options

["ueshin (via GitHub)" <gi...@apache.org>]

ueshin commented on code in PR #40498:
URL: https://github.com/apache/spark/pull/40498#discussion_r1144066373


##########
connector/connect/common/src/main/protobuf/spark/connect/relations.proto:
##########
@@ -122,6 +122,9 @@ message Read {
   message NamedTable {
     // (Required) Unparsed identifier for the table.
     string unparsed_identifier = 1;
+
+    // Options for the named table. The map key is case insensitive.
+    map<string, string> options = 3;

Review Comment:
   nit: Why is this id `3`? seems like `2` is still available?



##########
python/pyspark/sql/connect/plan.py:
##########
@@ -302,13 +302,16 @@ def plan(self, session: "SparkConnectClient") -> proto.Relation:
 
 
 class Read(LogicalPlan):
-    def __init__(self, table_name: str) -> None:
+    def __init__(self, table_name: str, options: Dict[str, str] = {}) -> None:

Review Comment:
   Please avoid `{}` as a default argument, which could cause unexpected behaviors.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org