[GitHub] [spark] LuciferYang commented on a diff in pull request #40510: [SPARK-42889][CONNECT][PYTHON] Implement cache, persist, unpersist, and storageLevel

["LuciferYang (via GitHub)" <gi...@apache.org>]

LuciferYang commented on code in PR #40510:
URL: https://github.com/apache/spark/pull/40510#discussion_r1144269110


##########
connector/connect/common/src/main/protobuf/spark/connect/base.proto:
##########
@@ -54,6 +54,20 @@ message UserContext {
   repeated google.protobuf.Any extensions = 999;
 }
 
+// StorageLevel for persisting Datasets/Tables.
+message StorageLevel {

Review Comment:
   friendly ping @ueshin @HyukjinKwon 
   
   When I try to import `import "spark/connect/base.proto";` in `catalog.proto` to reuse `StorageLevel` for `message CacheTable`, I found the following compile messages:
   
   ```
   spark/connect/base.proto:23:1: File recursively imports itself: spark/connect/base.proto -> spark/connect/commands.proto -> spark/connect/relations.proto -> spark/connect/catalog.proto -> spark/connect/base.proto
   spark/connect/catalog.proto:22:1: Import "spark/connect/base.proto" was not found or had errors.
   spark/connect/catalog.proto:144:12: "spark.connect.DataType" seems to be defined in "spark/connect/types.proto", which is not imported by "spark/connect/catalog.proto".  To use it here, please add the necessary import.
   spark/connect/catalog.proto:161:12: "spark.connect.DataType" seems to be defined in "spark/connect/types.proto", which is not imported by "spark/connect/catalog.proto".  To use it here, please add the necessary import.
   spark/connect/relations.proto:25:1: Import "spark/connect/catalog.proto" was not found or had errors.
   spark/connect/relations.proto:84:5: "Catalog" is not defined.
   spark/connect/commands.proto:22:1: Import "spark/connect/relations.proto" was not found or had errors.
   spark/connect/commands.proto:63:3: "Relation" is not defined.
   spark/connect/commands.proto:81:3: "Relation" is not defined.
   spark/connect/commands.proto:142:3: "Relation" is not defined.
   spark/connect/base.proto:23:1: Import "spark/connect/commands.proto" was not found or had errors.
   spark/connect/base.proto:25:1: Import "spark/connect/relations.proto" was not found or had errors.
   ....
   ```
   
   So should we move `StorageLevel` to a more suitable place?
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org