[GitHub] [spark] HyukjinKwon commented on a diff in pull request #41075: [SPARK-43361][CONNECT][PROTOBUF] spark-protobuf: allow serde with enum as ints

["HyukjinKwon (via GitHub)" <gi...@apache.org>]

HyukjinKwon commented on code in PR #41075:
URL: https://github.com/apache/spark/pull/41075#discussion_r1186958551


##########
connector/protobuf/src/main/scala/org/apache/spark/sql/protobuf/utils/ProtobufOptions.scala:
##########
@@ -141,6 +141,33 @@ private[sql] class ProtobufOptions(
   //      what information is available in a serialized proto.
   val emitDefaultValues: Boolean =
     parameters.getOrElse("emit.default.values", false.toString).toBoolean
+
+  // Whether to render enum fields as their integer values.
+  //
+  // As an example, consider the following message type:
+  // ```
+  // syntax = "proto3";
+  // message Person {
+  //   enum Job {
+  //     NONE = 0;
+  //     ENGINEER = 1;
+  //     DOCTOR = 2;
+  //   }
+  //   Job job = 1;
+  // }
+  // ```
+  //
+  // If we have an instance of this message like `Person(job = ENGINEER)`, then the
+  // default deserialization will be:
+  // `{"job": "ENGINEER"}`
+  //
+  // But with this option set the deserialization will be:
+  // `{"job": 1}`
+  //
+  // Please note the output struct type will now contain an int column
+  // instead of string, so use caution if changing existing parsing logic.
+  val enumsAsInts: Boolean =
+    parameters.getOrElse("enums.as.ints", false.toString).toBoolean

Review Comment:
   Actually those options should better be documented at https://github.com/apache/spark/blob/master/docs/sql-data-sources-protobuf.md#deploying.
   
   Since they are already undocumented there, I am fine without doing that in this PR though.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org