You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by GitBox <gi...@apache.org> on 2020/08/09 21:02:18 UTC

[GitHub] [parquet-mr] qinghui-xu commented on a change in pull request #561: PARQUET-1455: [parquet-protobuf] Handle protobuf enum schema evolution and unknown enum value

qinghui-xu commented on a change in pull request #561:
URL: https://github.com/apache/parquet-mr/pull/561#discussion_r467628479



##########
File path: parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoWriteSupport.java
##########
@@ -130,13 +137,36 @@ public WriteContext init(Configuration configuration) {
 
     this.messageWriter = new MessageWriter(messageDescriptor, rootSchema);
 
-    Map<String, String> extraMetaData = new HashMap<String, String>();
+    Map<String, String> extraMetaData = new HashMap<>();
     extraMetaData.put(ProtoReadSupport.PB_CLASS, protoMessage.getName());
     extraMetaData.put(ProtoReadSupport.PB_DESCRIPTOR, serializeDescriptor(protoMessage));
     extraMetaData.put(PB_SPECS_COMPLIANT_WRITE, String.valueOf(writeSpecsCompliant));
     return new WriteContext(rootSchema, extraMetaData);
   }
 
+  @Override
+  public FinalizedWriteContext finalizeWrite() {
+    Map<String, String> protoMetadata = new HashMap<>();
+    for (Map.Entry<String, Map<String, Integer>> enumNameNumberMapping : protoEnumBookKeeper.entrySet()) {
+      StringBuilder nameNumberPairs = new StringBuilder();
+      if (enumNameNumberMapping.getValue().isEmpty()) {
+        // No enum is ever written to any column of this file, put an empty string as the value in the metadata
+        LOG.info("No enum is written for " + enumNameNumberMapping.getKey());
+      }
+      int idx = 0;
+      for (Map.Entry<String, Integer> nameNumberPair : enumNameNumberMapping.getValue().entrySet()) {

Review comment:
       Sorry for the late reply. @costimuraru
   I refactor out all the logic of building enum metadata in a method. I think this would be more clear.
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org