[GitHub] [hudi] jtmzheng commented on issue #7829: [SUPPORT] Using monotonically_increasing_id to generate record key causing duplicates on upsert

["jtmzheng (via GitHub)" <gi...@apache.org>]

jtmzheng commented on issue #7829:
URL: https://github.com/apache/hudi/issues/7829#issuecomment-1450922768

   Added some comments inline for clarification below
   
   ```
   write_options = {
               "hoodie.table.name": name,
               "hoodie.datasource.write.table.name": name,
               "hoodie.datasource.write.operation": "upsert",
               "hoodie.datasource.write.table.type": table_type.value, # always COW for these tables
               "hoodie.datasource.write.partitionpath.field": "",
               "hoodie.datasource.write.recordkey.field": ",".join(primary_keys), # `_autogenerated_primary_key` if none specified
               "hoodie.datasource.write.precombine.field": precombine_field, # `_autogenerated_primary_key` if none specified
               "hoodie.datasource.write.keygenerator.class": "org.apache.hudi.keygen.CustomKeyGenerator",
               "hoodie.datasource.write.hive_style_partitioning": "true",
               "hoodie.metadata.enable": "false",
               "hoodie.bootstrap.index.enable": "false",
               # Metrics configs
               "hoodie.metrics.on": "true",
               "hoodie.metrics.reporter.type": "GRAPHITE",
               "hoodie.metrics.graphite.host": {graphite_host},
               "hoodie.metrics.graphite.port": "{graphite_port}",
               "hoodie.metrics.graphite.metric.prefix": f"lake_{environment.value}",
           }
   
           if partition_cols:
               write_options["hoodie.datasource.write.partitionpath.field"] = ",".join(
                   f"{c}:SIMPLE" for c in partition_cols
               )
               write_options["hoodie.index.type"] = "BLOOM" # use default of SIMPLE on non-partitioned
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org