[GitHub] [hudi] BenjMaq commented on issue #4230: [SUPPORT] org.apache.hudi.exception.HoodieRemoteException: Failed to create marker file

[GitBox <gi...@apache.org>]

BenjMaq commented on issue #4230:
URL: https://github.com/apache/hudi/issues/4230#issuecomment-1054197087


   I was wondering if someone could help me understand how to set `set hoodie.write.markers.type=direct;` when launching a `spark-sql` session?
   
   I know I can run it manually, but I would love to be able to pass it as a `--conf` parameter, like we do for `--conf 'spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension'`.
   
   I tried
   ```
   spark-sql --packages org.apache.hudi:hudi-spark-bundle_2.11:0.10.1,org.apache.spark:spark-avro_2.11:2.4.4 \
   --conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer' \
   --conf 'spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension'
   --conf 'hoodie.write.markers.type=direct'
   ```
   but it does not work. Would anyone be able to help?
   
   Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org