[GitHub] [hudi] BenjMaq edited a comment on issue #4230: [SUPPORT] org.apache.hudi.exception.HoodieRemoteException: Failed to create marker file

[GitBox <gi...@apache.org>]

BenjMaq edited a comment on issue #4230:
URL: https://github.com/apache/hudi/issues/4230#issuecomment-1054197087


   I was wondering if someone could help me understand how to set `hoodie.write.markers.type=direct` when launching a `spark-sql` session?
   
   I know I can run it manually `set hoodie.write.markers.type=direct;` before I run any SQL statement, but I would love to be able to pass it as a `--conf` parameter, like we do for `--conf 'spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension'`.
   
   I tried
   ```
   spark-sql --packages org.apache.hudi:hudi-spark-bundle_2.11:0.10.1,org.apache.spark:spark-avro_2.11:2.4.4 \
   --conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer' \
   --conf 'spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension'
   --conf 'hoodie.write.markers.type=direct'
   ```
   but it does not work. Would anyone be able to help?
   
   Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org