[GitHub] [iceberg] openinx commented on issue #1286: Slow parallel operations fail to commit

[GitBox <gi...@apache.org>]

openinx commented on issue #1286:
URL: https://github.com/apache/iceberg/issues/1286#issuecomment-668018068


   > continuously appends the table with trigger period 10s.
   
   You mean the spark streaming job will commit to iceberg table for every 10 seconds ?  If sure, then it seems  we committed iceberg too frequently and it's easy to cause transaction conflicts.
   Or just write to the data file without committing the transaction and after some another interval to commit the transaction.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org