Spark writing to table in iceberg

[Gaurav Agarwal <ga...@gmail.com>]

Hi

I have 12.5 m records with 19 parquet files and have 1700 columns
the table has one date partition and 16 bucket
I am reading the table with the date criteria and do the  left anti join
from the source dataset where we read parquet files
It gives the unmatched data to us did the union with the source data and
then perform overwrite partition

It is giving oome error to us in some executor how to find where it is
failing  and why it is failing for oome
We did the count on dataset with partition id it has uniform distribution
for 100 partitions

Need help to understand what is going on .