iceberg-go : how to update table metadata without reading data file

[ramesh ganesan <ra...@aqfer.com>]

Hi Team,

This question is specifically on the iceberg-go library.  
I have a use case where I want to write the data files (avro & parquet files) using my own libraries.

Once the data files are written to the destination, I want to update the iceberg table just with the metadata, using a transaction. 
My particular interest is in the partition metadata.

I have tried the Transaction.AddFiles method, it works for parquet, where the iceberg-go library extracts the partition values from parquet files and it automatically updates the metadata.
However I can not use this approach for avro files, which does not these partition specific details.

So what should be the approach for
1. Without using iceberg-go libary, first create avro file. During this process, gather necessary meta data including the partition meta data. 
2. Now using the iceberg-go library, update the iceberg table metadata using a transaction

Thanks,
Ramesh.

Re: iceberg-go : how to update table metadata without reading data file

[ramesh ganesan <ra...@aqfer.com>]

Hi,

Adding a clarification. 

In the below query, I am taking care of step 1 myself. 
I need help only on step 2 ie. updating the metadata using iceberg-go library.

thanks,
Ramesh.

> On Jul 23, 2025, at 2:39 PM, ramesh ganesan <ra...@aqfer.com> wrote:
> 
> Hi Team,
> 
> This question is specifically on the iceberg-go library.  
> I have a use case where I want to write the data files (avro & parquet files) using my own libraries.
> 
> Once the data files are written to the destination, I want to update the iceberg table just with the metadata, using a transaction. 
> My particular interest is in the partition metadata.
> 
> I have tried the Transaction.AddFiles method, it works for parquet, where the iceberg-go library extracts the partition values from parquet files and it automatically updates the metadata.
> However I can not use this approach for avro files, which does not these partition specific details.
> 
> So what should be the approach for
> 1. Without using iceberg-go libary, first create avro file. During this process, gather necessary meta data including the partition meta data. 
> 2. Now using the iceberg-go library, update the iceberg table metadata using a transaction
> 
> Thanks,
> Ramesh.
> 
> 
>