You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "oakad (via GitHub)" <gi...@apache.org> on 2026/05/03 11:08:32 UTC
[I] Error when attempting a table append to table in S3 table bucket [iceberg-go]
oakad opened a new issue, #974:
URL: https://github.com/apache/iceberg-go/issues/974
### Apache Iceberg version
main (development)
### Please describe the bug 🐞
I am trying to write some data into an S3 table bucket. It is failing with error:
> panic: blob (key "data/00000-0-03c029fe-c451-4409-9a77-f0439c451793-00001.parquet") (code=NotFound): operation error S3: PutObject, https response error StatusCode: 301, RequestID: A6B9E1W9Z7YMPBKD, HostID: /Y0pd6gme9LdZOvjoLdMQvFOy0MSHC1pUBnwmfw2CluDKKuws6r31QzocJ76PuNEUP4hC0dNvP8DzwQ+rgt54kQDEri0A5sg, api error PermanentRedirect: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
In general, rest client is able to connect to table buckets, list tables, obtain schema, and so on. However, append always breaks, and error lacks sufficient information for me to trace it to the origin.
The callsite on my side looks as following (`check` helper simply panics on ```err != nil```):
```
func (wk *worker) sendBatch(batch arrow.RecordBatch) {
defer batch.Release()
rd := check(array.NewRecordReader(wk.app.schema, []arrow.RecordBatch {
batch,
}))
defer rd.Release()
wk.iceTab = check(wk.iceTab.Append(context.Background(), rd, nil))
}
```
More generally, I am trying to test #830
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Error when attempting a table append to table in S3 table bucket [iceberg-go]
Posted by "tanmayrauth (via GitHub)" <gi...@apache.org>.
tanmayrauth commented on issue #974:
URL: https://github.com/apache/iceberg-go/issues/974#issuecomment-4366645262
Thanks for catching this @oakad! The S3 data client wasn't inheriting the region from the REST catalog's SigV4 config, causing the redirect. Fixed in in this [PR](https://github.com/apache/iceberg-go/pull/975)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Error when attempting a table append to table in S3 table bucket [iceberg-go]
Posted by "oakad (via GitHub)" <gi...@apache.org>.
oakad commented on issue #974:
URL: https://github.com/apache/iceberg-go/issues/974#issuecomment-4369059057
Thank you for a prompt assistance.
On a related note, to access the catalog, I had to use the following options:
```
rest.WithSigV4RegionSvc("us-east-1", "s3tables"),
rest.WithAwsConfig(cfg),
```
Will it make sense to provide a canned option for s3 table bucket access, so that users don't have to figure out the sigv4 service name "s3tables" and don't have to specify the region explicitly on top of the AWS config?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Error when attempting a table append to table in S3 table bucket [iceberg-go]
Posted by "laskoviymishka (via GitHub)" <gi...@apache.org>.
laskoviymishka closed issue #974: Error when attempting a table append to table in S3 table bucket
URL: https://github.com/apache/iceberg-go/issues/974
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Error when attempting a table append to table in S3 table bucket [iceberg-go]
Posted by "oakad (via GitHub)" <gi...@apache.org>.
oakad commented on issue #974:
URL: https://github.com/apache/iceberg-go/issues/974#issuecomment-4366031210
I think, the present code always assumes a "normal" S3 bucket. Table buckets have a bunch of quirks: https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-tables-buckets.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org