You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "dicarcab (via GitHub)" <gi...@apache.org> on 2025/01/08 12:54:12 UTC
[I] Quick start guide missing configuration [polaris]
dicarcab opened a new issue, #625:
URL: https://github.com/apache/polaris/issues/625
### Describe the bug
Hello, I've trying to test Polaris locally using the official Docker Image and a local gradle build, but have and issue while writing data to a catalog using S3.
I followed the Quick start guide and everything works fine until I tried to insert data to the table but I kept getting this answer
`Server error: S3Exception: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint. (Service: S3, Status Code: 301,)` is there any configuration that must be specified to the docker image to be able to write data to the desired location ?
I set up the aws access and secret key and my bucket is in the same region that is used in the docker compose image. Is there missing steps in the official documentation ?
### To Reproduce
1. Follow the quick start guide create roles and catalog
2. start local spark 3.5
3. Insert data using spark SQL
### Actual Behavior
error returned by the rest api `Server error: S3Exception: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint. (Service: S3, Status Code: 301,)` is there any configuration that must be specified to the docker image to be able to write data to the desired location ?
### Expected Behavior
Data should be written in the table
### Additional context
_No response_
### System information
WSL ubuntu, S3 as backend
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Quick start guide missing configuration [polaris]
Posted by "adutra (via GitHub)" <gi...@apache.org>.
adutra closed issue #625: Quick start guide missing configuration
URL: https://github.com/apache/polaris/issues/625
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Quick start guide missing configuration [polaris]
Posted by "snazy (via GitHub)" <gi...@apache.org>.
snazy commented on issue #625:
URL: https://github.com/apache/polaris/issues/625#issuecomment-2589731075
@hangc0276 this is the issue tracker for Apache Polaris. Support for any commercial product has to go to the respective vendor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Quick start guide missing configuration [polaris]
Posted by "dicarcab (via GitHub)" <gi...@apache.org>.
dicarcab commented on issue #625:
URL: https://github.com/apache/polaris/issues/625#issuecomment-2580322478
@MonkeyCanCode I updated the steps to reproduce the issue. The bucket is located in us-west-2 region as the default value inside the docker compose file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Quick start guide missing configuration [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on issue #625:
URL: https://github.com/apache/polaris/issues/625#issuecomment-2579155877
@dicarcab I think you will need to set AWS region as well depends on your setup. Maybe provide a bit more info on how to reproduce this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Quick start guide missing configuration [polaris]
Posted by "hangc0276 (via GitHub)" <gi...@apache.org>.
hangc0276 commented on issue #625:
URL: https://github.com/apache/polaris/issues/625#issuecomment-2588833292
I encountered the same issue.
- My iceberg client runs in the us-west-1,
- The target bucket is located on the us-west-1
- The Snowflake open catalog is located on us-west-2
When I use the Iceberg client to create an Iceberg table, it throws the following exception
```
org.apache.iceberg.exceptions.ServiceFailureException: Server error: S3Exception: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint. (Service: S3, Status Code: 301,
```
Do I need to deploy the Snowflake open catalog in the same region as the target bucket?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Quick start guide missing configuration [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on issue #625:
URL: https://github.com/apache/polaris/issues/625#issuecomment-2585843537
So this is actually due to missing region set on spark side instead of polaris side. Here is how to reproduce this:
Having the same issue:
```
scala> spark.sql("INSERT INTO quickstart_table VALUES (1, 'some data')")
...
software.amazon.awssdk.core.exception.SdkClientException: Unable to load region from any of the providers in the chain software.amazon.awssdk.regions.providers.DefaultAwsRegionProviderChain@1315b5e8: [software.amazon.awssdk.regions.providers.SystemSettingsRegionProvider@350a6bf2: Unable to load region from system settings. Region must be specified either via environment variable (AWS_REGION) or system property (aws.region)., software.amazon.awssdk.regions.providers.AwsProfileRegionProvider@54b391ae: No region provided in profile: default, software.amazon.awssdk.regions.providers.InstanceProfileRegionProvider@2be11aa0: Unable to contact EC2 metadata service.]
```
One way to fix this is set region on the client side (spark) via global variable but this can be problematic when distributed compute kicked in as worker nodes won't have this set:
```
➜ spark-3.5.4-bin-hadoop3 export AWS_REGION=us-west-2
...
scala> spark.sql("INSERT INTO quickstart_table VALUES (1, 'some data')")
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
res7: org.apache.spark.sql.DataFrame = []
scala> spark.sql("select * from quickstart_table limit 10").show()
+---+---------+
| id| data|
+---+---------+
| 1|some data|
+---+---------+
```
Thus, it is better to set via catalog property such as following:
```
--conf spark.sql.catalog.quickstart_catalog.client.region=us-west-2
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Quick start guide missing configuration [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on issue #625:
URL: https://github.com/apache/polaris/issues/625#issuecomment-2581606128
> @MonkeyCanCode I updated the steps to reproduce the issue. The bucket is located in us-west-2 region as the default value inside the docker compose file.
K. I will take a look later this weekend then update. I recalled this part used to work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org