You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "Zevrap-81 (via GitHub)" <gi...@apache.org> on 2026/03/29 03:00:37 UTC
[I] Creating catalogs using the cli is buggy [polaris]
Zevrap-81 opened a new issue, #4076:
URL: https://github.com/apache/polaris/issues/4076
### Describe the bug
I am trying to use the cli command to create catalog using `polaris setup apply test-setup.yaml` command:
even though i set the sts_unavailable flag to true it still asks for ARN like so
```
apache_polaris/sdk/management/exceptions.py", line 161, in from_response
raise BadRequestException(http_resp=http_resp, body=body, data=data)
apache_polaris.sdk.management.exceptions.BadRequestException: (400)
Reason: Bad Request
HTTP response headers: HTTPHeaderDict({'content-length': '90', 'Content-Type': 'application/json', 'X-Request-ID': '819c92cb-c6d7-4fda-8093-82c2a352fdf9_0000000000000000251'})
HTTP response body: {"error":{"message":"ARN must not be empty","type":"IllegalArgumentException","code":400}}
```
when i provide some dummy ARN value then the catalog gets created and also the namespaces but when creating tables i get the following error
```
IllegalArgumentException: Credential vending was requested for table Senec.Stage.Bronze.Ampace_V3LFP.BmsAmpaceV3ModuleMeasurement, but no credentials are available
```
however, using the `polaris catalogs create` commands works as expected.
### To Reproduce
The minimal test-setup.yaml file is as follows:
```
catalogs:
- name: "senec_catalog_6"
storage_type: "s3"
type: internal
default_base_location: "s3://senec-bucket/senec_catalog_6/"
allowed_locations:
- "s3://senec-bucket/senec_catalog_6/"
region: "us-east-1"
# Custom S3 endpoint.
endpoint: "http://localhost:9000/"
endpoint_internal: "http://rustfs:9000"
path_style_access: true
# sts_unavailable: true
# kms_unavailable: true
properties:
catalog: senec_catalog_6
```
my yaml file has a bit more things like setting up the principals, and other entities but creating catalogs is what causing the issue.
and the exact command that i used is
`polaris --profile root setup apply ./test-setup.yaml`
### Actual Behavior
_No response_
### Expected Behavior
_No response_
### Additional context
I am trying to use Polaris REST catalog along with PyIceberg engine and RustFS for storage. I am still new to this so I might be doing something wrong. I am happy to share more info as required.
### System information
I am running ubuntu and i have latest polaris version from main.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "Zevrap-81 (via GitHub)" <gi...@apache.org>.
Zevrap-81 commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4183141733
> > "stsEndpoint": "",
> > "stsUnavailable": false,
>
> These two settings combined do not look correct to me.
Even when i set the stsUnavailable property i would get the same error. I will share the catalog settings on monday with this property turned on
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4167355459
@Zevrap-81 so the issue is due to missing role-arn as you pointed out above (the 400 bad request). A simple example you can find is https://github.com/apache/polaris/blob/main/site/content/guides/rustfs/docker-compose.yml where it invokes [create-catalog.sh](https://github.com/apache/polaris/blob/main/site/content/guides/assets/polaris/create-catalog.sh) when using non-AWS S3-compatible storage (which is what I am assuming on what you are using based on the minimal config). As what @dimas-b mentioned above, when sts/kms are not supported by the S3-compatible storage you are using, those would need to be set on the catalog property (as what you have set above but have it uncommented). One easy way to allow us debug further is show us the catalog you currently have by using following:
```
./polaris --profile root catalogs list
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4177617289
> > [@Zevrap-81](https://github.com/Zevrap-81) so the issue is due to missing role-arn as you pointed out above (the 400 bad request). A simple example you can find is https://github.com/apache/polaris/blob/main/site/content/guides/rustfs/docker-compose.yml where it invokes [create-catalog.sh](https://github.com/apache/polaris/blob/main/site/content/guides/assets/polaris/create-catalog.sh) when using non-AWS S3-compatible storage (which is what I am assuming on what you are using based on the minimal config). As what [@dimas-b](https://github.com/dimas-b) mentioned above, when sts/kms are not supported by the S3-compatible storage you are using, those would need to be set on the catalog property (as what you have set above but have it uncommented). One easy way to allow us debug further is show us the catalog you currently have by using following:
> >
> > ```
> > ./polaris --profile root catalogs list
> > ```
>
> here's the output to this:
> ```
> [
> {
> "type": "INTERNAL",
> "name": "senec_catalog_6",
> "properties": {
> "default-base-location": "s3://senec-bucket/senec_catalog_6/",
> "catalog": "senec_catalog_6"
> },
> "createTimestamp": 1775113025345,
> "lastUpdateTimestamp": 1775113025345,
> "entityVersion": 1,
> "storageConfigInfo": {
> "storageType": "S3",
> "allowedLocations": [
> "s3://senec-bucket/senec_catalog_6/"
> ],
> "roleArn": "arn:aws:iam::123456789012:role/data_ops_role_2",
> "externalId": "",
> "region": "us-east-1",
> "endpoint": "http://localhost:9000/",
> "stsEndpoint": "",
> "stsUnavailable": false,
> "endpointInternal": "http://rustfs:9000",
> "pathStyleAccess": true
> }
> },
> {
> "type": "INTERNAL",
> "name": "senec_catalog",
> "properties": {
> "default-base-location": "s3://senec-bucket/senec_catalog/"
> },
> "createTimestamp": 1775113285322,
> "lastUpdateTimestamp": 1775113285322,
> "entityVersion": 1,
> "storageConfigInfo": {
> "storageType": "S3",
> "allowedLocations": [
> "s3://senec-bucket/senec_catalog/"
> ],
> "endpoint": "http://localhost:9000",
> "endpointInternal": "http://rustfs:9000",
> "pathStyleAccess": true
> }
> }
> ]
> ```
> One thing that stands out to me is ` "externalId": "",`. Is this the problem?
I think the problem is sts unavailable is not set.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4172536760
@Zevrap-81 : It does not matter how a catalog was created. What matters in the settings in the catalog. Please provide catalog information (for both cases) using the command from @MonkeyCanCode 's message.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4172539842
> @MonkeyCanCode yes i understand that as i already mentioned the create catalog command works, however, when i try to do the same using the setup command and a yaml file (because it’s convenient and everything in one place), it no longer works. Do you have an example of setup command for non S3 compatible storage like minio?
So setup should work the same way as running individual CLI commands. I don't have one with MinIO but it should be straightforward. I can look into this a bit later this week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "Zevrap-81 (via GitHub)" <gi...@apache.org>.
Zevrap-81 commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4163574010
Edit:
The following is the yaml file that i used to get this error since if i did not pass in the role_arn it would not work already raising an error that S3 requires ARN
```
catalogs:
- name: "senec_catalog_6"
storage_type: "s3"
type: internal
default_base_location: "s3://senec-bucket/senec_catalog_6/"
allowed_locations:
- "s3://senec-bucket/senec_catalog_6/"
region: "us-east-1"
# Custom S3 endpoint.
endpoint: "http://localhost:9000/"
endpoint_internal: "http://rustfs:9000"
path_style_access: true
sts_unavailable: true
kms_unavailable: true
properties:
catalog: senec_catalog_6
# --- S3-Specific Properties ---
# The ARN of the IAM role Polaris assumes to access S3.
role_arn: "arn:aws:iam::123456789012:role/data_ops_role_2"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "Zevrap-81 (via GitHub)" <gi...@apache.org>.
Zevrap-81 commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4330780990
@MonkeyCanCode I am sorry for the late reply. I didn't have time to test it. But now i can confirm that it works as intended. Thank you very much!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4293022839
@Zevrap-81 we good to close this one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "Zevrap-81 (via GitHub)" <gi...@apache.org>.
Zevrap-81 commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4163547759
So the stack trace would be of the rest api that pyiceberg uses:
```
Failed to create table Senec.Stage.Bronze.Ampace_V3LFP.BmsAmpaceV3ModuleMeasurement: IllegalArgumentException: Credential vending was requested for table Senec.Stage.Bronze.Ampace_V3LFP.BmsAmpaceV3ModuleMeasurement, but no credentials are available
Traceback (most recent call last):
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/pyiceberg/catalog/rest/__init__.py", line 919, in load_table
response.raise_for_status()
~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/requests/models.py", line 1026, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://192.168.0.125:8181/api/catalog/v1/senec_catalog_6/namespaces/Senec%1FStage%1FBronze%1FAmpace_V3LFP/tables/BmsAmpaceV3ModuleMeasurement
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/novum_iceberg/iceberg_client.py", line 192, in _ensure_table_exists_from_config
self.catalog.load_table(full_table_name)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/tenacity/__init__.py", line 331, in wrapped_f
return copy(f, *args, **kw)
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/tenacity/__init__.py", line 470, in __call__
do = self.iter(retry_state=retry_state)
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/tenacity/__init__.py", line 371, in iter
result = action(retry_state)
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/tenacity/__init__.py", line 393, in <lambda>
self._add_action_func(lambda rs: rs.outcome.result())
~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/tenacity/__init__.py", line 473, in __call__
result = fn(*args, **kwargs)
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/pyiceberg/catalog/rest/__init__.py", line 921, in load_table
_handle_non_200_response(exc, {404: NoSuchTableError})
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/pyiceberg/catalog/rest/response.py", line 111, in _handle_non_200_response
raise exception(response) from exc
pyiceberg.exceptions.NoSuchTableError: NoSuchTableException: Table does not exist: Senec.Stage.Bronze.Ampace_V3LFP.BmsAmpaceV3ModuleMeasurement
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/pyiceberg/catalog/rest/__init__.py", line 812, in _create_table
response.raise_for_status()
~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/requests/models.py", line 1026, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://192.168.0.125:8181/api/catalog/v1/senec_catalog_6/namespaces/Senec%1FStage%1FBronze%1FAmpace_V3LFP/tables
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/novum_iceberg/iceberg_client.py", line 179, in initialize_from_yaml
self._ensure_table_exists_from_config(full_table_name, config)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/novum_iceberg/iceberg_client.py", line 205, in _ensure_table_exists_from_config
self.catalog.create_table(
~~~~~~~~~~~~~~~~~~~~~~~~~^
identifier=full_table_name, schema=schema, partition_spec=partition_spec, sort_order=sort_order
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/tenacity/__init__.py", line 331, in wrapped_f
return copy(f, *args, **kw)
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/tenacity/__init__.py", line 470, in __call__
do = self.iter(retry_state=retry_state)
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/tenacity/__init__.py", line 371, in iter
result = action(retry_state)
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/tenacity/__init__.py", line 393, in <lambda>
self._add_action_func(lambda rs: rs.outcome.result())
~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/tenacity/__init__.py", line 473, in __call__
result = fn(*args, **kwargs)
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/pyiceberg/catalog/rest/__init__.py", line 827, in create_table
table_response = self._create_table(
identifier=identifier,
...<5 lines>...
stage_create=False,
)
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/pyiceberg/catalog/rest/__init__.py", line 814, in _create_table
_handle_non_200_response(exc, {409: TableAlreadyExistsError, 404: NoSuchNamespaceError})
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.13/site-packages/pyiceberg/catalog/rest/response.py", line 111, in _handle_non_200_response
raise exception(response) from exc
pyiceberg.exceptions.BadRequestError: IllegalArgumentException: Credential vending was requested for table Senec.Stage.Bronze.Ampace_V3LFP.BmsAmpaceV3ModuleMeasurement, but no credentials are available
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/scripts/purge_tables.py", line 7, in <module>
client.initialize_from_yaml("./lfp_tables.yaml")
~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
File "/home/parvez/Downloads/23-0325_senec_monitoring/novum-iceberg/novum_iceberg/iceberg_client.py", line 184, in initialize_from_yaml
raise ConfigurationError(f"Failed to initialize tables from YAML: {e}")
novum_iceberg.iceberg_client.ConfigurationError: Failed to initialize tables from YAML: IllegalArgumentException: Credential vending was requested for table Senec.Stage.Bronze.Ampace_V3LFP.BmsAmpaceV3ModuleMeasurement, but no credentials are available
```
but here are the logs from the polaris container
```
2026-03-31 15:27:57,084 INFO [io.qua.htt.access-log] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000029,default] [,,,] (executor-thread-1) 192.168.0.125 - data_ops [31/Mar/2026:15:27:57 +0000] "GET /api/catalog/v1/config?warehouse=senec_catalog_6 HTTP/1.1" 200
2127
2026-03-31 15:27:57,093 INFO [io.qua.htt.access-log] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000030,default] [,,,] (executor-thread-1) 192.168.0.125 - - [31/Mar/2026:15:27:57 +0000] "POST /api/catalog/v1/oauth/tokens HTTP/1.1" 200 802
2026-03-31 15:27:57,131 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000031,default] [,,,] (executor-thread-1) Initializing non-federated catalog
2026-03-31 15:27:57,176 INFO [io.qua.htt.access-log] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000031,default] [,,,] (executor-thread-1) 192.168.0.125 - data_ops [31/Mar/2026:15:27:57 +0000] "POST /api/catalog/v1/senec_catalog_6/namespaces HTTP/1.1" 200 92
2026-03-31 15:27:57,187 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000032,default] [,,,] (executor-thread-1) Initializing non-federated catalog
2026-03-31 15:27:57,199 INFO [io.qua.htt.access-log] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000032,default] [,,,] (executor-thread-1) 192.168.0.125 - data_ops [31/Mar/2026:15:27:57 +0000] "POST /api/catalog/v1/senec_catalog_6/namespaces HTTP/1.1" 200 106
2026-03-31 15:27:57,210 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000033,default] [,,,] (executor-thread-1) Initializing non-federated catalog
2026-03-31 15:27:57,219 INFO [io.qua.htt.access-log] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000033,default] [,,,] (executor-thread-1) 192.168.0.125 - data_ops [31/Mar/2026:15:27:57 +0000] "POST /api/catalog/v1/senec_catalog_6/namespaces HTTP/1.1" 200 122
2026-03-31 15:27:57,229 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000034,default] [,,,] (executor-thread-1) Initializing non-federated catalog
2026-03-31 15:27:57,238 INFO [io.qua.htt.access-log] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000034,default] [,,,] (executor-thread-1) 192.168.0.125 - data_ops [31/Mar/2026:15:27:57 +0000] "POST /api/catalog/v1/senec_catalog_6/namespaces HTTP/1.1" 200 150
2026-03-31 15:27:57,255 INFO [org.apa.pol.ser.exc.IcebergExceptionMapper] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000035,default] [,,,] (executor-thread-1) Handling runtimeException Table does not exist: Senec.Stage.Bronze.Ampace_V3LFP.BmsAmpaceV3ModuleMea
surement
2026-03-31 15:27:57,255 INFO [io.qua.htt.access-log] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000035,default] [,,,] (executor-thread-1) 192.168.0.125 - data_ops [31/Mar/2026:15:27:57 +0000] "GET /api/catalog/v1/senec_catalog_6/namespaces/Senec%1FStage%1FBro
nze%1FAmpace_V3LFP/tables/BmsAmpaceV3ModuleMeasurement HTTP/1.1" 404 147
2026-03-31 15:27:57,293 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000036,default] [,,,] (executor-thread-1) Initializing non-federated catalog
2026-03-31 15:27:57,294 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000036,default] [,,,] (executor-thread-1) Catalog type: INTERNAL
2026-03-31 15:27:57,295 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000036,default] [,,,] (executor-thread-1) allow external catalog credential vending: true
2026-03-31 15:27:57,304 INFO [org.apa.ice.BaseMetastoreCatalog] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000036,default] [,,,] (executor-thread-1) Table properties set at catalog level through catalog properties: {}
2026-03-31 15:27:57,311 INFO [org.apa.ice.BaseMetastoreCatalog] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000036,default] [,,,] (executor-thread-1) Table properties enforced at catalog level through catalog properties: {}
2026-03-31 15:27:57,353 INFO [org.apa.ice.CatalogUtil] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000036,default] [,,,] (executor-thread-1) Loading custom FileIO implementation: org.apache.iceberg.aws.s3.S3FileIO
2026-03-31 15:27:58,068 INFO [org.apa.pol.ser.cat.ice.IcebergCatalog] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000036,default] [,,,] (executor-thread-1) Successfully committed to table senec_catalog_6.Senec.Stage.Bronze.Ampace_V3LFP.BmsAmpaceV3ModuleMeasure
ment in 732 ms
2026-03-31 15:27:58,083 INFO [org.apa.pol.ser.exc.IcebergExceptionMapper] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000036,default] [,,,] (executor-thread-1) Handling runtimeException Credential vending was requested for table Senec.Stage.Bronze.Ampace_V3LFP
.BmsAmpaceV3ModuleMeasurement, but no credentials are available
2026-03-31 15:27:58,084 INFO [io.qua.htt.access-log] [076e800c-9b9e-41e6-82d9-d651aba89f85_0000000000000000036,default] [,,,] (executor-thread-1) 192.168.0.125 - data_ops [31/Mar/2026:15:27:58 +0000] "POST /api/catalog/v1/senec_catalog_6/namespaces/Senec%1FStage%1FBr
onze%1FAmpace_V3LFP/tables HTTP/1.1" 400 206
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4163661582
@Zevrap-81 : With `sts_unavailable: true` Polaris cannot vend credentials to the client. Please double check your client setup and make sure it does not ask for credential vending.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "Zevrap-81 (via GitHub)" <gi...@apache.org>.
Zevrap-81 closed issue #4076: Creating catalogs using the cli is buggy
URL: https://github.com/apache/polaris/issues/4076
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "Zevrap-81 (via GitHub)" <gi...@apache.org>.
Zevrap-81 commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4172457295
@MonkeyCanCode yes i understand that as i already mentioned the create catalog command works, however, when i try to do the same using the setup command and a yaml file (because it’s convenient and everything in one place), it no longer works. Do you have an example of setup command for non S3 compatible storage like minio?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4227526529
@Zevrap-81 : Would you be able to provide step-by-step instruction for how to reproduce the error you're hitting? Thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "Zevrap-81 (via GitHub)" <gi...@apache.org>.
Zevrap-81 commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4163705671
I am using pyiceberg, I am not sure how to check for that. However, here is one detail, when i create the catalog using `catalogs` cli command or the management api endpoint, this doesn't happen, as you can see from the logs:
```
2026-03-31 15:48:45,473 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000047,default] [,,,] (executor-thread-1) Initializing non-federated catalog
2026-03-31 15:48:45,473 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000047,default] [,,,] (executor-thread-1) Catalog type: INTERNAL
2026-03-31 15:48:45,473 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000047,default] [,,,] (executor-thread-1) allow external catalog credential vending: true
2026-03-31 15:48:45,474 INFO [org.apa.ice.BaseMetastoreCatalog] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000047,default] [,,,] (executor-thread-1) Table properties set at catalog level through catalog properties: {}
2026-03-31 15:48:45,476 INFO [org.apa.ice.BaseMetastoreCatalog] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000047,default] [,,,] (executor-thread-1) Table properties enforced at catalog level through catalog properties: {}
2026-03-31 15:48:45,489 INFO [org.apa.ice.CatalogUtil] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000047,default] [,,,] (executor-thread-1) Loading custom FileIO implementation: org.apache.iceberg.aws.s3.S3FileIO
2026-03-31 15:48:45,503 INFO [org.apa.pol.ser.cat.ice.IcebergCatalog] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000047,default] [,,,] (executor-thread-1) Successfully committed to table senec_catalog.Senec.Stage.Bronze.Hiconics_E4.BmsHiconicsCellMeasurement
in 26 ms
2026-03-31 15:48:45,516 INFO [io.qua.htt.access-log] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000047,default] [,,,] (executor-thread-1) 192.168.0.125 - data_ops [31/Mar/2026:15:48:45 +0000] "POST /api/catalog/v1/senec_catalog/namespaces/Senec%1FStage%1FBron
ze%1FHiconics_E4/tables HTTP/1.1" 200 6060
2026-03-31 15:48:45,524 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000048,default] [,,,] (executor-thread-1) Initializing non-federated catalog
2026-03-31 15:48:45,524 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000048,default] [,,,] (executor-thread-1) Catalog type: INTERNAL
2026-03-31 15:48:45,524 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000048,default] [,,,] (executor-thread-1) allow external catalog credential vending: true
2026-03-31 15:48:45,526 INFO [org.apa.pol.ser.cat.ice.IcebergCatalog] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000048,default] [,,,] (executor-thread-1) Refreshing table metadata from new version: s3://senec-bucket/senec_catalog/Senec/Stage/Bronze/Ampace_P4
/BmsAmpaceMeasurement/metadata/00000-0c1fd557-6d71-4aca-b7b6-a82f06024ba5.metadata.json
2026-03-31 15:48:45,537 INFO [org.apa.ice.CatalogUtil] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000048,default] [,,,] (executor-thread-1) Loading custom FileIO implementation: org.apache.iceberg.aws.s3.S3FileIO
2026-03-31 15:48:45,583 INFO [org.apa.ice.BaseMetastoreCatalog] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000048,default] [,,,] (executor-thread-1) Table loaded by catalog: senec_catalog.Senec.Stage.Bronze.Ampace_P4.BmsAmpaceMeasurement
2026-03-31 15:48:45,585 INFO [io.qua.htt.access-log] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000048,default] [,,,] (executor-thread-1) 192.168.0.125 - data_ops [31/Mar/2026:15:48:45 +0000] "GET /api/catalog/v1/senec_catalog/namespaces/Senec%1FStage%1FBronz
e%1FAmpace_P4/tables/BmsAmpaceMeasurement HTTP/1.1" 200 5798
(env) parvez@PC:~/Downloads/23-0325_senec_monitoring/novum-iceberg
```
The setup remains the same yet the credential vending error is not happening. Please let me know if i am making a mistake in my yaml file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "Zevrap-81 (via GitHub)" <gi...@apache.org>.
Zevrap-81 commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4163743756
And for reference i also tried to comment out sts_unavailable and i get the following error:
```
2026-03-31 16:03:05,375 INFO [io.qua.htt.access-log] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000079,default] [,,,] (executor-thread-1) 192.168.0.125 - data_ops [31/Mar/2026:16:03:05 +0000] "GET /api/catalog/v1/senec_catalog_7/namespaces/Senec%1FStage%1FBro
nze%1FAmpace_V3LFP/tables/BmsAmpaceV3ModuleMeasurement HTTP/1.1" 404 147
2026-03-31 16:03:05,384 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000080,default] [,,,] (executor-thread-1) Initializing non-federated catalog
2026-03-31 16:03:05,385 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000080,default] [,,,] (executor-thread-1) Catalog type: INTERNAL
2026-03-31 16:03:05,385 INFO [org.apa.pol.ser.cat.ice.IcebergCatalogHandler] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000080,default] [,,,] (executor-thread-1) allow external catalog credential vending: true
2026-03-31 16:03:05,387 INFO [org.apa.ice.BaseMetastoreCatalog] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000080,default] [,,,] (executor-thread-1) Table properties set at catalog level through catalog properties: {}
2026-03-31 16:03:05,388 INFO [org.apa.ice.BaseMetastoreCatalog] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000080,default] [,,,] (executor-thread-1) Table properties enforced at catalog level through catalog properties: {}
2026-03-31 16:03:05,394 INFO [org.apa.pol.ser.exc.IcebergExceptionMapper] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000080,default] [,,,] (executor-thread-1) Handling runtimeException Failed to get subscoped credentials: protocol must not be null.
2026-03-31 16:03:05,395 INFO [io.qua.htt.access-log] [c0960217-864c-4bba-992a-86fde364ee01_0000000000000000080,default] [,,,] (executor-thread-1) 192.168.0.125 - data_ops [31/Mar/2026:16:03:05 +0000] "POST /api/catalog/v1/senec_catalog_7/namespaces/Senec%1FStage%1FBr
onze%1FAmpace_V3LFP/tables HTTP/1.1" 422 136
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4240958323
@Zevrap-81 https://github.com/apache/polaris/pull/4172 is merged and this should fix the problem for this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4180402355
> "stsEndpoint": "",
> "stsUnavailable": false,
These two settings combined do not look correct to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "Zevrap-81 (via GitHub)" <gi...@apache.org>.
Zevrap-81 commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4250246867
Thanks I will check and let you know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4156640058
@Zevrap-81 : Could you provide the exception stack trace for the `IllegalArgumentException: Credential vending was requested [...]` error?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "Zevrap-81 (via GitHub)" <gi...@apache.org>.
Zevrap-81 commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4212002439
Sorry for writing so late but here is the output of a new catalog that i created with stsUnavailable set to true:
```
[
{
"type": "INTERNAL",
"name": "senec_catalog_6",
"properties": {
"default-base-location": "s3://senec-bucket/senec_catalog_6/",
"catalog": "senec_catalog_6"
},
"createTimestamp": 1775716001870,
"lastUpdateTimestamp": 1775716001870,
"entityVersion": 1,
"storageConfigInfo": {
"storageType": "S3",
"allowedLocations": [
"s3://senec-bucket/senec_catalog_6/"
],
"roleArn": "arn:aws:iam::123456789012:role/data_ops_role_2",
"externalId": "",
"region": "us-east-1",
"endpoint": "http://localhost:9000/",
"stsEndpoint": "",
"stsUnavailable": false,
"endpointInternal": "http://rustfs:9000",
"pathStyleAccess": true
}
},
{
"type": "INTERNAL",
"name": "senec_catalog_7",
"properties": {
"default-base-location": "s3://senec-bucket/senec_catalog_7/",
"catalog": "senec_catalog_7"
},
"createTimestamp": 1775716001566,
"lastUpdateTimestamp": 1775716001566,
"entityVersion": 1,
"storageConfigInfo": {
"storageType": "S3",
"allowedLocations": [
"s3://senec-bucket/senec_catalog_7/"
],
"roleArn": "arn:aws:iam::123456789012:role/data_ops_role_2",
"externalId": "",
"region": "us-east-1",
"endpoint": "http://localhost:9000/",
"stsEndpoint": "",
"stsUnavailable": true,
"endpointInternal": "http://rustfs:9000",
"pathStyleAccess": true
}
},
{
"type": "INTERNAL",
"name": "senec_catalog",
"properties": {
"default-base-location": "s3://senec-bucket/senec_catalog/"
},
"createTimestamp": 1775716531367,
"lastUpdateTimestamp": 1775716531367,
"entityVersion": 1,
"storageConfigInfo": {
"storageType": "S3",
"allowedLocations": [
"s3://senec-bucket/senec_catalog/"
],
"endpoint": "http://localhost:9000",
"endpointInternal": "http://rustfs:9000",
"pathStyleAccess": true
}
}
]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4230603422
I think i may know why. Let me get a sample fix PR for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Creating catalogs using the cli is buggy [polaris]
Posted by "Zevrap-81 (via GitHub)" <gi...@apache.org>.
Zevrap-81 commented on issue #4076:
URL: https://github.com/apache/polaris/issues/4076#issuecomment-4175165290
> [@Zevrap-81](https://github.com/Zevrap-81) so the issue is due to missing role-arn as you pointed out above (the 400 bad request). A simple example you can find is https://github.com/apache/polaris/blob/main/site/content/guides/rustfs/docker-compose.yml where it invokes [create-catalog.sh](https://github.com/apache/polaris/blob/main/site/content/guides/assets/polaris/create-catalog.sh) when using non-AWS S3-compatible storage (which is what I am assuming on what you are using based on the minimal config). As what [@dimas-b](https://github.com/dimas-b) mentioned above, when sts/kms are not supported by the S3-compatible storage you are using, those would need to be set on the catalog property (as what you have set above but have it uncommented). One easy way to allow us debug further is show us the catalog you currently have by using following:
>
> ```
> ./polaris --profile root catalogs list
> ```
here's the output to this:
```
[
{
"type": "INTERNAL",
"name": "senec_catalog_6",
"properties": {
"default-base-location": "s3://senec-bucket/senec_catalog_6/",
"catalog": "senec_catalog_6"
},
"createTimestamp": 1775113025345,
"lastUpdateTimestamp": 1775113025345,
"entityVersion": 1,
"storageConfigInfo": {
"storageType": "S3",
"allowedLocations": [
"s3://senec-bucket/senec_catalog_6/"
],
"roleArn": "arn:aws:iam::123456789012:role/data_ops_role_2",
"externalId": "",
"region": "us-east-1",
"endpoint": "http://localhost:9000/",
"stsEndpoint": "",
"stsUnavailable": false,
"endpointInternal": "http://rustfs:9000",
"pathStyleAccess": true
}
},
{
"type": "INTERNAL",
"name": "senec_catalog",
"properties": {
"default-base-location": "s3://senec-bucket/senec_catalog/"
},
"createTimestamp": 1775113285322,
"lastUpdateTimestamp": 1775113285322,
"entityVersion": 1,
"storageConfigInfo": {
"storageType": "S3",
"allowedLocations": [
"s3://senec-bucket/senec_catalog/"
],
"endpoint": "http://localhost:9000",
"endpointInternal": "http://rustfs:9000",
"pathStyleAccess": true
}
}
]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org