You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "rinzool (via GitHub)" <gi...@apache.org> on 2025/10/23 08:23:11 UTC

[I] Unable to create polaris catalog due to missing headers [iceberg-python]

rinzool opened a new issue, #2655:
URL: https://github.com/apache/iceberg-python/issues/2655

   ### Apache Iceberg version
   
   0.10.0 (latest release)
   
   ### Please describe the bug 🐞
   
   # Description
   Since version 0.10.0 it is not possible to create a Polaris catalog with `credential` parameter. 
   
   For example:
   ```python
   catalog_parameters = {
       "type": "rest",
       # ...
       "scope": "PRINCIPAL_ROLE:ALL",
       "header.Polaris-Realm": "my-realm",
       "credential": f"{client_id}:{client_secret}"
   }
   
   catalog = load_catalog("default, **catalog_parameters)
   ```
   
   This code fails with error `RESTError: MissingOrInvalidRealm: Missing or invalid realm`.
   
   The exact same code works with pyiceberg 0.9.1.
   
   
   # Cause
   
   Polaris needs header `Polaris-Realm`. 
   
   We can see in the full stacktrace (see below) that the error occurs when we create the AuthManager line 266: https://github.com/apache/iceberg-python/blob/pyiceberg-0.10.0/pyiceberg/catalog/rest/__init__.py#L266.
   
   But headers are injected **after** initializing AuthManager at line 269:
   https://github.com/apache/iceberg-python/blob/pyiceberg-0.10.0/pyiceberg/catalog/rest/__init__.py#L269.
   
   
   <details>
   <summary>Full stacktrace</summary>
   
   ```
   File [/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/__init__.py:128](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/__init__.py#line=127), in load_rest(name, conf)
       125 def load_rest(name: str, conf: Properties) -> Catalog:
       126     from pyiceberg.catalog.rest import RestCatalog
   --> 128     return RestCatalog(name, **conf)
   
   File [/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py:234](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py#line=233), in RestCatalog.__init__(self, name, **properties)
       232 super().__init__(name, **properties)
       233 self.uri = properties[URI]
   --> 234 self._fetch_config()
       235 self._session = self._create_session()
   
   File [/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py:364](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py#line=363), in RestCatalog._fetch_config(self)
       361 if warehouse_location := self.properties.get(WAREHOUSE_LOCATION):
       362     params[WAREHOUSE_LOCATION] = warehouse_location
   --> 364 with self._create_session() as session:
       365     response = session.get(self.url(Endpoints.get_config, prefixed=False), params=params)
       366 try:
   
   File [/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py:266](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py#line=265), in RestCatalog._create_session(self)
       264     session.auth = AuthManagerAdapter(AuthManagerFactory.create(auth_impl or auth_type, auth_type_config))
       265 else:
   --> 266     session.auth = AuthManagerAdapter(self._create_legacy_oauth2_auth_manager(session))
       268 # Set HTTP headers
       269 self._config_headers(session)
   
   File [/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py:295](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py#line=294), in RestCatalog._create_legacy_oauth2_auth_manager(self, session)
       285 auth_url = self.auth_url if client_credentials is not None else None
       287 auth_config = {
       288     "session": session,
       289     "auth_url": auth_url,
      (...)
       292     "optional_oauth_params": self._extract_optional_oauth_params(),
       293 }
   --> 295 return AuthManagerFactory.create("legacyoauth2", auth_config)
   
   File [/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py:321](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py#line=320), in AuthManagerFactory.create(cls, class_or_name, config)
       318     except Exception as err:
       319         raise ValueError(f"Could not load AuthManager class for '{class_or_name}'") from err
   --> 321 return manager_cls(**config)
   
   File [/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py:94](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py#line=93), in LegacyOAuth2AuthManager.__init__(self, session, auth_url, credential, initial_token, optional_oauth_params)
        92 self._credential = credential
        93 self._optional_oauth_params = optional_oauth_params
   ---> 94 self._refresh_token()
   
   File [/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py:122](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py#line=121), in LegacyOAuth2AuthManager._refresh_token(self)
       120 def _refresh_token(self) -> None:
       121     if self._credential is not None:
   --> 122         self._token = self._fetch_access_token(self._credential)
   
   File [/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py:116](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py#line=115), in LegacyOAuth2AuthManager._fetch_access_token(self, credential)
       114     response.raise_for_status()
       115 except HTTPError as exc:
   --> 116     _handle_non_200_response(exc, {400: OAuthError, 401: OAuthError})
       118 return TokenResponse.model_validate_json(response.text).access_token
   
   File [/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/response.py:111](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/response.py#line=110), in _handle_non_200_response(exc, error_handler)
       108     errs = ", ".join(err["msg"] for err in e.errors())
       109     response = f"RESTError {exc.response.status_code}: Received unexpected JSON Payload: {exc.response.text}, errors: {errs}"
   --> 111 raise exception(response) from exc
   
   RESTError: MissingOrInvalidRealm: Missing or invalid realm
   ```
   
   </details>
   
   # Solution
   Move `self._config_headers(session)` instruction in the beginning of `_create_session` method.
   
   ### Willingness to contribute
   
   - [x] I can contribute a fix for this bug independently
   - [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
   - [ ] I cannot contribute a fix for this bug at this time


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] Unable to create polaris catalog due to missing headers [iceberg-python]

Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko closed issue #2655: Unable to create polaris catalog due to missing headers
URL: https://github.com/apache/iceberg-python/issues/2655


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org