You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "ebyhr (via GitHub)" <gi...@apache.org> on 2026/05/09 01:05:21 UTC
[PR] REST: Add pagination support for list_tables [iceberg-python]
ebyhr opened a new pull request, #3348:
URL: https://github.com/apache/iceberg-python/pull/3348
<!--
Thanks for opening a pull request!
-->
<!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. -->
<!-- Closes #${GITHUB_ISSUE_ID} -->
# Rationale for this change
Follows REST catalog spec:
- /v1/{prefix}/tables: https://github.com/apache/iceberg/blob/e7a5a87f26f9de5b200254155aa037368b13a29c/open-api/rest-catalog-open-api.yaml#L525-L562
- ListTablesResponse: https://github.com/apache/iceberg/blob/e7a5a87f26f9de5b200254155aa037368b13a29c/open-api/rest-catalog-open-api.yaml#L4210-L4219
- PageToken: https://github.com/apache/iceberg/blob/e7a5a87f26f9de5b200254155aa037368b13a29c/open-api/rest-catalog-open-api.yaml#L2233-L2256
## Are these changes tested?
Yes, includes unit tests for paginated cases.
## Are there any user-facing changes?
- Before: returned incomplete table list when server paginated (only first page)
- After: returns complete table list (fetches all pages)
<!-- In the case of user-facing changes, please add the changelog label. -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] REST: Add pagination support for list_tables [iceberg-python]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on code in PR #3348:
URL: https://github.com/apache/iceberg-python/pull/3348#discussion_r3293616920
##########
pyiceberg/catalog/rest/__init__.py:
##########
@@ -1034,12 +1035,27 @@ def list_tables(self, namespace: str | Identifier) -> list[Identifier]:
self._check_endpoint(Capability.V1_LIST_TABLES)
namespace_tuple = self._check_valid_namespace_identifier(namespace)
namespace_concat = self._encode_namespace_path(namespace_tuple)
- response = self._session.get(self.url(Endpoints.list_tables, namespace=namespace_concat))
- try:
- response.raise_for_status()
- except HTTPError as exc:
- _handle_non_200_response(exc, {404: NoSuchNamespaceError})
- return [(*table.namespace, table.name) for table in ListTablesResponse.model_validate_json(response.text).identifiers]
+ url = self.url(Endpoints.list_tables, namespace=namespace_concat)
+
+ tables: list[Identifier] = []
+ page_token: str | None = None
+
+ while True:
+ params = {"pageToken": page_token} if page_token else None
Review Comment:
```suggestion
params: dict[str, str] = {}
if page_token:
params["pageToken"] = page_token
```
nit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] REST: Add pagination support for list_tables [iceberg-python]
Posted by "rambleraptor (via GitHub)" <gi...@apache.org>.
rambleraptor commented on code in PR #3348:
URL: https://github.com/apache/iceberg-python/pull/3348#discussion_r3261156256
##########
tests/catalog/test_rest.py:
##########
@@ -479,6 +479,90 @@ def test_list_tables_200(rest_mock: Mocker) -> None:
assert RestCatalog("rest", uri=TEST_URI, token=TEST_TOKEN).list_tables(namespace) == [("examples", "fooshare")]
+def test_list_tables_paginated_200(rest_mock: Mocker) -> None:
+ namespace = "examples"
+ # First page with next-page-token
+ rest_mock.get(
Review Comment:
small nit: consider some kind of helper function that attaches a new page to the rest_mock
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] REST: Add pagination support for list_tables [iceberg-python]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on PR #3348:
URL: https://github.com/apache/iceberg-python/pull/3348#issuecomment-4526820631
Thanks @ebyhr for the PR! And thanks for reviewing @rambleraptor
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] REST: Add pagination support for list_tables [iceberg-python]
Posted by "ebyhr (via GitHub)" <gi...@apache.org>.
ebyhr commented on PR #3348:
URL: https://github.com/apache/iceberg-python/pull/3348#issuecomment-4464550985
@Fokko @geruh @rambleraptor Could you review this PR when you have time? It's similar to list_views PR https://github.com/apache/iceberg-python/pull/3349
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] REST: Add pagination support for list_tables [iceberg-python]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu merged PR #3348:
URL: https://github.com/apache/iceberg-python/pull/3348
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org