You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "Fokko (via GitHub)" <gi...@apache.org> on 2023/12/05 12:40:19 UTC
[I] Automatically create the tables for the `SqlCatalog` [iceberg-python]
Fokko opened a new issue, #184:
URL: https://github.com/apache/iceberg-python/issues/184
### Feature Request / Improvement
The table creation process is still manual:
```python
from pyiceberg.catalog import load_catalog
catalog = load_catalog(
type='sql',
uri='sqlite:////tmp/database.db'
)
catalog.create_tables()
```
However, it would be better to check if the table exists by running `SELECT 1 FROM iceberg_tables` when initializing the catalog. If it raises a `NoSuchTableError` then we automatically create the tables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Automatically create the tables for the `SqlCatalog` [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko closed issue #184: Automatically create the tables for the `SqlCatalog`
URL: https://github.com/apache/iceberg-python/issues/184
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Automatically create the tables for the `SqlCatalog` [iceberg-python]
Posted by "cosmastech (via GitHub)" <gi...@apache.org>.
cosmastech commented on issue #184:
URL: https://github.com/apache/iceberg-python/issues/184#issuecomment-1841735588
Hi @Fokko
I was interested in contributing to this, but I'm a little bit confused.
I see that `Catalog` class has this included in the docstring for `load_table()`
```py
You can also use this method to check for table existence using 'try catalog.table() except NoSuchTableError'.
```
However, the `Catalog.table()` method does not exist. I tried calling `Catalog.load_table()` but I don't think that's exactly getting us what we want (it's not checking if the table
If I were to take this on, I would have to create an abstract method to check if a table exists and then implement it for each class that extends Catalog, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Automatically create the tables for the `SqlCatalog` [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #184:
URL: https://github.com/apache/iceberg-python/issues/184#issuecomment-1842280596
Hey @cosmastech Thanks for chiming in here!
The confusion here is between the Iceberg tables in general and the tables used by the SqlCatalog (which uses Alembic). We don't want to auto-create any Iceberg tables.
When you create a new SqlCatalog, there will be no tables in which to store the catalog information. This means that it is specific to the SqlCatalog, and we don't need to create any abstract methods. When initializing a SqlCatalog, we want to fire a query just to check if the two tables ([`iceberg_namespace_properties`](https://github.com/apache/iceberg-python/blob/34b18e4b7b3d7d44e90a7b6ae266ad026da26150/pyiceberg/catalog/sql.py#L83) and [`iceberg_tables`](https://github.com/apache/iceberg-python/blob/main/pyiceberg/catalog/sql.py#L73C22-L73C36)). Does this help?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org