You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "kevinjqliu (via GitHub)" <gi...@apache.org> on 2024/10/27 21:38:51 UTC
[I] [discuss] `Transaction` API's `autocommit` [iceberg-python]
kevinjqliu opened a new issue, #1253:
URL: https://github.com/apache/iceberg-python/issues/1253
### Feature Request / Improvement
Porting over from #1246
Can be a potential footgun (https://github.com/apache/iceberg-python/pull/1246#issuecomment-2439024843)
Autocommit usage (https://github.com/apache/iceberg-python/pull/1246#issuecomment-2439780764)
> However, there may still be some concerns around this since Transaction is a public class. If this is the case, I think we can start from making the parameter "private" (autocommit -> _autocommit) and/or adding some doc to explain the usage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] [discuss] `Transaction` API's `autocommit` [iceberg-python]
Posted by "jiakai-li (via GitHub)" <gi...@apache.org>.
jiakai-li commented on issue #1253:
URL: https://github.com/apache/iceberg-python/issues/1253#issuecomment-2583844373
Just raise another possibility for discussion. We could also modify the `_transaction._autocommit` when calling `UpdateSchema.__enter__` method, which indicates multiple updates is coming and we would like to avoid the autocommit. Something like:
```python
class UpdateSchema(UpdateTableMetadata["UpdateSchema"]):
...
def __enter__(self):
self._transaction._autocommit = False
return self
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] [discuss] `Transaction` API's `autocommit` [iceberg-python]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #1253:
URL: https://github.com/apache/iceberg-python/issues/1253#issuecomment-2440186317
> The idea is to make the code simpler if we only want to evolve schema/spec/...
i.e.
```
with table.update_schema() as update:
update.add_column("some_field", IntegerType(), "doc")
```
instead of another with..transaction wrapper
```
with table.transaction() as transaction:
with transaction.update_schema() as update_schema:
update.add_column("some_other_field", IntegerType(), "doc")
```
....
What about moving the `autocommit` logic out of `Transaction` and into the class that uses it instead?
For example, `UpdateSchema` can implement `__enter__` and `__exit__` to commit the transaction automatically?
Currently, these classes use `autocommit=True`:
- ManageSnapshots
- UpdateSchema
- UpdateSpec
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] [discuss] `Transaction` API's `autocommit` [iceberg-python]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #1253:
URL: https://github.com/apache/iceberg-python/issues/1253#issuecomment-3059680622
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] [discuss] `Transaction` API's `autocommit` [iceberg-python]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #1253:
URL: https://github.com/apache/iceberg-python/issues/1253#issuecomment-3741156383
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] [discuss] `Transaction` API's `autocommit` [iceberg-python]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #1253:
URL: https://github.com/apache/iceberg-python/issues/1253#issuecomment-3808250338
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] [discuss] `Transaction` API's `autocommit` [iceberg-python]
Posted by "jiakai-li (via GitHub)" <gi...@apache.org>.
jiakai-li commented on issue #1253:
URL: https://github.com/apache/iceberg-python/issues/1253#issuecomment-2583876079
Adding an example for the issue:
```python
from pyiceberg.catalog.sql import SqlCatalog
from pyiceberg.schema import Schema
from pyiceberg.types import IntegerType, NestedField, StringType
WAREHOUSE_PATH = "/workspaces/iceberg-python/warehouse"
catalog = SqlCatalog(
"default",
uri=f"sqlite:///{WAREHOUSE_PATH}/pyiceberg_catalog.db", warehouse=f"file://{WAREHOUSE_PATH}",
)
catalog.create_namespace_if_not_exists("default")
try:
catalog.drop_table("default.test")
except:
pass
schema = Schema(
NestedField(1, "field1", StringType(), required=False)
)
table = catalog.create_table("default.test", schema)
with table.update_schema() as update:
update.add_column("field_should_not_exist", IntegerType())
raise Exception("Error!")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] [discuss] `Transaction` API's `autocommit` [iceberg-python]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #1253:
URL: https://github.com/apache/iceberg-python/issues/1253#issuecomment-2584974311
> The reason behind my head is, to my understanding, autocommit is still somewhat a related concept with transaction.
I think its fair to ask if we consider `UpdateSchema` to be a transaction. If so, we cannot allow `autocommit` since that'll break the atomicity of the transaction.
> We could also modify the _transaction._autocommit when calling UpdateSchema.__enter__ method, which indicates multiple updates is coming and we would like to avoid the autocommit. Something like:
I feel like in this case, we can just get rid of `autocommit` entirely
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] [discuss] `Transaction` API's `autocommit` [iceberg-python]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #1253: [discuss] `Transaction` API's `autocommit`
URL: https://github.com/apache/iceberg-python/issues/1253
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org