You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "mkatychev (via GitHub)" <gi...@apache.org> on 2024/01/09 23:11:42 UTC
[D] Exposing a transaction API for transactional DBs [incubator-opendal]
GitHub user mkatychev created a discussion: Exposing a transaction API for transactional DBs
I'm looking into unifying databases we currently work with (SledDb and TiKV), we depend on writing and reading several times from one transaction scope.
Sled exposes a [trait](https://docs.rs/sled/latest/sled/struct.Tree.html#method.transaction):
```rust
// Use write-only transactions as a writebatch:
db.transaction(|tx_db| {
tx_db.insert(b"k1", b"cats")?;
tx_db.insert(b"k2", b"dogs")?;
Ok(())
})?;
```
While `TiKV` has a more [fine grained object](https://docs.rs/tikv-client/latest/tikv_client/struct.Transaction.html#examples):
```rust
let client = TransactionClient::new(vec!["192.168.0.100"]).await.unwrap();
let mut txn = client.begin_optimistic().await.unwrap();
let foo = txn.get("foo".to_owned()).await.unwrap().unwrap();
txn.put("bar".to_owned(), foo).await.unwrap();
txn.commit().await.unwrap();
```
What would be the best way to handle Operators that can conditionally declare "start a transaction" and "commit transaction",
should the [`Builder`](https://docs.rs/opendal/latest/opendal/trait.Builder.html) trait be implemented on the closure and struct referenced above?
GitHub link: https://github.com/apache/incubator-opendal/discussions/3962
----
This is an automatically sent email for commits@opendal.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@opendal.apache.org
Re: [D] Exposing a transaction API for transactional DBs [incubator-opendal]
Posted by "mkatychev (via GitHub)" <gi...@apache.org>.
GitHub user mkatychev edited a discussion: Exposing a transaction API for transactional DBs
I'm looking into unifying databases we currently work with (SledDb and TiKV), we depend on writing and reading several times from one transaction scope.
Sled exposes a [method](https://docs.rs/sled/latest/sled/struct.Tree.html#method.transaction):
```rust
// Use write-only transactions as a writebatch:
db.transaction(|tx_db| {
tx_db.insert(b"k1", b"cats")?;
tx_db.insert(b"k2", b"dogs")?;
Ok(())
})?;
```
While `TiKV` has a more [fine grained object](https://docs.rs/tikv-client/latest/tikv_client/struct.Transaction.html#examples):
```rust
let client = TransactionClient::new(vec!["192.168.0.100"]).await.unwrap();
let mut txn = client.begin_optimistic().await.unwrap();
let foo = txn.get("foo".to_owned()).await.unwrap().unwrap();
txn.put("bar".to_owned(), foo).await.unwrap();
txn.commit().await.unwrap();
```
What would be the best way to handle Operators that can conditionally declare "start a transaction" and "commit transaction",
should the [`Builder`](https://docs.rs/opendal/latest/opendal/trait.Builder.html) trait be implemented on the closure and struct referenced above? if an operator could spawn a transaction object itself would be great:
```rust
txn = op.begin().await?;
txn.write("k1", "v1").await?;
txn.write("k2", "v2").await?;
let v3 = txn.write("k2", "v2").await?;
txn.commit().await?;
```
similar to this TiKV snippet:
```rust
let mut txn = client.begin_pessimistic().await.unwrap();
let key = "TiKV".to_owned();
let result: Value = txn.get_for_update(key).await.unwrap().unwrap();
// now the key "TiKV" is locked, other transactions cannot modify it
// Finish the transaction...
txn.commit().await.unwrap();
```
GitHub link: https://github.com/apache/incubator-opendal/discussions/3962
----
This is an automatically sent email for commits@opendal.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@opendal.apache.org
Re: [D] Exposing a transaction API for transactional DBs [incubator-opendal]
Posted by "mkatychev (via GitHub)" <gi...@apache.org>.
GitHub user mkatychev edited a discussion: Exposing a transaction API for transactional DBs
I'm looking into unifying databases we currently work with (SledDb and TiKV), we depend on writing and reading several times from one transaction scope.
Sled exposes a [trait](https://docs.rs/sled/latest/sled/struct.Tree.html#method.transaction):
```rust
// Use write-only transactions as a writebatch:
db.transaction(|tx_db| {
tx_db.insert(b"k1", b"cats")?;
tx_db.insert(b"k2", b"dogs")?;
Ok(())
})?;
```
While `TiKV` has a more [fine grained object](https://docs.rs/tikv-client/latest/tikv_client/struct.Transaction.html#examples):
```rust
let client = TransactionClient::new(vec!["192.168.0.100"]).await.unwrap();
let mut txn = client.begin_optimistic().await.unwrap();
let foo = txn.get("foo".to_owned()).await.unwrap().unwrap();
txn.put("bar".to_owned(), foo).await.unwrap();
txn.commit().await.unwrap();
```
What would be the best way to handle Operators that can conditionally declare "start a transaction" and "commit transaction",
should the [`Builder`](https://docs.rs/opendal/latest/opendal/trait.Builder.html) trait be implemented on the closure and struct referenced above? if an operator could spawn a transaction object itself would be great:
```rust
txn = op.begin().await?;
txn.write("k1", "v1").await?;
txn.write("k2", "v2").await?;
let v3 = txn.write("k2", "v2").await?;
txn.commit().await?;
```
similar to this TiKV snippet:
```rust
let mut txn = client.begin_pessimistic().await.unwrap();
let key = "TiKV".to_owned();
let result: Value = txn.get_for_update(key).await.unwrap().unwrap();
// now the key "TiKV" is locked, other transactions cannot modify it
// Finish the transaction...
txn.commit().await.unwrap();
```
GitHub link: https://github.com/apache/incubator-opendal/discussions/3962
----
This is an automatically sent email for commits@opendal.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@opendal.apache.org
Re: [D] Exposing a transaction API for transactional DBs [incubator-opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
GitHub user Xuanwo added a comment to the discussion: Exposing a transaction API for transactional DBs
```rust
txn = op.begin().await?;
txn.write("k1", "v1").await?;
txn.write("k2", "v2").await?;
let v3 = txn.write("k2", "v2").await?;
txn.commit().await?;
```
It seems interesting, but I'm a bit worried about how it can be extended to other services. Maybe we can expose a `KvOperator` to users instead?
So we will have:
- `Operator`: users can perform file-oriented operations (no transaction) to s3, fs, sled, tikv
- `KvOperator`: users can perform key-oriented operations (transaction! batch!) to s3, fs, sled, tikv
Services that doesn't support transaction could simply return `Unsupported`.
GitHub link: https://github.com/apache/incubator-opendal/discussions/3962#discussioncomment-8074447
----
This is an automatically sent email for commits@opendal.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@opendal.apache.org