You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by afs <gi...@git.apache.org> on 2018/01/01 19:59:37 UTC
[GitHub] jena pull request #336: JENA-1458: Transaction promotion
GitHub user afs opened a pull request:
https://github.com/apache/jena/pull/336
JENA-1458: Transaction promotion
This PR provides promotion (read mode goes to write mode as needed) for TIM, TDB1 and TDB2.
Transactions have a type (new) and a current mode.
* TxnMode (enum `ReadWrite` - name retained for compatibility)
* `TxnType` - `WRITE`, `READ`, `READ_PROMOTE`, `READ_COMMITTED_PROMOTE`
Transactions can go from "read" to "write", if their initial setup allows it, when an update operation is attempted (implicit promotion) or if the app asks for the transaction to be promoted (explicit promotion).
`Transactional` and hence the DatasetGraph/Dataset interfaces get new operations:
WRITE and READ are the same as the previous use of ReadWrite for W and R transactions.
The two different types of promote differ in whether strict transaction isolation is
maintained (`READ_PROMOTE`) and no changes from other writers
are seen during the transaction, or (`READ_COMMITTED_PROMOTE`) whether at the point of promotion, changes become visible.
`READ_PROMOTE` can fail (implicit promote - transaction aborts; explicit promote, `promote()` returns false) whereas `READ_COMMITTED_PROMOTE` does not fail - it's like ending a READ transaction and starting a fresh WRITE transaction; changes from other writers can be seen across the change.
The new operations are:
```
public void begin()
public void begin(TxnType type)
public boolean promote()
public ReadWrite transactionMode();
public TxnMode transactionType();
```
and `public void begin(ReadWrite mode)` remains with the same semantics as before. No application needs to change.
`begin()` is `begin(TxnType.READ_PROMOTE)`, which can cause a system abort.
This is a choice - it could be `begin(TxnType.READ_COMMITTED_PROMOTE)` which does not cause system aborts but any reads before the promote point are not no longer guaranteed to be the state of the data (the transaction is "read committed", not strong isolation properties) leading to possible inconsistent data update.
Graph transactions handlers call `begin()`.
To do:
* Update `Txn`
* More checking
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/afs/jena promotion
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/jena/pull/336.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #336
----
commit 74808450e635418cca758a564288fc80994b3ad6
Author: Andy Seaborne <an...@...>
Date: 2017-12-29T18:56:57Z
Remove duplicate ThreadTxn from DBOE.
commit 143ac13b69a26b91d0f6f97991794231c4edd0cd
Author: Andy Seaborne <an...@...>
Date: 2017-12-30T12:30:35Z
JENA-1458: Promotion API and convert TIM
commit ee7507df9fe24abcf076e449807dc8d056699392
Author: Andy Seaborne <an...@...>
Date: 2017-12-30T22:57:06Z
JENA-1458: Use TxnType, replacing ReadWrite mode
commit 64f6357c25494a961f77db670cbe5b2d4cc35eab
Author: Andy Seaborne <an...@...>
Date: 2017-12-30T22:57:46Z
JENA-1458: Promotion API integration for TDB
commit d6770eaf62f13d60bc5f414cfa6b2f3a50fca445
Author: Andy Seaborne <an...@...>
Date: 2017-12-31T17:15:04Z
JENA-1458: Promotion API integration for TDB2
commit 55983cdae25a2a9d23b695486e4488f76fd94786
Author: Andy Seaborne <an...@...>
Date: 2017-12-31T20:49:18Z
Fix to work with tests. Cleanup.
----
---
[GitHub] jena pull request #336: JENA-1458: Transaction promotion
Posted by ajs6f <gi...@git.apache.org>.
Github user ajs6f commented on a diff in the pull request:
https://github.com/apache/jena/pull/336#discussion_r159162566
--- Diff: jena-arq/src/main/java/org/apache/jena/query/TxnType.java ---
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.query;
+
+import java.util.Objects;
+
+import org.apache.jena.sparql.JenaTransactionException;
+
+public enum TxnType {
+ /** Transaction mode:
+ * <ul>
+ * <li>{@code WRITE}: this gaurantees a WRITE will complete if {@code commit()} is
+ * called. The same as {@code begin(ReadWrite.WRITE)}.
+ *
+ * <li>{@code READ}: the transaction can not promote to WRITE,ensuring read-only
+ * access to the data. The same as {@code begin(ReadWrite.READ)}.
+ *
+ * <li>{@code READ_PROMOTE}: the transaction will go from "read" to "write" if the
+ * dataset has not been modified but if it has, the promotion fails with
+ * exception.
+ *
+ * <li>{@code READ_COMMITTED_PROMOTE}: Use this with care. The promotion will succeed but
+ * changes from oher transactions become visible.
+ * </ul>
+ *
+ * Read committed: at the point transaction attempts promotion from "read" to
+ * "write", the sytem checks if the datset has chnage since the trsnaction started
+ * (called {@code begin}). If {@code READ_PROMOTE}, the dataset must not have
+ * changed; if {@code READ_COMMITTED_PROMOTE} anyh intermediate changes are
--- End diff --
typo : `anyh` -> `any`
---
[GitHub] jena pull request #336: JENA-1458: Transaction promotion
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/jena/pull/336
---
[GitHub] jena pull request #336: JENA-1458: Transaction promotion
Posted by afs <gi...@git.apache.org>.
Github user afs commented on a diff in the pull request:
https://github.com/apache/jena/pull/336#discussion_r159746359
--- Diff: jena-arq/src/main/java/org/apache/jena/sparql/core/mem/DatasetGraphInMemory.java ---
@@ -146,25 +162,84 @@ private void _begin(ReadWrite readWrite) {
}
/** Called transaction start code at most once per transaction. */
- private void startTransaction(ReadWrite mode) {
- transactionLock.enterCriticalSection(mode.equals(READ)); // get the dataset write lock, if needed.
- transactionType(mode);
+ private void startTransaction(TxnType txnType, ReadWrite mode) {
+ transactionLock.enterCriticalSection(mode.equals(ReadWrite.READ)); // get the dataset write lock, if needed.
+ transactionType.set(txnType);
+ transactionMode(mode);
isInTransaction(true);
}
/** Called transaction ending code at most once per transaction. */
private void finishTransaction() {
isInTransaction.remove();
transactionType.remove();
+ transactionMode.remove();
version.remove();
transactionLock.leaveCriticalSection();
}
+ @Override
+ public boolean promote() {
+ if (!isInTransaction())
+ throw new JenaTransactionException("Tried to promote outside a transaction!");
+ if ( transactionMode().equals(ReadWrite.WRITE) )
+ return true;
+
+ boolean readCommitted;
+ // Initial state
+ switch(transactionType.get()) {
+ case WRITE :
+ return true;
+ case READ :
+ throw new JenaTransactionException("Tried to promote READ transaction");
+ case READ_COMMITTED_PROMOTE :
+ readCommitted = true;
+ case READ_PROMOTE :
+ readCommitted = false;
+ // Maybe!
+ break;
+ default:
+ throw new NullPointerException();
+ }
+
+ try {
+ _promote(readCommitted);
+ return true;
+ } catch (JenaTransactionException ex) {
+ return false ;
+ }
+ }
+
+ private void _promote(boolean readCommited) {
+ //System.err.printf("Promote: version=%d generation=%d\n", version.get() , generation.get()) ;
+
+ // Outside lock.
+ if ( ! readCommited && version.get() != generation.get() ) {
--- End diff --
Yes- that's what happens. This part of the code isn't new - only moved. The check is done twice, once before the lock, once again after to avoid unnecessary `transactionLock` taking.
It's the same algorithm in TDB and TDB2. TDB and TDB2 have explicit `Transaction` objects that carry the start version. The shared tests cover all cases.
---
[GitHub] jena pull request #336: JENA-1458: Transaction promotion
Posted by afs <gi...@git.apache.org>.
Github user afs commented on a diff in the pull request:
https://github.com/apache/jena/pull/336#discussion_r159750664
--- Diff: jena-arq/src/main/java/org/apache/jena/sparql/core/mem/DatasetGraphInMemory.java ---
@@ -146,25 +162,84 @@ private void _begin(ReadWrite readWrite) {
}
/** Called transaction start code at most once per transaction. */
- private void startTransaction(ReadWrite mode) {
- transactionLock.enterCriticalSection(mode.equals(READ)); // get the dataset write lock, if needed.
- transactionType(mode);
+ private void startTransaction(TxnType txnType, ReadWrite mode) {
+ transactionLock.enterCriticalSection(mode.equals(ReadWrite.READ)); // get the dataset write lock, if needed.
+ transactionType.set(txnType);
+ transactionMode(mode);
isInTransaction(true);
}
/** Called transaction ending code at most once per transaction. */
private void finishTransaction() {
isInTransaction.remove();
transactionType.remove();
+ transactionMode.remove();
version.remove();
transactionLock.leaveCriticalSection();
}
+ @Override
+ public boolean promote() {
+ if (!isInTransaction())
+ throw new JenaTransactionException("Tried to promote outside a transaction!");
+ if ( transactionMode().equals(ReadWrite.WRITE) )
+ return true;
+
+ boolean readCommitted;
+ // Initial state
+ switch(transactionType.get()) {
+ case WRITE :
+ return true;
+ case READ :
+ throw new JenaTransactionException("Tried to promote READ transaction");
+ case READ_COMMITTED_PROMOTE :
+ readCommitted = true;
+ case READ_PROMOTE :
+ readCommitted = false;
+ // Maybe!
+ break;
+ default:
+ throw new NullPointerException();
+ }
+
+ try {
+ _promote(readCommitted);
+ return true;
+ } catch (JenaTransactionException ex) {
+ return false ;
+ }
+ }
+
+ private void _promote(boolean readCommited) {
+ //System.err.printf("Promote: version=%d generation=%d\n", version.get() , generation.get()) ;
+
+ // Outside lock.
+ if ( ! readCommited && version.get() != generation.get() ) {
--- End diff --
I'll take that as an "approved" :-) and merge it.
This PR is big so we can keep reviewing after the merge if people want to make further comments.
---
[GitHub] jena issue #336: JENA-1458: Transaction promotion
Posted by afs <gi...@git.apache.org>.
Github user afs commented on the issue:
https://github.com/apache/jena/pull/336
(:clap: :clap: :clap:). :grinning:
---
[GitHub] jena pull request #336: JENA-1458: Transaction promotion
Posted by ajs6f <gi...@git.apache.org>.
Github user ajs6f commented on a diff in the pull request:
https://github.com/apache/jena/pull/336#discussion_r159729634
--- Diff: jena-arq/src/main/java/org/apache/jena/sparql/core/mem/DatasetGraphInMemory.java ---
@@ -146,25 +162,84 @@ private void _begin(ReadWrite readWrite) {
}
/** Called transaction start code at most once per transaction. */
- private void startTransaction(ReadWrite mode) {
- transactionLock.enterCriticalSection(mode.equals(READ)); // get the dataset write lock, if needed.
- transactionType(mode);
+ private void startTransaction(TxnType txnType, ReadWrite mode) {
+ transactionLock.enterCriticalSection(mode.equals(ReadWrite.READ)); // get the dataset write lock, if needed.
+ transactionType.set(txnType);
+ transactionMode(mode);
isInTransaction(true);
}
/** Called transaction ending code at most once per transaction. */
private void finishTransaction() {
isInTransaction.remove();
transactionType.remove();
+ transactionMode.remove();
version.remove();
transactionLock.leaveCriticalSection();
}
+ @Override
+ public boolean promote() {
+ if (!isInTransaction())
+ throw new JenaTransactionException("Tried to promote outside a transaction!");
+ if ( transactionMode().equals(ReadWrite.WRITE) )
+ return true;
+
+ boolean readCommitted;
+ // Initial state
+ switch(transactionType.get()) {
+ case WRITE :
+ return true;
+ case READ :
+ throw new JenaTransactionException("Tried to promote READ transaction");
+ case READ_COMMITTED_PROMOTE :
+ readCommitted = true;
+ case READ_PROMOTE :
+ readCommitted = false;
+ // Maybe!
+ break;
+ default:
+ throw new NullPointerException();
+ }
+
+ try {
+ _promote(readCommitted);
+ return true;
+ } catch (JenaTransactionException ex) {
+ return false ;
+ }
+ }
+
+ private void _promote(boolean readCommited) {
+ //System.err.printf("Promote: version=%d generation=%d\n", version.get() , generation.get()) ;
+
+ // Outside lock.
+ if ( ! readCommited && version.get() != generation.get() ) {
--- End diff --
Just checking that I understand the algo here...
1. Every transaction pulls a version number from the current generation number.
2. A commit increments the generation number.
3. When a transaction attempts to promote, it checks to see that the generation hasn't incremented out from under it (indicating that some other transaction committed since this one began). If that happened, this transaction will throw an exception as shown below.
4. Otherwise, it's okay to promote.
So two questions:
1. Is that understanding accurate?
2. Is that essentially how this is implemented in other dataset impls, or do I need to go read them as well?
---
[GitHub] jena pull request #336: JENA-1458: Transaction promotion
Posted by ajs6f <gi...@git.apache.org>.
Github user ajs6f commented on a diff in the pull request:
https://github.com/apache/jena/pull/336#discussion_r159748796
--- Diff: jena-arq/src/main/java/org/apache/jena/sparql/core/mem/DatasetGraphInMemory.java ---
@@ -146,25 +162,84 @@ private void _begin(ReadWrite readWrite) {
}
/** Called transaction start code at most once per transaction. */
- private void startTransaction(ReadWrite mode) {
- transactionLock.enterCriticalSection(mode.equals(READ)); // get the dataset write lock, if needed.
- transactionType(mode);
+ private void startTransaction(TxnType txnType, ReadWrite mode) {
+ transactionLock.enterCriticalSection(mode.equals(ReadWrite.READ)); // get the dataset write lock, if needed.
+ transactionType.set(txnType);
+ transactionMode(mode);
isInTransaction(true);
}
/** Called transaction ending code at most once per transaction. */
private void finishTransaction() {
isInTransaction.remove();
transactionType.remove();
+ transactionMode.remove();
version.remove();
transactionLock.leaveCriticalSection();
}
+ @Override
+ public boolean promote() {
+ if (!isInTransaction())
+ throw new JenaTransactionException("Tried to promote outside a transaction!");
+ if ( transactionMode().equals(ReadWrite.WRITE) )
+ return true;
+
+ boolean readCommitted;
+ // Initial state
+ switch(transactionType.get()) {
+ case WRITE :
+ return true;
+ case READ :
+ throw new JenaTransactionException("Tried to promote READ transaction");
+ case READ_COMMITTED_PROMOTE :
+ readCommitted = true;
+ case READ_PROMOTE :
+ readCommitted = false;
+ // Maybe!
+ break;
+ default:
+ throw new NullPointerException();
+ }
+
+ try {
+ _promote(readCommitted);
+ return true;
+ } catch (JenaTransactionException ex) {
+ return false ;
+ }
+ }
+
+ private void _promote(boolean readCommited) {
+ //System.err.printf("Promote: version=%d generation=%d\n", version.get() , generation.get()) ;
+
+ // Outside lock.
+ if ( ! readCommited && version.get() != generation.get() ) {
--- End diff --
Yeah, I knew this stuff was here but I didn't read it carefully when you put it in months ago because we weren't turning it on. Okay, I'm :+1: . Let 'er rip!
---
[GitHub] jena issue #336: JENA-1458: Transaction promotion
Posted by afs <gi...@git.apache.org>.
Github user afs commented on the issue:
https://github.com/apache/jena/pull/336
That's it for now - the last commits have been converting and tidying up the test machinery. This in turn showed up some corner cases.
There is one design issue with TDB2 where `TransactionalComponentLifecycle` should probably split lifecycles into read and write but the main components (`BPTree` and `TransBinaryData`) do this them selves and now so does the classed used for testing `TransCounter`); or may I've forgotten why it is the way it is (because of promotion?). Needs investigation sometime. But it all seems to work as is.
---
[GitHub] jena pull request #336: JENA-1458: Transaction promotion
Posted by ajs6f <gi...@git.apache.org>.
Github user ajs6f commented on a diff in the pull request:
https://github.com/apache/jena/pull/336#discussion_r159162581
--- Diff: jena-arq/src/main/java/org/apache/jena/query/TxnType.java ---
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.query;
+
+import java.util.Objects;
+
+import org.apache.jena.sparql.JenaTransactionException;
+
+public enum TxnType {
+ /** Transaction mode:
+ * <ul>
+ * <li>{@code WRITE}: this gaurantees a WRITE will complete if {@code commit()} is
--- End diff --
typo: `gaurantees` -> `guarantees`
---