You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by afs <gi...@git.apache.org> on 2018/05/06 14:12:00 UTC
[GitHub] jena pull request #414: JENA-1542: Integrate Lucene index in transaction lif...
GitHub user afs opened a pull request:
https://github.com/apache/jena/pull/414
JENA-1542: Integrate Lucene index in transaction lifecycle (TDB1, TDB2).
Includes the fix for JENA-1302.
Integrate a lucene text index into the transaction systems for TDB1 and DBOE/TDB2. This way, the prepare/commit for Lucene happens during database commit and the synchronization dance (JENA-1302) does not happen.
JENA-1302 only effects TDB1, not TDB2. Only TDB1 has a commit queue that can build up under load and need to be flushed, which in turn causes a thread to take the exclusive lock and so cause the JENA-1302 lockup.
Includes the fix for JENA-1302.
Integrate a Lucene text index into the transaction systems for TDB1 and DBOE/TDB2. This way, the prepare/commit for Lucene happens during database commit and the synchronization dance (JENA-1302) does not happen.
JENA-1302 only effects TDB1, not TDB2 or any other DatasetGraph implementation. Only TDB1 has a commit queue that can build up under load and need to be flushed, which in turn causes a thread to take the exclusive lock and so cause the JENA-1302 lockup.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/afs/jena text-txn
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/jena/pull/414.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #414
----
commit 39f14a1f5030424e093af26d465e9add39cc5209
Author: Andy Seaborne <an...@...>
Date: 2018-05-04T15:12:28Z
Remove false check. 'destination' can null if services are URLs.
commit f6844797aa639e0fe44f2c77c24a622673247c74
Author: Andy Seaborne <an...@...>
Date: 2018-05-04T15:56:00Z
Constructor for detailed setup.
commit 06f713d9d11b15bb55858ca778d31496da89daec
Author: Andy Seaborne <an...@...>
Date: 2018-05-04T16:05:53Z
JENA-1542: Add JenaLifecycle elements to an existing TransactionManager
Clearup the JenaLifecycle phases:
* add a post-commit step
* rename the write-back steps
commit 27ad5b74d8322d35e203cab41024f32a7b0eda24
Author: Andy Seaborne <an...@...>
Date: 2018-05-04T16:06:54Z
JENA-1542: Allow TransactionComponents to be added.
commit 2d6330f4712c26390213861f3891126f1ae69e2d
Author: Andy Seaborne <an...@...>
Date: 2018-05-06T13:24:45Z
JENA-1542, JENA-1302: Integrate jena-text with TDB1 and TDB2 transactions.
----
---
[GitHub] jena pull request #414: JENA-1542: Integrate Lucene index in transaction lif...
Posted by rvesse <gi...@git.apache.org>.
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/414#discussion_r190532549
--- Diff: jena-db/jena-dboe-transaction/src/main/java/org/apache/jena/dboe/transaction/txn/TransactionCoordinator.java ---
@@ -142,41 +141,56 @@ public TransactionCoordinator add(TransactionalComponent elt) {
* @see #add
*/
public TransactionCoordinator remove(TransactionalComponent elt) {
- checkSetup() ;
- synchronized(coordinatorLock) {
- components.remove(elt.getComponentId()) ;
- }
+ checklAllowModification() ;
+ components.remove(elt.getComponentId()) ;
return this ;
}
+
+ /**
+ * Perform modification of this {@code TransaxctionCoordiator} after it has been started.
--- End diff --
Typo in the `{@code }` here?
---
[GitHub] jena pull request #414: JENA-1542: Integrate Lucene index in transaction lif...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/jena/pull/414
---
[GitHub] jena pull request #414: JENA-1542: Integrate Lucene index in transaction lif...
Posted by rvesse <gi...@git.apache.org>.
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/414#discussion_r190533210
--- Diff: jena-text/src/test/java/org/apache/jena/query/text/TestTextTxn.java ---
@@ -0,0 +1,341 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.query.text;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assume.assumeTrue;
+
+import java.io.Reader;
+import java.io.StringReader;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List ;
+
+import org.apache.jena.atlas.iterator.Iter ;
+import org.apache.jena.atlas.lib.Creator;
+import org.apache.jena.atlas.lib.StrUtils ;
+import org.apache.jena.query.* ;
+import org.apache.jena.rdf.model.Model ;
+import org.apache.jena.sparql.core.Quad ;
+import org.apache.jena.sparql.sse.SSE ;
+import org.apache.jena.system.Txn;
+import org.apache.jena.tdb.TDBFactory ;
+import org.apache.jena.tdb2.TDB2Factory;
+import org.apache.jena.vocabulary.RDFS ;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexReader;
+import org.apache.lucene.index.IndexWriter;
+import org.apache.lucene.store.Directory ;
+import org.apache.lucene.store.RAMDirectory ;
+import org.junit.Test ;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameters;
+
+/** Text dataset tests using datasets transactionally, named unionDefaultGraph.
+ * <p>
+ * Note that in Lucene, writes are not visible to a reader of the {@code Directory}
+ * until committed. A special {@link IndexReader} is needed. See
+ * {@link DirectoryReader#open(IndexWriter)}. jena-text does not currentyl do this.
--- End diff --
Typo - `currentyl`
---