You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@jena.apache.org by GitBox <gi...@apache.org> on 2021/03/09 11:55:29 UTC
[GitHub] [jena] afs opened a new pull request #951: JENA-2065: RDF-star
afs opened a new pull request #951:
URL: https://github.com/apache/jena/pull/951
This update RDF-star handling to the current state of the RDF-star group work.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a change in pull request #951: JENA-2065: RDF-star
Posted by GitBox <gi...@apache.org>.
afs commented on a change in pull request #951:
URL: https://github.com/apache/jena/pull/951#discussion_r591321034
##########
File path: jena-db/jena-tdb2/src/main/java/org/apache/jena/tdb2/solver/Solver.java
##########
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.tdb2.solver;
+
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.List;
+import java.util.function.Predicate;
+
+import org.apache.jena.atlas.iterator.Iter;
+import org.apache.jena.atlas.lib.tuple.Tuple;
+import org.apache.jena.atlas.lib.tuple.TupleFactory;
+import org.apache.jena.graph.Node;
+import org.apache.jena.graph.Triple;
+import org.apache.jena.sparql.core.BasicPattern;
+import org.apache.jena.sparql.core.Quad;
+import org.apache.jena.sparql.engine.ExecutionContext;
+import org.apache.jena.sparql.engine.QueryIterator;
+import org.apache.jena.sparql.engine.binding.Binding;
+import org.apache.jena.sparql.engine.iterator.QueryIterPlainWrapper;
+import org.apache.jena.tdb2.TDBException;
+import org.apache.jena.tdb2.store.DatasetGraphTDB;
+import org.apache.jena.tdb2.store.GraphTDB;
+import org.apache.jena.tdb2.store.NodeId;
+import org.apache.jena.tdb2.store.nodetable.NodeTable;
+import org.apache.jena.tdb2.store.nodetupletable.NodeTupleTable;
+
+/** Entry to the basic pattern solver for TDB2 */
+public class Solver {
+
+ /** Non-reordering execution of a basic graph pattern, given a iterator of bindings as input */
+ public static QueryIterator execute(GraphTDB graph, BasicPattern pattern,
+ QueryIterator input, Predicate<Tuple<NodeId>> filter,
+ ExecutionContext execCxt)
+ {
+ // Maybe default graph or named graph.
+ NodeTupleTable ntt = graph.getNodeTupleTable();
+ return execute(ntt, graph.getGraphName(), pattern, input, filter, execCxt);
+ }
+
+ /** Non-reordering execution of a quad pattern, given a iterator of bindings as input.
Review comment:
Another global search done ...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] kinow commented on a change in pull request #951: JENA-2065: RDF-star
Posted by GitBox <gi...@apache.org>.
kinow commented on a change in pull request #951:
URL: https://github.com/apache/jena/pull/951#discussion_r591031747
##########
File path: jena-arq/src/main/java/org/apache/jena/riot/lang/extra/javacc/TurtleJavacc.java
##########
@@ -369,8 +366,34 @@ final public void Annotation(Node s, Node p, Node o) throws ParseException {
throw new Error("Missing return statement in function");
}
-// RDF-star objectXTurtle [12] object
- final public Node ObjectX() throws ParseException {Node o ; String iri;
+ final public Node EmbSubject() throws ParseException {Node o ; String iri;
+ switch ((jj_ntk==-1)?jj_ntk_f():jj_ntk) {
+ case IRIref:
+ case PNAME_NS:
+ case PNAME_LN:{
+ iri = iri();
+o = createNode(iri, token.beginLine, token.beginColumn) ;
Review comment:
I remember we had a javacc grammar file somewhere I think, used to produce these files, but I didn't see the grammar file when I was sifting through the changes (I skip a few files that are not *.java, or that are tests, so I may have missed it).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] kinow commented on pull request #951: JENA-2065: RDF-star
Posted by GitBox <gi...@apache.org>.
kinow commented on pull request #951:
URL: https://github.com/apache/jena/pull/951#issuecomment-794413662
A good chance to try to read the latest w3c draft, go through some points, and see if I can get my Jena project to build and run from `main` locally. No worry if it's merged before :+1:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs merged pull request #951: JENA-2065: RDF-star
Posted by GitBox <gi...@apache.org>.
afs merged pull request #951:
URL: https://github.com/apache/jena/pull/951
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] kinow commented on a change in pull request #951: JENA-2065: RDF-star
Posted by GitBox <gi...@apache.org>.
kinow commented on a change in pull request #951:
URL: https://github.com/apache/jena/pull/951#discussion_r590887785
##########
File path: jena-arq/src/main/java/org/apache/jena/riot/lang/extra/javacc/TurtleJavacc.java
##########
@@ -369,8 +366,34 @@ final public void Annotation(Node s, Node p, Node o) throws ParseException {
throw new Error("Missing return statement in function");
}
-// RDF-star objectXTurtle [12] object
- final public Node ObjectX() throws ParseException {Node o ; String iri;
+ final public Node EmbSubject() throws ParseException {Node o ; String iri;
+ switch ((jj_ntk==-1)?jj_ntk_f():jj_ntk) {
+ case IRIref:
+ case PNAME_NS:
+ case PNAME_LN:{
+ iri = iri();
+o = createNode(iri, token.beginLine, token.beginColumn) ;
+ break;
+ }
+ case ANON:
+ case BLANK_NODE_LABEL:{
+ o = BlankNode();
+ break;
+ }
+ case LT2:{
+ o = EmbTriple();
+ break;
+ }
+ default:
+ jj_la1[14] = jj_gen;
+ jj_consume_token(-1);
+ throw new ParseException();
+ }
+{if ("" != null) return o;}
Review comment:
and here
##########
File path: jena-arq/src/main/java/org/apache/jena/sparql/engine/iterator/RX.java
##########
@@ -176,5 +74,121 @@ private static boolean tripleHasNodeTriple(Triple triple) {
/*|| triple.getPredicate().isNodeTriple()*/
|| triple.getObject().isNodeTriple();
}
-}
+ private static QueryIterator rdfStarTripleSub(QueryIterator input, Triple pattern, ExecutionContext execCxt) {
+ Iterator<Binding> matches = Iter.flatMap(input, binding->rdfStarTripleSub(binding, pattern, execCxt));
+ return new QueryIterPlainWrapper(matches, execCxt);
+ }
+
+ private static Iterator<Binding> rdfStarTripleSub(Binding input, Triple tPattern, ExecutionContext execCxt) {
+ Node s = nodeTopLevel(tPattern.getSubject());
+ Node p = nodeTopLevel(tPattern.getPredicate());
+ Node o = nodeTopLevel(tPattern.getObject());
+ Graph graph = execCxt.getActiveGraph();
+ ExtendedIterator<Triple> graphIter = graph.find(s, p, o);
+ ExtendedIterator<Binding> matched = graphIter.mapWith(tData->matchTriple(input, tData, tPattern)).filterDrop(Objects::isNull);
+ return matched;
+ }
+
+ /** Convert a pattern node into ANY, or leave as a constant term. */
+ public static Node nodeTopLevel(Node node) {
+ // Public so TDB code can use it.
+ if ( Var.isVar(node) )
+ return Node.ANY;
+ if ( node.isNodeTriple() ) { //|| node.isNodeGraph() )
+ if ( ! node.getTriple().isConcrete() )
+ // Nested variables.
+ return Node.ANY;
+ }
+ return node;
+ }
+
+ public static Binding matchTriple(Binding input, Triple tData, Triple tPattern) {
+ BindingBuilder bb = Binding.builder(input);
+ boolean r = matchTriple(bb, tData, tPattern);
+ return r ? bb.build() : null;
+ }
+
+ private static boolean matchTriple(BindingBuilder bb, Triple tData, Triple tPattern) {
+ Node sPattern = tPattern.getSubject();
+ Node pPattern = tPattern.getPredicate();
+ Node oPattern = tPattern.getObject();
+
+ Node sData = tData.getSubject();
+ Node pData = tData.getPredicate();
+ Node oData = tData.getObject();
+
+ if ( ! match(bb, sData, sPattern) )
+ return false;
+ if ( ! match(bb, pData, pPattern) )
+ return false;
+ if ( ! match(bb, oData, oPattern) )
+ return false;
+ return true;
+ }
+
+ /** Match data "qData" against "tGraphNode, tPattern", including RDF-star terms. */
+ public static Binding matchQuad(Binding input, Quad qData, Node tGraphNode, Triple tPattern) {
+ BindingBuilder bb = Binding.builder(input);
+ boolean r = matchQuad(bb, qData, tGraphNode, tPattern);
+ return r ? bb.build() : null;
+ }
+
+ private static boolean matchQuad(BindingBuilder bb, Quad qData, Node tGraphNode, Triple tPattern) {
+ Node sPattern = tPattern.getSubject();
+ Node pPattern = tPattern.getPredicate();
+ Node oPattern = tPattern.getObject();
+
+ Node gData = qData.getGraph();
+ Node sData = qData.getSubject();
+ Node pData = qData.getPredicate();
+ Node oData = qData.getObject();
+
+ if ( ! match(bb, gData, tGraphNode) )
+ return false;
+ if ( ! match(bb, sData, sPattern) )
+ return false;
+ if ( ! match(bb, pData, pPattern) )
+ return false;
+ if ( ! match(bb, oData, oPattern) )
+ return false;
+ return true;
+ }
+
+ // RDF term matcher - may recurse and call matchTriple is a triple term is in the pattern.
+ private static boolean match(BindingBuilder bb, Node nData, Node nPattern) {
+ if ( nPattern == null )
+ return true;
+ if ( nData == Node.ANY )
+ return true;
+
+ // Deep substitute. This happens anyway as we walk structures.
+ // nPattern = Substitute.substitute(nPattern, input);
Review comment:
Extra space here, but not important.
##########
File path: jena-tdb/src/main/java/org/apache/jena/tdb/solver/SolverRX.java
##########
@@ -18,298 +18,136 @@
package org.apache.jena.tdb.solver;
-import static org.apache.jena.graph.Node_Triple.triple;
+import static org.apache.jena.tdb.solver.SolverLib.convFromBinding;
+import static org.apache.jena.tdb.solver.SolverLib.tripleHasNodeTriple;
import java.util.Iterator;
+import java.util.function.Function;
import java.util.function.Predicate;
import org.apache.jena.atlas.iterator.Iter;
-import org.apache.jena.atlas.lib.Pair;
+import org.apache.jena.atlas.lib.InternalErrorException;
import org.apache.jena.atlas.lib.tuple.Tuple;
import org.apache.jena.atlas.lib.tuple.TupleFactory;
-import org.apache.jena.atlas.logging.FmtLog;
import org.apache.jena.graph.Node;
-import org.apache.jena.graph.NodeFactory;
import org.apache.jena.graph.Triple;
-import org.apache.jena.sparql.ARQConstants;
-import org.apache.jena.sparql.core.Var;
-import org.apache.jena.sparql.core.VarAlloc;
+import org.apache.jena.sparql.core.Quad;
import org.apache.jena.sparql.engine.ExecutionContext;
+import org.apache.jena.sparql.engine.binding.Binding;
+import org.apache.jena.sparql.engine.binding.BindingFactory;
import org.apache.jena.sparql.engine.iterator.RX;
-import org.apache.jena.sparql.util.Context;
-import org.apache.jena.tdb.TDBException;
+import org.apache.jena.tdb.lib.TupleLib;
import org.apache.jena.tdb.store.NodeId;
import org.apache.jena.tdb.store.nodetable.NodeTable;
import org.apache.jena.tdb.store.nodetupletable.NodeTupleTable;
-/**
- * See {@link RX} which is the same algorithm for Triple/Node space.
- */
public class SolverRX {
-
- // These argument get passed around a lot, making the argument lists long.
- private static class Args {
- final NodeTupleTable nodeTupleTable;
- final boolean anyGraph;
- final Predicate<Tuple<NodeId>> filter;
- final ExecutionContext execCxt;
- final VarAlloc varAlloc;
- Args(NodeTupleTable nodeTupleTable, boolean anyGraph, Predicate<Tuple<NodeId>> filter, ExecutionContext execCxt) {
- super();
- this.nodeTupleTable = nodeTupleTable;
- this.anyGraph = anyGraph;
- this.filter = filter;
- this.execCxt = execCxt;
- this.varAlloc = varAlloc(execCxt);
+ // Only set 'false' in development
+ public // For integration testing only
+ static final boolean DATAPATH = true;
+
+ // Entry point from SolverLib.
+ /*package*/
+ static Iterator<BindingNodeId> matchQuadPattern(Iterator<BindingNodeId> chain, Node graphNode, Triple tPattern,
+ NodeTupleTable nodeTupleTable, Tuple<Node> patternTuple,
+ boolean anyGraph, Predicate<Tuple<NodeId>> filter, ExecutionContext execCxt) {
+ if ( DATAPATH ) {
+ if ( ! tripleHasNodeTriple(tPattern) || tPattern.isConcrete() ) {
+ // No RDF-star <<>> with variables.
+ return StageMatchTuple.access(nodeTupleTable, chain, patternTuple, filter, anyGraph, execCxt);
+ }
}
- }
-
- private static VarAlloc varAlloc(ExecutionContext execCxt) {
- Context context = execCxt.getContext();
- VarAlloc varAlloc = VarAlloc.get(context, ARQConstants.sysVarAllocRDFStar);
- if ( varAlloc == null ) {
- varAlloc = new VarAlloc(ARQConstants.allocVarTripleTerm);
- context.set(ARQConstants.sysVarAllocRDFStar, varAlloc);
- }
- return varAlloc;
- }
-
- // Call point for SolverLib.execute
- public static Iterator<BindingNodeId> solveRX(NodeTupleTable nodeTupleTable, Tuple<Node> pattern, boolean anyGraph,
- Iterator<BindingNodeId> chain, Predicate<Tuple<NodeId>> filter,
- ExecutionContext execCxt) {
- if ( ! tripleHasNodeTriple(pattern) )
- SolverLib.solve(nodeTupleTable, pattern, anyGraph, chain, filter, execCxt);
-
- Args args = new Args(nodeTupleTable, anyGraph, filter, execCxt);
- return rdfStarTriple(chain, pattern, args);
- }
-
- /**
- * Match a single triple pattern that may involve RDF-star terms. This is the top
- * level function for matching triples. The function {@link #matchTripleStar}
- * matches a triple term and assigns the triple matched to a variable. It is used
- * within {@link #rdfStarTriple} for nested triple term and a temporary allocated
- * variable as well can for {@code FIND(<<...>> AS ?t)}.
- *
- * @implNote
- * Without RDF-star, this would be a plain call of {@link #matchData} which
- * is simply a call to {@link SolverLib#solve}.
- */
- private static Iterator<BindingNodeId> rdfStarTriple(Iterator<BindingNodeId> input, Tuple<Node> pattern, Args args) {
- // Should all work without this trap for plain RDF.
- if ( ! tripleHasNodeTriple(pattern) )
- return matchData( input, pattern, args);
- return rdfStarTripleSub(input, pattern, args);
- }
-
- /**
- * Insert the stages necessary for a triple with triple pattern term inside it.
- * If the triple pattern has a triple term, possibly with variables, introduce
- * an iterator to solve for that, assign the matching triple term to a hidden
- * variable, and put allocated variable in to main triple pattern. Do for subject
- * and object positions, and also any nested triple pattern terms.
- */
- private static Iterator<BindingNodeId> rdfStarTripleSub(Iterator<BindingNodeId> input,
- Tuple<Node> pattern, Args args) {
- Pair<Iterator<BindingNodeId>, Tuple<Node>> pair = preprocessForTripleTerms(input, pattern, args);
- Iterator<BindingNodeId> chain2 = matchData(pair.getLeft(), pair.getRight(), args);
- return chain2;
- }
- /**
- * Match a triple pattern (which may have nested triple terms in it).
- * Any matched triples are added as triple terms bound to the supplied variable.
- */
- private static Iterator<BindingNodeId> matchTripleStar(Iterator<BindingNodeId> chain, Var var, Tuple<Node> pattern, Args args) {
- if ( tripleHasNodeTriple(pattern) ) {
- Pair<Iterator<BindingNodeId>, Tuple<Node>> pair =
- preprocessForTripleTerms(chain, pattern, args);
- chain = pair.getLeft();
- pattern = pair.getRight();
+ // RDF-star <<>> with variables.
+ // This path should work regardless.
+
+ boolean isTriple = (patternTuple.len() == 3);
+ NodeTable nodeTable = nodeTupleTable.getNodeTable();
+
+ Function<BindingNodeId, Iterator<BindingNodeId>> step =
+ bnid -> find(bnid, nodeTupleTable, graphNode, tPattern, anyGraph, filter, execCxt);
+ return Iter.flatMap(chain, step);
+ }
+
+ private static Iterator<BindingNodeId> find(BindingNodeId bnid, NodeTupleTable nodeTupleTable,
+ Node graphNode, Triple tPattern,
+ boolean anyGraph, Predicate<Tuple<NodeId>> filter,
+ ExecutionContext execCxt) {
+ Node tGraphNode = anyGraph ? Quad.unionGraph : graphNode ;
+ // graphNode is ANY for union graph and null for default graph.
+ // Var to ANY, Triple Term to ANY.
+
+ Node g = ( graphNode == null ) ? null : RX.nodeTopLevel(graphNode);
+ Node s = RX.nodeTopLevel(tPattern.getSubject());
+ Node p = RX.nodeTopLevel(tPattern.getPredicate());
+ Node o = RX.nodeTopLevel(tPattern.getObject());
+ NodeTable nodeTable = nodeTupleTable.getNodeTable();
+ Tuple<Node> patternTuple = ( g == null )
+ ? TupleFactory.create3(s,p,o)
+ : TupleFactory.create4(g,s,p,o);
+
+ Iterator<Quad> dsgIter = accessData(patternTuple, nodeTupleTable, anyGraph, filter, execCxt);
+
+ Binding input = bnid.isEmpty() ? BindingFactory.empty() : new BindingTDB(bnid, nodeTable);
+ Iterator<Binding> matched = Iter.iter(dsgIter).map(dQuad->RX.matchQuad(input, dQuad, tGraphNode, tPattern)).removeNulls();
+ return convFromBinding(matched, nodeTable);
+ }
+
+ static Iterator<Quad> accessData(Tuple<Node> patternTuple, NodeTupleTable nodeTupleTable,
+ boolean anyGraph, Predicate<Tuple<NodeId>> filter,
+ ExecutionContext execCxt) {
+ NodeTable nodeTable = nodeTupleTable.getNodeTable();
+ Function<Tuple<NodeId>, Quad> asQuad = asQuad(nodeTable, nodeTupleTable.getTupleLen(), anyGraph);
+ Tuple<NodeId> patternTupleId = TupleLib.tupleNodeIds(nodeTable, patternTuple);
+ if ( patternTupleId.contains(NodeId.NodeDoesNotExist) )
+ // Can not match.
+ return Iter.nullIterator();
+
+ // -- DRY/StageMatchTuple ??
+ Iterator<Tuple<NodeId>> iterMatches = nodeTupleTable.find(patternTupleId);
+ // Add filter
+ if ( filter != null )
+ iterMatches = Iter.filter(iterMatches, filter);
+ // Add anyGraph
+ if ( anyGraph ) {
+ // See StageMatchTuple for discussion.
+ iterMatches = Iter.map(iterMatches, quadsToAnyTriples);
+ iterMatches = Iter.distinctAdjacent(iterMatches);
}
- // Match to data and assign to var in each binding, based on the triple pattern grounded by the match.
- Iterator<BindingNodeId> qIter = bindTripleTerm(chain, var, pattern, args);
+ // -- DRY/StageMatchTuple
+ //Iterator<Quad> qIter = TupleLib.convertToQuads(nodeTable, iterMatches) ;
+ Iterator<Quad> qIter = Iter.map(iterMatches, asQuad);
return qIter;
}
- /**
- * Process a triple for triple terms.
- * <p>
- * This creates additional matchers for triple terms in the pattern triple recursively.
- */
- private static Pair<Iterator<BindingNodeId>, Tuple<Node>>
- preprocessForTripleTerms(Iterator<BindingNodeId> chain, Tuple<Node> patternTuple, Args args) {
- int sIdx = subjectIdx(patternTuple);
- int oIdx = objectIdx(patternTuple);
-
- Node subject = patternTuple.get(sIdx);
- Node object = patternTuple.get(oIdx);
- Node subject1 = null;
- Node object1 = null;
-
- if ( subject.isNodeTriple() ) {
- Triple tripleTerm = triple(subject);
- Var var = args.varAlloc.allocVar();
- patternTuple = createTuple(patternTuple, var, sIdx);
- Tuple<Node> patternTuple2 = tuple(patternTuple, tripleTerm);
- chain = matchTripleStar(chain, var, patternTuple2, args);
- subject1 = var;
- }
-
- if ( object.isNodeTriple() ) {
- Triple tripleTerm = triple(object);
- Var var = args.varAlloc.allocVar();
- patternTuple = createTuple(patternTuple, var, oIdx);
- Tuple<Node> patternTuple2 = tuple(patternTuple, tripleTerm);
- chain = matchTripleStar(chain, var, patternTuple2, args);
- object1 = var;
- }
-
- if ( subject1 == null && object1 == null )
- return Pair.create(chain, patternTuple);
- return Pair.create(chain, patternTuple);
- }
-
- /**
- * Add a binding to each row with triple grounded by the current row.
- * If the triple isn't concrete, then just return the row as-is.
- */
- private static Iterator<BindingNodeId> bindTripleTerm(Iterator<BindingNodeId> chain, Var var, Tuple<Node> pattern, Args args) {
- NodeTable nodeTable = args.nodeTupleTable.getNodeTable();
- chain = matchData(chain, pattern, args);
- // Add (var, triple term), filter no matches.
- chain = Iter.iter(chain).map(b->bindVarTripleTerm(var, pattern, b, nodeTable)).removeNulls();
- return chain;
- }
-
- // We need to reconstruct the reason the pattern matched
- // to find the NodeId for the Node_Triple.
- // This involves creating a Node_Triple and looking it up.
- // This isn't ideal but without triple ids in the database,
- // there isn't much we can do.
- private static BindingNodeId bindVarTripleTerm(Var var, Tuple<Node> pattern, BindingNodeId binding, NodeTable nodeTable) {
- // Get triple out of tuple of length 3 or 4.
- int idx = (pattern.len()==4) ? 1 : 0;
-
- // Access to Nodes.
- Node s = pattern.get(idx);
- Node s1 = substitute(s, binding, nodeTable);
- if ( s1 == null || ! s1.isConcrete() )
- return null;
-
- Node p = pattern.get(idx+1);
- Node p1 = substitute(p, binding, nodeTable);
- if ( p1 == null || ! p1.isConcrete() )
- return null;
-
- Node o = pattern.get(idx+2);
- Node o1 = substitute(o, binding, nodeTable);
- if ( o1 == null || ! o1.isConcrete() )
- return null;
-
- // Does it exist?
- Node t = NodeFactory.createTripleNode(s1,p1,o1);
- NodeId tid = nodeTable.getNodeIdForNode(t);
- // Should not happen.
- if ( NodeId.isDoesNotExist(tid) )
- return null;
- // Already bound (FIND)?
- if ( binding.containsKey(var) ) {
- NodeId tid2 = binding.get(var);
- if ( tid.equals(tid2) )
- return binding;
- return null;
- }
-
- BindingNodeId b2 = new BindingNodeId(binding);
- b2.put(var, tid);
- return b2;
- }
-
- private static Node substitute(Node node, BindingNodeId binding, NodeTable nodeTable) {
- if ( ! Var.isVar(node) )
- return node;
- Var var = Var.alloc(node);
- try {
- NodeId id = binding.get(var) ;
- if ( id == null )
- return null ;
- if ( NodeId.isDoesNotExist(id) )
- return null;
- Node n = nodeTable.getNodeForNodeId(id) ;
- if ( n == null )
- // But there was to put it in the BindingNodeId.
- throw new TDBException("No node in NodeTable for NodeId "+id);
- return n ;
- } catch (Exception ex)
- {
- FmtLog.error(SolverRX.class, ex, "SolverRX: substitute(%s) %s", node, binding) ;
- return null ;
- }
- }
-
- /**
- * Match the NodeTupleTable with a tuple pattern.
- * This is the accessor to the data.
- * It assumes any triple terms have been dealt with.
- */
-
- private static Iterator<BindingNodeId> matchData(Iterator<BindingNodeId> chain, Tuple<Node> pattern, Args args) {
- return SolverLib.solve(args.nodeTupleTable, pattern, args.anyGraph, chain, args.filter, args.execCxt);
- }
-
- private static Tuple<Node> createTuple(Tuple<Node> tuple, Var var, int idx) {
- switch(idx) {
- case 0: return TupleFactory.create3(var, tuple.get(1), tuple.get(2));
- case 1: return TupleFactory.create4(tuple.get(0), var, tuple.get(2), tuple.get(3));
- case 2: return TupleFactory.create3(tuple.get(0), tuple.get(1), var);
- case 3: return TupleFactory.create4(tuple.get(0), tuple.get(1), tuple.get(2), var);
+ private static Function<Tuple<NodeId>, Quad> asQuad(NodeTable nodeTable, int tupleLen, boolean anyGraph) {
+ switch (tupleLen) {
+ case 3:
+ return (Tuple<NodeId> t) -> {
+ Node gx = Quad.defaultGraphIRI;
+ Node sx = toNode(t.get(0), nodeTable);
+ Node px = toNode(t.get(1), nodeTable);
+ Node ox = toNode(t.get(2), nodeTable);
+ return Quad.create(gx, sx, px, ox);
+ };
+ case 4:
+ return (Tuple<NodeId> t) -> {
+ Node gx = (anyGraph)? Quad.unionGraph : toNode(t.get(0), nodeTable);
+ Node sx = toNode(t.get(1), nodeTable);
+ Node px = toNode(t.get(2), nodeTable);
+ Node ox = toNode(t.get(3), nodeTable);
+ return Quad.create(gx, sx, px, ox);
+ };
default:
- throw new TDBException("Index is not recognized: "+idx);
+ throw new InternalErrorException("Tuple of unknow length");
Review comment:
s/unknow/unknown
##########
File path: jena-db/jena-tdb2/src/main/java/org/apache/jena/tdb2/solver/Solver.java
##########
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.tdb2.solver;
+
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.List;
+import java.util.function.Predicate;
+
+import org.apache.jena.atlas.iterator.Iter;
+import org.apache.jena.atlas.lib.tuple.Tuple;
+import org.apache.jena.atlas.lib.tuple.TupleFactory;
+import org.apache.jena.graph.Node;
+import org.apache.jena.graph.Triple;
+import org.apache.jena.sparql.core.BasicPattern;
+import org.apache.jena.sparql.core.Quad;
+import org.apache.jena.sparql.engine.ExecutionContext;
+import org.apache.jena.sparql.engine.QueryIterator;
+import org.apache.jena.sparql.engine.binding.Binding;
+import org.apache.jena.sparql.engine.iterator.QueryIterPlainWrapper;
+import org.apache.jena.tdb2.TDBException;
+import org.apache.jena.tdb2.store.DatasetGraphTDB;
+import org.apache.jena.tdb2.store.GraphTDB;
+import org.apache.jena.tdb2.store.NodeId;
+import org.apache.jena.tdb2.store.nodetable.NodeTable;
+import org.apache.jena.tdb2.store.nodetupletable.NodeTupleTable;
+
+/** Entry to the basic pattern solver for TDB2 */
+public class Solver {
+
+ /** Non-reordering execution of a basic graph pattern, given a iterator of bindings as input */
+ public static QueryIterator execute(GraphTDB graph, BasicPattern pattern,
+ QueryIterator input, Predicate<Tuple<NodeId>> filter,
+ ExecutionContext execCxt)
+ {
+ // Maybe default graph or named graph.
+ NodeTupleTable ntt = graph.getNodeTupleTable();
+ return execute(ntt, graph.getGraphName(), pattern, input, filter, execCxt);
+ }
+
+ /** Non-reordering execution of a quad pattern, given a iterator of bindings as input.
Review comment:
a/a iterator/an iterator
##########
File path: jena-arq/src/main/java/org/apache/jena/sparql/util/FmtUtils.java
##########
@@ -42,11 +42,12 @@
/** Presentation forms of various kinds of objects.
* Beware that bNodes are abbreviated to _:b0 etc.
+ * @see NodeFmtLib
*/
public class FmtUtils
{
- // OLD CODE - being replaced by riot.NodeFmtLib
+ // See alsoriot.NodeFmtLib
Review comment:
space? Not really important
##########
File path: jena-arq/src/main/java/org/apache/jena/sparql/expr/ExprLib.java
##########
@@ -221,4 +223,23 @@ public void visit(ExprFunctionN func) {
@Override
public Throwable fillInStackTrace() { return this ; }
}
+
+ /** Go from a node to an expression. */
+ public static Expr nodeToExpr(Node n)
+ {
+ if ( n.isVariable() )
+ return new ExprVar(n) ;
+ if ( n.isNodeTriple() ) {
+ Node_Triple tripleTerm = (Node_Triple)n;
+ return new ExprTripleTerm(tripleTerm);
+ }
+ return NodeValue.makeNode(n) ;
+ }
Review comment:
Extra spaces indented. Not sure if intentional.
##########
File path: jena-arq/src/main/java/org/apache/jena/sparql/engine/iterator/RX_PG.java
##########
@@ -0,0 +1,183 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.engine.iterator;
+
+import org.apache.jena.atlas.lib.Pair;
+import org.apache.jena.graph.Node;
+import org.apache.jena.graph.Triple;
+import org.apache.jena.sparql.ARQConstants;
+import org.apache.jena.sparql.core.Var;
+import org.apache.jena.sparql.core.VarAlloc;
+import org.apache.jena.sparql.engine.ExecutionContext;
+import org.apache.jena.sparql.engine.QueryIterator;
+import org.apache.jena.sparql.util.Context;
+
+/**
+ * For reference only: this code uses the triple index for matchign and it is PG mode..
Review comment:
s/matchign/matching (also extra period at the end of sentence?)
##########
File path: jena-arq/src/main/java/org/apache/jena/sparql/sse/builders/BuilderExpr.java
##########
@@ -1253,168 +767,261 @@ public Expr make(ItemList list)
// (count distinct ?var)
// Need a canonical name for the variable that will be set by the aggregation.
// Aggregator.getVarName.
-
- static boolean startsWithDistinct(ItemList x)
- {
+
+ private static boolean startsWithDistinct(ItemList x) {
if ( x.size() > 0 && x.car().isSymbol(Tags.tagDistinct) )
- return true ;
- return false ;
+ return true;
+ return false;
}
-
+
// All the one expression cases
- static abstract class BuildAggCommon implements Build
- {
+ private static abstract class BuildAggCommon implements Build {
@Override
- public final Expr make(ItemList list)
- {
+ public Expr make(ItemList list) {
ItemList x = list.cdr(); // drop "sum"
- boolean distinct = startsWithDistinct(x) ;
+ boolean distinct = startsWithDistinct(x);
if ( distinct )
x = x.cdr();
- BuilderLib.checkLength(1, x, "Broken syntax: "+list.shortString()) ;
- // (sum ?var)
- Expr expr = buildExpr(x.get(0)) ;
- return make(distinct, expr) ;
+ BuilderLib.checkLength(1, x, "Broken syntax: "+list.shortString());
+ // (sum ?var)
+ Expr expr = buildExpr(x.get(0));
+ return make(distinct, expr);
}
- public abstract Expr make(boolean distinct, Expr expr) ;
+ public abstract Expr make(boolean distinct, Expr expr);
}
-
- final protected Build buildCount = new Build()
- {
- @Override
- public Expr make(final ItemList list)
- {
- ItemList x = list.cdr(); // drop "count"
- boolean distinct = startsWithDistinct(x) ;
- if ( distinct )
- x = x.cdr();
-
- BuilderLib.checkLength(0, 1, x, "Broken syntax: "+list.shortString()) ;
-
- Aggregator agg = null ;
- if ( x.size() == 0 )
- agg = AggregatorFactory.createCount(distinct) ;
- else
- {
- Expr expr = BuilderExpr.buildExpr(x.get(0)) ;
- agg = AggregatorFactory.createCountExpr(distinct, expr) ;
- }
- return new ExprAggregator(null, agg) ;
- }
- };
-
- final protected Build buildSum = new BuildAggCommon()
- {
- @Override
- public Expr make(boolean distinct, Expr expr)
- {
- Aggregator agg = AggregatorFactory.createSum(distinct, expr) ;
- return new ExprAggregator(null, agg) ;
- }
- };
-
- final protected Build buildMin = new BuildAggCommon()
- {
- @Override
- public Expr make(boolean distinct, Expr expr)
- {
- Aggregator agg = AggregatorFactory.createMin(distinct, expr) ;
- return new ExprAggregator(null, agg) ;
- }
- };
-
- final protected Build buildMax = new BuildAggCommon()
- {
- @Override
- public Expr make(boolean distinct, Expr expr)
- {
- Aggregator agg = AggregatorFactory.createMax(distinct, expr) ;
- return new ExprAggregator(null, agg) ;
- }
- };
- final protected Build buildAvg = new BuildAggCommon()
- {
- @Override
- public Expr make(boolean distinct, Expr expr)
- {
- Aggregator agg = AggregatorFactory.createAvg(distinct, expr) ;
- return new ExprAggregator(null, agg) ;
- }
- };
+ private static Build buildCount = (ItemList list) -> {
+ ItemList x = list.cdr(); // drop "count"
+ boolean distinct = startsWithDistinct(x);
+ if ( distinct )
+ x = x.cdr();
- final protected Build buildSample = new BuildAggCommon()
- {
- @Override
- public Expr make(boolean distinct, Expr expr)
- {
- Aggregator agg = AggregatorFactory.createSample(distinct, expr) ;
- return new ExprAggregator(null, agg) ;
- }
+ BuilderLib.checkLength(0, 1, x, "Broken syntax: " + list.shortString());
+
+ Aggregator agg = null;
+ if ( x.size() == 0 )
+ agg = AggregatorFactory.createCount(distinct);
+ else {
+ Expr expr = BuilderExpr.buildExpr(x.get(0));
+ agg = AggregatorFactory.createCountExpr(distinct, expr);
+ }
+ return new ExprAggregator(null, agg);
};
-
- final protected Build buildGroupConcat = new Build()
- {
- @Override
- public Expr make(final ItemList list)
- {
- ItemList x = list.cdr(); // drop "group_concat"
- boolean distinct = startsWithDistinct(x) ;
+
+ private interface BuildAgg { Expr aggExpr(boolean distinct, Expr arg); }
+
+ private static Build aggCommonOne(BuildAgg make) {
+ return (ItemList list) -> {
+ ItemList x = list.cdr(); // drop "sum", "min" etc
+ boolean distinct = startsWithDistinct(x);
if ( distinct )
x = x.cdr();
-
- // Complex syntax:
- // (groupConcat (separator "string) expr )
- if ( x.size() == 0 )
- BuilderLib.broken(list, "Broken syntax: "+list.shortString()) ;
- String separator = null ;
- if ( x.get(0).isTagged(Tags.tagSeparator))
- {
- // What about ORDERED BY
- ItemList y = x.get(0).getList() ;
- BuilderLib.checkLength(2, y, "Broken syntax: "+list) ;
- Node n = y.get(1).getNode() ;
- if ( ! NodeUtils.isSimpleString(n) )
- BuilderLib.broken(y, "Need string for separator: "+y) ;
- separator = n.getLiteralLexicalForm() ;
- x = x.cdr();
- }
-
- Expr expr = buildExpr(x.get(0)) ;
- Aggregator agg = AggregatorFactory.createGroupConcat(distinct, expr, separator, null) ;
- return new ExprAggregator(null, agg) ;
+ BuilderLib.checkLength(1, x, "Broken syntax: "+list.shortString());
+ // (sum ?var)
+ Expr expr = buildExpr(x.get(0));
+ return make.aggExpr(distinct, expr);
+ };
+ }
+
+ private static Build buildSum = aggCommonOne((distinct, expr)->{
+ Aggregator agg = AggregatorFactory.createSum(distinct, expr);
+ return new ExprAggregator(null, agg);
+ });
+
+ private static Build buildMin = aggCommonOne((distinct, expr)->{
+ Aggregator agg = AggregatorFactory.createMin(distinct, expr);
+ return new ExprAggregator(null, agg);
+ });
+
+ private static Build buildMax = aggCommonOne((distinct, expr)->{
+ Aggregator agg = AggregatorFactory.createMax(distinct, expr);
+ return new ExprAggregator(null, agg);
+ });
+
+ private static Build buildAvg = aggCommonOne((distinct, expr)->{
+ Aggregator agg = AggregatorFactory.createAvg(distinct, expr);
+ return new ExprAggregator(null, agg);
+ });
+
+ private static Build buildSample = aggCommonOne((distinct, expr)->{
+ Aggregator agg = AggregatorFactory.createSample(distinct, expr);
+ return new ExprAggregator(null, agg);
+ });
+
+ private static Build buildGroupConcat = (ItemList list)-> {
+ ItemList x = list.cdr(); // drop "group_concat"
+ boolean distinct = startsWithDistinct(x);
+ if ( distinct )
+ x = x.cdr();
+
+ // Complex syntax:
+ // (groupConcat (separator "string) expr )
+ if ( x.size() == 0 )
+ BuilderLib.broken(list, "Broken syntax: "+list.shortString());
+ String separator = null;
+ if ( x.get(0).isTagged(Tags.tagSeparator))
+ {
+ // What about ORDERED BY
+ ItemList y = x.get(0).getList();
+ BuilderLib.checkLength(2, y, "Broken syntax: "+list);
+ Node n = y.get(1).getNode();
+ if ( ! NodeUtils.isSimpleString(n) )
+ BuilderLib.broken(y, "Need string for separator: "+y);
+ separator = n.getLiteralLexicalForm();
+ x = x.cdr();
}
+
+ Expr expr = buildExpr(x.get(0));
+ Aggregator agg = AggregatorFactory.createGroupConcat(distinct, expr, separator, null);
+ return new ExprAggregator(null, agg);
};
-
- final protected Build buildCustomAggregate = new Build() {
- @Override
- public Expr make(final ItemList list)
- {
- ItemList x = list.cdr(); // drop "agg"
- if ( x.size() == 0 )
- BuilderLib.broken(list, "Missing IRI for aggregate") ;
-
- Item z = x.car() ;
- if ( ! z.isNodeURI() )
- BuilderLib.broken(list, "Not an IRI for aggregate: "+z) ;
- x = x.cdr() ;
- boolean distinct = startsWithDistinct(x) ;
- if ( distinct )
- x = x.cdr();
- ExprList e = buildExprListUntagged(x, 0) ;
- Aggregator agg = AggregatorFactory.createCustom(z.getNode().getURI(), distinct, e) ;
- return new ExprAggregator(null, agg) ;
- }
- } ;
-
- final protected Build buildAggNull = new Build()
- {
- @Override
- public Expr make(ItemList list)
- {
- BuilderLib.checkLength(1, list, "Broken syntax: "+list.shortString()) ;
- return new ExprAggregator(null, AggregatorFactory.createAggNull()) ;
- }
+
+ private static Build buildCustomAggregate = (ItemList list) -> {
+ ItemList x = list.cdr(); // drop "agg"
+ if ( x.size() == 0 )
+ BuilderLib.broken(list, "Missing IRI for aggregate");
+
+ Item z = x.car();
+ if ( ! z.isNodeURI() )
+ BuilderLib.broken(list, "Not an IRI for aggregate: "+z);
+ x = x.cdr();
+ boolean distinct = startsWithDistinct(x);
+ if ( distinct )
+ x = x.cdr();
+ ExprList e = buildExprListUntagged(x, 0);
+ Aggregator agg = AggregatorFactory.createCustom(z.getNode().getURI(), distinct, e);
+ return new ExprAggregator(null, agg);
};
+
+ private static Build buildAggNull = (ItemList list) -> {
+ BuilderLib.checkLength(1, list, "Broken syntax: "+list.shortString());
+ return new ExprAggregator(null, AggregatorFactory.createAggNull());
+ };
+
+ private static Map<String, Build> createDispatchTable() {
+ Map<String, Build> dispatch = new HashMap<>();
+ dispatch.put(Tags.tagRegex, buildRegex);
+ dispatch.put(Tags.symEQ, buildEQ);
+ dispatch.put(Tags.tagEQ, buildEQ);
+ dispatch.put(Tags.symNE, buildNE);
+ dispatch.put(Tags.tagNE, buildNE);
+ dispatch.put(Tags.symGT, buildGT);
+ dispatch.put(Tags.tagGT, buildGT);
+ dispatch.put(Tags.symLT, buildLT);
+ dispatch.put(Tags.tagLT, buildLT);
+ dispatch.put(Tags.symLE, buildLE);
+ dispatch.put(Tags.tagLE, buildLE);
+ dispatch.put(Tags.symGE, buildGE);
+ dispatch.put(Tags.tagGE, buildGE);
+ dispatch.put(Tags.symOr, buildOr);
+ dispatch.put(Tags.tagOr, buildOr);
+ dispatch.put(Tags.symAnd, buildAnd);
+ dispatch.put(Tags.tagAnd, buildAnd);
+ dispatch.put(Tags.symPlus, buildPlus);
+ dispatch.put(Tags.tagAdd, buildPlus);
+ dispatch.put(Tags.symMinus, buildMinus);
+ dispatch.put(Tags.tagSubtract, buildMinus);
+ dispatch.put(Tags.tagMinus, buildMinus); // Not to be confused with Op for SPARQL MINUS
Review comment:
Looked up at the `Tags` class to see if there was an `Tags.tagPlus`… just because I thought that as `symMinus` is for `symPlus`, and `tagSubtract` is for `tagAdd`, then `tagMinus` would be for `tagPlus` (there's even an unary plus) :laughing:
##########
File path: jena-arq/src/main/java/org/apache/jena/sparql/engine/iterator/RX.java
##########
@@ -18,145 +18,43 @@
package org.apache.jena.sparql.engine.iterator;
-import static org.apache.jena.graph.Node_Triple.triple;
+import java.util.Iterator;
+import java.util.Objects;
-import org.apache.jena.atlas.lib.Pair;
+import org.apache.jena.atlas.iterator.Iter;
+import org.apache.jena.graph.Graph;
import org.apache.jena.graph.Node;
import org.apache.jena.graph.Triple;
-import org.apache.jena.sparql.ARQConstants;
+import org.apache.jena.sparql.core.Quad;
import org.apache.jena.sparql.core.Var;
-import org.apache.jena.sparql.core.VarAlloc;
import org.apache.jena.sparql.engine.ExecutionContext;
import org.apache.jena.sparql.engine.QueryIterator;
-import org.apache.jena.sparql.util.Context;
-
-/**
- * Solver library for RDF-star.
- * <p>
- * There are two entry points.
- * <p>
- * Function {@link #rdfStarTriple} for matching a single triple pattern in a basic
- * graph pattern that may involve RDF-star terms.
- * <p>
- * Function {@link #matchTripleStar} for matches a triple term and assigning the
- * triple matched to a variable. It is used within {@link #rdfStarTriple} for nested
- * triple term and a temporary allocated variable as well can for
- * {@code FIND(<<...>> AS ?t)}.
- */
+import org.apache.jena.sparql.engine.binding.Binding;
+import org.apache.jena.sparql.engine.binding.BindingBuilder;
+import org.apache.jena.util.iterator.ExtendedIterator;
+
+/** RDF-star - when there is an embedded triple with variables. */
public class RX {
+ // Note: running with DATAPATH = false
- /**
- * Match a single triple pattern that may involve RDF-star terms.
- * This is the top level function for matching triples.
- *
- * The function {@link #matchTripleStar} matches a triple term and assigns the triple matched to a variable.
- * It is used within {@link #rdfStarTriple} for nested triple term and a temporary allocated variable
- * as well can for {@code FIND(<<...>> AS ?t)}.
- *
- * @implNote
- * Without RDF-star, this would be a plain call of {@link #matchData} which is simply:
- * <pre>
- * new QueryIterTriplePattern(chain, triple, execCxt)}
- * </pre>
- */
- public static QueryIterator rdfStarTriple(QueryIterator chain, Triple triple, ExecutionContext execCxt) {
- // Should all work without this trap for plain RDF.
- if ( ! tripleHasNodeTriple(triple) )
- // No RDF-star : direct to data.
- return matchData(chain, triple, execCxt);
- return rdfStarTripleSub(chain, triple, execCxt);
- }
+ // GraphMem is sameValueAs (historical)
+ // and tests use GraphPlain, which is langtag case insensitive (There is one DAWG test that needs this).
- private static VarAlloc varAlloc(ExecutionContext execCxt) {
- Context context = execCxt.getContext();
- VarAlloc varAlloc = VarAlloc.get(context, ARQConstants.sysVarAllocRDFStar);
- if ( varAlloc == null ) {
- varAlloc = new VarAlloc(ARQConstants.allocVarTripleTerm);
- context.set(ARQConstants.sysVarAllocRDFStar, varAlloc);
- }
- return varAlloc;
- }
+ // If running in development with "DATAPATH = false" two failures ("lang-3", run twice).
- /**
- * Insert the stages necessary for a triple with triple pattern term inside it.
- * If the triple pattern has a triple term, possibly with variables, introduce
- * an iterator to solve for that, assign the matching triple term to a hidden
- * variable, and put allocated variable in to main triple pattern. Do for subject
- * and object positions, and also any nested triple pattern terms.
- */
- private static QueryIterator rdfStarTripleSub(QueryIterator chain, Triple triple, ExecutionContext execCxt) {
- Pair<QueryIterator, Triple> pair = preprocessForTripleTerms(chain, triple, execCxt);
- QueryIterator chain2 = matchData(pair.getLeft(), pair.getRight(), execCxt);
- return chain2;
- }
+ // Normally true.
+ public // For integration testing only
+ static final boolean DATAPATH = true;
Review comment:
:+1: or should we make it into a Javadoc code so users don't use it? I remember someone asking about some code recently-ish in the mailing list that changed… I think the code was not supposed to be used externally, or maybe there was an alternative but it wasn't clear to the user.
##########
File path: jena-arq/src/main/java/org/apache/jena/riot/lang/extra/javacc/TurtleJavacc.java
##########
@@ -369,8 +366,34 @@ final public void Annotation(Node s, Node p, Node o) throws ParseException {
throw new Error("Missing return statement in function");
}
-// RDF-star objectXTurtle [12] object
- final public Node ObjectX() throws ParseException {Node o ; String iri;
+ final public Node EmbSubject() throws ParseException {Node o ; String iri;
+ switch ((jj_ntk==-1)?jj_ntk_f():jj_ntk) {
+ case IRIref:
+ case PNAME_NS:
+ case PNAME_LN:{
+ iri = iri();
+o = createNode(iri, token.beginLine, token.beginColumn) ;
Review comment:
Is this auto-generated? If not maybe we can fix the indentation here
##########
File path: jena-arq/src/main/java/org/apache/jena/sparql/expr/ExprTripleTerm.java
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.expr;
+
+import java.util.Objects;
+
+import org.apache.jena.graph.Node;
+import org.apache.jena.graph.Node_Triple;
+import org.apache.jena.graph.Triple;
+import org.apache.jena.sparql.core.Substitute;
+import org.apache.jena.sparql.engine.binding.Binding ;
+import org.apache.jena.sparql.function.FunctionEnv ;
+import org.apache.jena.sparql.graph.NodeTransform ;
+
+/**
+ * RDF-start triple term in an expression.
Review comment:
s/RDF-start/star (worth looking for RDF-start in the IDE if using one; that's one error that spell/grammar checkers would miss I guess)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a change in pull request #951: JENA-2065: RDF-star
Posted by GitBox <gi...@apache.org>.
afs commented on a change in pull request #951:
URL: https://github.com/apache/jena/pull/951#discussion_r591295697
##########
File path: jena-tdb/src/main/java/org/apache/jena/tdb/solver/SolverRX.java
##########
@@ -18,298 +18,136 @@
package org.apache.jena.tdb.solver;
-import static org.apache.jena.graph.Node_Triple.triple;
+import static org.apache.jena.tdb.solver.SolverLib.convFromBinding;
+import static org.apache.jena.tdb.solver.SolverLib.tripleHasNodeTriple;
import java.util.Iterator;
+import java.util.function.Function;
import java.util.function.Predicate;
import org.apache.jena.atlas.iterator.Iter;
-import org.apache.jena.atlas.lib.Pair;
+import org.apache.jena.atlas.lib.InternalErrorException;
import org.apache.jena.atlas.lib.tuple.Tuple;
import org.apache.jena.atlas.lib.tuple.TupleFactory;
-import org.apache.jena.atlas.logging.FmtLog;
import org.apache.jena.graph.Node;
-import org.apache.jena.graph.NodeFactory;
import org.apache.jena.graph.Triple;
-import org.apache.jena.sparql.ARQConstants;
-import org.apache.jena.sparql.core.Var;
-import org.apache.jena.sparql.core.VarAlloc;
+import org.apache.jena.sparql.core.Quad;
import org.apache.jena.sparql.engine.ExecutionContext;
+import org.apache.jena.sparql.engine.binding.Binding;
+import org.apache.jena.sparql.engine.binding.BindingFactory;
import org.apache.jena.sparql.engine.iterator.RX;
-import org.apache.jena.sparql.util.Context;
-import org.apache.jena.tdb.TDBException;
+import org.apache.jena.tdb.lib.TupleLib;
import org.apache.jena.tdb.store.NodeId;
import org.apache.jena.tdb.store.nodetable.NodeTable;
import org.apache.jena.tdb.store.nodetupletable.NodeTupleTable;
-/**
- * See {@link RX} which is the same algorithm for Triple/Node space.
- */
public class SolverRX {
-
- // These argument get passed around a lot, making the argument lists long.
- private static class Args {
- final NodeTupleTable nodeTupleTable;
- final boolean anyGraph;
- final Predicate<Tuple<NodeId>> filter;
- final ExecutionContext execCxt;
- final VarAlloc varAlloc;
- Args(NodeTupleTable nodeTupleTable, boolean anyGraph, Predicate<Tuple<NodeId>> filter, ExecutionContext execCxt) {
- super();
- this.nodeTupleTable = nodeTupleTable;
- this.anyGraph = anyGraph;
- this.filter = filter;
- this.execCxt = execCxt;
- this.varAlloc = varAlloc(execCxt);
+ // Only set 'false' in development
+ public // For integration testing only
+ static final boolean DATAPATH = true;
+
+ // Entry point from SolverLib.
+ /*package*/
+ static Iterator<BindingNodeId> matchQuadPattern(Iterator<BindingNodeId> chain, Node graphNode, Triple tPattern,
+ NodeTupleTable nodeTupleTable, Tuple<Node> patternTuple,
+ boolean anyGraph, Predicate<Tuple<NodeId>> filter, ExecutionContext execCxt) {
+ if ( DATAPATH ) {
+ if ( ! tripleHasNodeTriple(tPattern) || tPattern.isConcrete() ) {
+ // No RDF-star <<>> with variables.
+ return StageMatchTuple.access(nodeTupleTable, chain, patternTuple, filter, anyGraph, execCxt);
+ }
}
- }
-
- private static VarAlloc varAlloc(ExecutionContext execCxt) {
- Context context = execCxt.getContext();
- VarAlloc varAlloc = VarAlloc.get(context, ARQConstants.sysVarAllocRDFStar);
- if ( varAlloc == null ) {
- varAlloc = new VarAlloc(ARQConstants.allocVarTripleTerm);
- context.set(ARQConstants.sysVarAllocRDFStar, varAlloc);
- }
- return varAlloc;
- }
-
- // Call point for SolverLib.execute
- public static Iterator<BindingNodeId> solveRX(NodeTupleTable nodeTupleTable, Tuple<Node> pattern, boolean anyGraph,
- Iterator<BindingNodeId> chain, Predicate<Tuple<NodeId>> filter,
- ExecutionContext execCxt) {
- if ( ! tripleHasNodeTriple(pattern) )
- SolverLib.solve(nodeTupleTable, pattern, anyGraph, chain, filter, execCxt);
-
- Args args = new Args(nodeTupleTable, anyGraph, filter, execCxt);
- return rdfStarTriple(chain, pattern, args);
- }
-
- /**
- * Match a single triple pattern that may involve RDF-star terms. This is the top
- * level function for matching triples. The function {@link #matchTripleStar}
- * matches a triple term and assigns the triple matched to a variable. It is used
- * within {@link #rdfStarTriple} for nested triple term and a temporary allocated
- * variable as well can for {@code FIND(<<...>> AS ?t)}.
- *
- * @implNote
- * Without RDF-star, this would be a plain call of {@link #matchData} which
- * is simply a call to {@link SolverLib#solve}.
- */
- private static Iterator<BindingNodeId> rdfStarTriple(Iterator<BindingNodeId> input, Tuple<Node> pattern, Args args) {
- // Should all work without this trap for plain RDF.
- if ( ! tripleHasNodeTriple(pattern) )
- return matchData( input, pattern, args);
- return rdfStarTripleSub(input, pattern, args);
- }
-
- /**
- * Insert the stages necessary for a triple with triple pattern term inside it.
- * If the triple pattern has a triple term, possibly with variables, introduce
- * an iterator to solve for that, assign the matching triple term to a hidden
- * variable, and put allocated variable in to main triple pattern. Do for subject
- * and object positions, and also any nested triple pattern terms.
- */
- private static Iterator<BindingNodeId> rdfStarTripleSub(Iterator<BindingNodeId> input,
- Tuple<Node> pattern, Args args) {
- Pair<Iterator<BindingNodeId>, Tuple<Node>> pair = preprocessForTripleTerms(input, pattern, args);
- Iterator<BindingNodeId> chain2 = matchData(pair.getLeft(), pair.getRight(), args);
- return chain2;
- }
- /**
- * Match a triple pattern (which may have nested triple terms in it).
- * Any matched triples are added as triple terms bound to the supplied variable.
- */
- private static Iterator<BindingNodeId> matchTripleStar(Iterator<BindingNodeId> chain, Var var, Tuple<Node> pattern, Args args) {
- if ( tripleHasNodeTriple(pattern) ) {
- Pair<Iterator<BindingNodeId>, Tuple<Node>> pair =
- preprocessForTripleTerms(chain, pattern, args);
- chain = pair.getLeft();
- pattern = pair.getRight();
+ // RDF-star <<>> with variables.
+ // This path should work regardless.
+
+ boolean isTriple = (patternTuple.len() == 3);
+ NodeTable nodeTable = nodeTupleTable.getNodeTable();
+
+ Function<BindingNodeId, Iterator<BindingNodeId>> step =
+ bnid -> find(bnid, nodeTupleTable, graphNode, tPattern, anyGraph, filter, execCxt);
+ return Iter.flatMap(chain, step);
+ }
+
+ private static Iterator<BindingNodeId> find(BindingNodeId bnid, NodeTupleTable nodeTupleTable,
+ Node graphNode, Triple tPattern,
+ boolean anyGraph, Predicate<Tuple<NodeId>> filter,
+ ExecutionContext execCxt) {
+ Node tGraphNode = anyGraph ? Quad.unionGraph : graphNode ;
+ // graphNode is ANY for union graph and null for default graph.
+ // Var to ANY, Triple Term to ANY.
+
+ Node g = ( graphNode == null ) ? null : RX.nodeTopLevel(graphNode);
+ Node s = RX.nodeTopLevel(tPattern.getSubject());
+ Node p = RX.nodeTopLevel(tPattern.getPredicate());
+ Node o = RX.nodeTopLevel(tPattern.getObject());
+ NodeTable nodeTable = nodeTupleTable.getNodeTable();
+ Tuple<Node> patternTuple = ( g == null )
+ ? TupleFactory.create3(s,p,o)
+ : TupleFactory.create4(g,s,p,o);
+
+ Iterator<Quad> dsgIter = accessData(patternTuple, nodeTupleTable, anyGraph, filter, execCxt);
+
+ Binding input = bnid.isEmpty() ? BindingFactory.empty() : new BindingTDB(bnid, nodeTable);
+ Iterator<Binding> matched = Iter.iter(dsgIter).map(dQuad->RX.matchQuad(input, dQuad, tGraphNode, tPattern)).removeNulls();
+ return convFromBinding(matched, nodeTable);
+ }
+
+ static Iterator<Quad> accessData(Tuple<Node> patternTuple, NodeTupleTable nodeTupleTable,
+ boolean anyGraph, Predicate<Tuple<NodeId>> filter,
+ ExecutionContext execCxt) {
+ NodeTable nodeTable = nodeTupleTable.getNodeTable();
+ Function<Tuple<NodeId>, Quad> asQuad = asQuad(nodeTable, nodeTupleTable.getTupleLen(), anyGraph);
+ Tuple<NodeId> patternTupleId = TupleLib.tupleNodeIds(nodeTable, patternTuple);
+ if ( patternTupleId.contains(NodeId.NodeDoesNotExist) )
+ // Can not match.
+ return Iter.nullIterator();
+
+ // -- DRY/StageMatchTuple ??
+ Iterator<Tuple<NodeId>> iterMatches = nodeTupleTable.find(patternTupleId);
+ // Add filter
+ if ( filter != null )
+ iterMatches = Iter.filter(iterMatches, filter);
+ // Add anyGraph
+ if ( anyGraph ) {
+ // See StageMatchTuple for discussion.
+ iterMatches = Iter.map(iterMatches, quadsToAnyTriples);
+ iterMatches = Iter.distinctAdjacent(iterMatches);
}
- // Match to data and assign to var in each binding, based on the triple pattern grounded by the match.
- Iterator<BindingNodeId> qIter = bindTripleTerm(chain, var, pattern, args);
+ // -- DRY/StageMatchTuple
+ //Iterator<Quad> qIter = TupleLib.convertToQuads(nodeTable, iterMatches) ;
+ Iterator<Quad> qIter = Iter.map(iterMatches, asQuad);
return qIter;
}
- /**
- * Process a triple for triple terms.
- * <p>
- * This creates additional matchers for triple terms in the pattern triple recursively.
- */
- private static Pair<Iterator<BindingNodeId>, Tuple<Node>>
- preprocessForTripleTerms(Iterator<BindingNodeId> chain, Tuple<Node> patternTuple, Args args) {
- int sIdx = subjectIdx(patternTuple);
- int oIdx = objectIdx(patternTuple);
-
- Node subject = patternTuple.get(sIdx);
- Node object = patternTuple.get(oIdx);
- Node subject1 = null;
- Node object1 = null;
-
- if ( subject.isNodeTriple() ) {
- Triple tripleTerm = triple(subject);
- Var var = args.varAlloc.allocVar();
- patternTuple = createTuple(patternTuple, var, sIdx);
- Tuple<Node> patternTuple2 = tuple(patternTuple, tripleTerm);
- chain = matchTripleStar(chain, var, patternTuple2, args);
- subject1 = var;
- }
-
- if ( object.isNodeTriple() ) {
- Triple tripleTerm = triple(object);
- Var var = args.varAlloc.allocVar();
- patternTuple = createTuple(patternTuple, var, oIdx);
- Tuple<Node> patternTuple2 = tuple(patternTuple, tripleTerm);
- chain = matchTripleStar(chain, var, patternTuple2, args);
- object1 = var;
- }
-
- if ( subject1 == null && object1 == null )
- return Pair.create(chain, patternTuple);
- return Pair.create(chain, patternTuple);
- }
-
- /**
- * Add a binding to each row with triple grounded by the current row.
- * If the triple isn't concrete, then just return the row as-is.
- */
- private static Iterator<BindingNodeId> bindTripleTerm(Iterator<BindingNodeId> chain, Var var, Tuple<Node> pattern, Args args) {
- NodeTable nodeTable = args.nodeTupleTable.getNodeTable();
- chain = matchData(chain, pattern, args);
- // Add (var, triple term), filter no matches.
- chain = Iter.iter(chain).map(b->bindVarTripleTerm(var, pattern, b, nodeTable)).removeNulls();
- return chain;
- }
-
- // We need to reconstruct the reason the pattern matched
- // to find the NodeId for the Node_Triple.
- // This involves creating a Node_Triple and looking it up.
- // This isn't ideal but without triple ids in the database,
- // there isn't much we can do.
- private static BindingNodeId bindVarTripleTerm(Var var, Tuple<Node> pattern, BindingNodeId binding, NodeTable nodeTable) {
- // Get triple out of tuple of length 3 or 4.
- int idx = (pattern.len()==4) ? 1 : 0;
-
- // Access to Nodes.
- Node s = pattern.get(idx);
- Node s1 = substitute(s, binding, nodeTable);
- if ( s1 == null || ! s1.isConcrete() )
- return null;
-
- Node p = pattern.get(idx+1);
- Node p1 = substitute(p, binding, nodeTable);
- if ( p1 == null || ! p1.isConcrete() )
- return null;
-
- Node o = pattern.get(idx+2);
- Node o1 = substitute(o, binding, nodeTable);
- if ( o1 == null || ! o1.isConcrete() )
- return null;
-
- // Does it exist?
- Node t = NodeFactory.createTripleNode(s1,p1,o1);
- NodeId tid = nodeTable.getNodeIdForNode(t);
- // Should not happen.
- if ( NodeId.isDoesNotExist(tid) )
- return null;
- // Already bound (FIND)?
- if ( binding.containsKey(var) ) {
- NodeId tid2 = binding.get(var);
- if ( tid.equals(tid2) )
- return binding;
- return null;
- }
-
- BindingNodeId b2 = new BindingNodeId(binding);
- b2.put(var, tid);
- return b2;
- }
-
- private static Node substitute(Node node, BindingNodeId binding, NodeTable nodeTable) {
- if ( ! Var.isVar(node) )
- return node;
- Var var = Var.alloc(node);
- try {
- NodeId id = binding.get(var) ;
- if ( id == null )
- return null ;
- if ( NodeId.isDoesNotExist(id) )
- return null;
- Node n = nodeTable.getNodeForNodeId(id) ;
- if ( n == null )
- // But there was to put it in the BindingNodeId.
- throw new TDBException("No node in NodeTable for NodeId "+id);
- return n ;
- } catch (Exception ex)
- {
- FmtLog.error(SolverRX.class, ex, "SolverRX: substitute(%s) %s", node, binding) ;
- return null ;
- }
- }
-
- /**
- * Match the NodeTupleTable with a tuple pattern.
- * This is the accessor to the data.
- * It assumes any triple terms have been dealt with.
- */
-
- private static Iterator<BindingNodeId> matchData(Iterator<BindingNodeId> chain, Tuple<Node> pattern, Args args) {
- return SolverLib.solve(args.nodeTupleTable, pattern, args.anyGraph, chain, args.filter, args.execCxt);
- }
-
- private static Tuple<Node> createTuple(Tuple<Node> tuple, Var var, int idx) {
- switch(idx) {
- case 0: return TupleFactory.create3(var, tuple.get(1), tuple.get(2));
- case 1: return TupleFactory.create4(tuple.get(0), var, tuple.get(2), tuple.get(3));
- case 2: return TupleFactory.create3(tuple.get(0), tuple.get(1), var);
- case 3: return TupleFactory.create4(tuple.get(0), tuple.get(1), tuple.get(2), var);
+ private static Function<Tuple<NodeId>, Quad> asQuad(NodeTable nodeTable, int tupleLen, boolean anyGraph) {
+ switch (tupleLen) {
+ case 3:
+ return (Tuple<NodeId> t) -> {
+ Node gx = Quad.defaultGraphIRI;
+ Node sx = toNode(t.get(0), nodeTable);
+ Node px = toNode(t.get(1), nodeTable);
+ Node ox = toNode(t.get(2), nodeTable);
+ return Quad.create(gx, sx, px, ox);
+ };
+ case 4:
+ return (Tuple<NodeId> t) -> {
+ Node gx = (anyGraph)? Quad.unionGraph : toNode(t.get(0), nodeTable);
+ Node sx = toNode(t.get(1), nodeTable);
+ Node px = toNode(t.get(2), nodeTable);
+ Node ox = toNode(t.get(3), nodeTable);
+ return Quad.create(gx, sx, px, ox);
+ };
default:
- throw new TDBException("Index is not recognized: "+idx);
+ throw new InternalErrorException("Tuple of unknow length");
Review comment:
It's "uck" that there is code copy on but unifying TDB1 into DBOE/TDB2 framework risks destablizing the code.
Copying the code, so it is just import changes, is slightly nudging them together. But in nearby classes the code is appreciably different.
"Sometime".
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #951: JENA-2065: RDF-star
Posted by GitBox <gi...@apache.org>.
afs commented on pull request #951:
URL: https://github.com/apache/jena/pull/951#issuecomment-793878984
Key points:
1. If the query does not have any RDF-star related syntax, processing is the normal path. Each of the three RDF-star point (in-memory, TDB1, TDB2) starts with an "if" that sees whether RDF-star processing is needed.
2. The test suite does run for general, TIM, TDB1, and TDB2 datasets if the redirection to RDF-star processing is off.
3. Some processing interfaces have changed: additional to ExprVisitor, removals from algebra visit and transform (due to taking out the old RDF* mode behaviour). This partially accounts for the large number of files.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #951: JENA-2065: RDF-star
Posted by GitBox <gi...@apache.org>.
afs commented on pull request #951:
URL: https://github.com/apache/jena/pull/951#issuecomment-795204630
Thank for the comments!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #951: JENA-2065: RDF-star
Posted by GitBox <gi...@apache.org>.
afs commented on pull request #951:
URL: https://github.com/apache/jena/pull/951#issuecomment-794449687
The spec (WIP - live editors draft) is https://w3c.github.io/rdf-star/cg-spec/editors_draft.html
I'm sure detail will change.
Hopefully not too much because there is a reasonable sized and growing test suite.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] kinow commented on pull request #951: JENA-2065: RDF-star
Posted by GitBox <gi...@apache.org>.
kinow commented on pull request #951:
URL: https://github.com/apache/jena/pull/951#issuecomment-794486518
> The spec (WIP - live editors draft) is https://w3c.github.io/rdf-star/cg-spec/editors_draft.html
>
> I'm sure detail will change.
> Hopefully not too much because there is a reasonable sized and growing test suite.
I'm watching the w3c repository and trying to at least read each PR/issue while lurking. But it's moving quite fast, have some issue/PR to read pretty much every day in my notifications.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org