You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@rya.apache.org by kchilton2 <gi...@git.apache.org> on 2018/02/15 18:26:42 UTC
[GitHub] incubator-rya pull request #275: RYA-466 Update the Rya Streams Client to st...
GitHub user kchilton2 opened a pull request:
https://github.com/apache/incubator-rya/pull/275
RYA-466 Update the Rya Streams Client to stream results to file.
## Description
Updated the Rya STreams Client to stream results to file.
### Tests
Added tests.
### Links
https://issues.apache.org/jira/browse/RYA-466
### Checklist
- [ ] Code Review
- [ ] Squash Commits
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kchilton2/incubator-rya RYA-466
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-rya/pull/275.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #275
----
commit 786989c14933fa4c4df143677406b75bd358738b
Author: kchilton2 <ke...@...>
Date: 2018-02-14T18:49:52Z
RYA-466 Update the Rya Streams Client to stream results to file.
----
---
[GitHub] incubator-rya pull request #275: RYA-466 Update the Rya Streams Client to st...
Posted by ejwhite922 <gi...@git.apache.org>.
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/275#discussion_r170329694
--- Diff: common/rya.api/src/test/java/org/apache/rya/api/utils/QueryInvestigatorTest.java ---
@@ -0,0 +1,123 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api.utils;
+
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
+
+import org.junit.Test;
+import org.openrdf.query.MalformedQueryException;
+
+/**
+ * Unit tests the methods of {@link }.
+ */
+public class QueryInvestigatorTest {
--- End diff --
Add tests for potential false positives mentioned above.
---
[GitHub] incubator-rya issue #275: RYA-466 Update the Rya Streams Client to stream re...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/incubator-rya/pull/275
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/694/<h2>Build result: FAILURE</span></h2>[...truncated 2.96 MB...][INFO] Apache Rya Spark Support ........................... SKIPPED[INFO] Apache Rya Web Projects ............................ SKIPPED[INFO] Apache Rya Web Implementation ...................... SKIPPED[INFO] ------------------------------------------------------------------------[INFO] BUILD FAILURE[INFO] ------------------------------------------------------------------------[INFO] Total time: 33:28 min[INFO] Finished at: 2018-02-16T20:21:47+00:00[INFO] Final Memory: 284M/2558M[INFO] ------------------------------------------------------------------------[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (check-style) on project rya.streams.client: You have 1 Checkstyle violation. -> [Help 1][ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run
Maven using the -X switch to enable full debug logging.[ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException[ERROR] [ERROR] After correcting the problems, you can resume the build with the command[ERROR] mvn <goals> -rf :rya.streams.clientchannel stoppedSetting status of 786989c14933fa4c4df143677406b75bd358738b to FAILURE with url https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/694/ and message: 'FAILURE 'Using context: Jenkins: clean package -Pgeoindexing
---
[GitHub] incubator-rya pull request #275: RYA-466 Update the Rya Streams Client to st...
Posted by ejwhite922 <gi...@git.apache.org>.
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/275#discussion_r171402268
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/utils/QueryInvestigator.java ---
@@ -0,0 +1,104 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api.utils;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.regex.Pattern;
+
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.parser.sparql.SPARQLParser;
+
+import edu.umd.cs.findbugs.annotations.DefaultAnnotation;
+import edu.umd.cs.findbugs.annotations.NonNull;
+
+/**
+ * A utility class that is used to glean insight into the structure of SPARQL queries.
+ */
+@DefaultAnnotation(NonNull.class)
+public class QueryInvestigator {
+
+ private static final SPARQLParser PARSER = new SPARQLParser();
+
+ private QueryInvestigator() { }
+
+ /**
+ * Determines whether a SPARQL command is a CONSTRUCT or not.
+ *
+ * @param sparql - The SPARQL to evaluate. (not null)
+ * @return {@code true} if the provided SPARQL is a CONSTRUCT query; otherwise {@code false}.
+ * @throws MalformedQueryException The SPARQL is neither a well formed query or update.
+ */
+ public static boolean isConstruct(final String sparql) throws MalformedQueryException {
+ requireNonNull(sparql);
+
+ try {
+ // Constructs are queries, so try to create a ParsedQuery.
+ PARSER.parseQuery(sparql, null);
+
+ // Check to see if the SPARQL looks like a CONSTRUCT query.
+ return Pattern.matches(".*?construct.*?where.*", sparql.toLowerCase());
--- End diff --
SELECT is supposed to create a ParsedTupleQuery based on looking at the [SPARQLParser.parseQuery()](https://github.com/ansell/openrdf-sesame/blob/fdf723d99ccdbdf0104e92c6a6433a8fbfe59750/core/queryparser/sparql/src/main/java/org/openrdf/query/parser/sparql/SPARQLParser.java) code but there's no documentation for any special cases. But it does look like DESCRIBE queries also use ParsedGraphQuery. Maybe also including a check similar to the INSERT example below would be best.
---
[GitHub] incubator-rya pull request #275: RYA-466 Update the Rya Streams Client to st...
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/275#discussion_r171065067
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/utils/QueryInvestigator.java ---
@@ -0,0 +1,104 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api.utils;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.regex.Pattern;
+
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.parser.sparql.SPARQLParser;
+
+import edu.umd.cs.findbugs.annotations.DefaultAnnotation;
+import edu.umd.cs.findbugs.annotations.NonNull;
+
+/**
+ * A utility class that is used to glean insight into the structure of SPARQL queries.
+ */
+@DefaultAnnotation(NonNull.class)
+public class QueryInvestigator {
+
+ private static final SPARQLParser PARSER = new SPARQLParser();
+
+ private QueryInvestigator() { }
+
+ /**
+ * Determines whether a SPARQL command is a CONSTRUCT or not.
+ *
+ * @param sparql - The SPARQL to evaluate. (not null)
+ * @return {@code true} if the provided SPARQL is a CONSTRUCT query; otherwise {@code false}.
+ * @throws MalformedQueryException The SPARQL is neither a well formed query or update.
+ */
+ public static boolean isConstruct(final String sparql) throws MalformedQueryException {
+ requireNonNull(sparql);
+
+ try {
+ // Constructs are queries, so try to create a ParsedQuery.
+ PARSER.parseQuery(sparql, null);
+
+ // Check to see if the SPARQL looks like a CONSTRUCT query.
+ return Pattern.matches(".*?construct.*?where.*", sparql.toLowerCase());
--- End diff --
Just write a better regex.
---
[GitHub] incubator-rya pull request #275: RYA-466 Update the Rya Streams Client to st...
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 closed the pull request at:
https://github.com/apache/incubator-rya/pull/275
---
[GitHub] incubator-rya issue #275: RYA-466 Update the Rya Streams Client to stream re...
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on the issue:
https://github.com/apache/incubator-rya/pull/275
asfbot build
---
[GitHub] incubator-rya pull request #275: RYA-466 Update the Rya Streams Client to st...
Posted by ejwhite922 <gi...@git.apache.org>.
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/275#discussion_r170329580
--- Diff: common/rya.api/src/test/java/org/apache/rya/api/utils/QueryInvestigatorTest.java ---
@@ -0,0 +1,123 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api.utils;
+
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
+
+import org.junit.Test;
+import org.openrdf.query.MalformedQueryException;
+
+/**
+ * Unit tests the methods of {@link }.
--- End diff --
fill in `{@link}`
---
[GitHub] incubator-rya issue #275: RYA-466 Update the Rya Streams Client to stream re...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/incubator-rya/pull/275
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/692/<h2>Build result: FAILURE</span></h2>[...truncated 3.01 MB...][INFO] Apache Rya Web Projects ............................ SKIPPED[INFO] Apache Rya Web Implementation ...................... SKIPPED[INFO] ------------------------------------------------------------------------[INFO] BUILD FAILURE[INFO] ------------------------------------------------------------------------[INFO] Total time: 36:42 min[INFO] Finished at: 2018-02-15T19:05:29+00:00[INFO] Final Memory: 304M/2426M[INFO] ------------------------------------------------------------------------Waiting for Jenkins to finish collecting data[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (check-style) on project rya.streams.client: You have 1 Checkstyle violation. -> [Help 1][ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X swi
tch to enable full debug logging.[ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException[ERROR] [ERROR] After correcting the problems, you can resume the build with the command[ERROR] mvn <goals> -rf :rya.streams.clientchannel stoppedSetting status of 786989c14933fa4c4df143677406b75bd358738b to FAILURE with url https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/692/ and message: 'FAILURE 'Using context: Jenkins: clean package -Pgeoindexing
---
[GitHub] incubator-rya pull request #275: RYA-466 Update the Rya Streams Client to st...
Posted by ejwhite922 <gi...@git.apache.org>.
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/275#discussion_r170336740
--- Diff: extras/rya.streams/client/src/main/java/org/apache/rya/streams/client/command/StreamResultsCommand.java ---
@@ -167,17 +182,38 @@ public void run() {
throw new ExecutionException("Could not parse the SPARQL for the query: " + sparql, e);
}
- // Iterate through the results and print them to the console until the program or the stream ends.
- try (final QueryResultStream<?> stream = getQueryResultStream.fromStart(queryId)) {
- while(!finished.get()) {
- for(final Object result : stream.poll(1000)) {
- System.out.println(result);
+ // Iterate through the results and print them to the configured output mechanism.
+ try (final QueryResultStream<?> resultsStream = getQueryResultStream.fromStart(queryId)) {
+ final TupleExpr tupleExpr = new SPARQLParser().parseQuery(sparql, null).getTupleExpr();
+ if(params.outputPath != null) {
+ final Path file = Paths.get(params.outputPath);
+ try(OutputStream out = Files.newOutputStream(file)) {
+ if(isStatementResults) {
+ final QueryResultStream<VisibilityStatement> stmtStream = (QueryResultStream<VisibilityStatement>) resultsStream;
+ QueryResultsOutputUtil.toNtriplesFile(out, stmtStream, finished);
+ } else {
+ final QueryResultStream<VisibilityBindingSet> bsStream = (QueryResultStream<VisibilityBindingSet>) resultsStream;
+ QueryResultsOutputUtil.toBindingSetJSONFile(out, tupleExpr, bsStream, finished);
+ }
}
+ } else {
+ streamToSystemOut(resultsStream, finished);
}
} catch (final Exception e) {
System.err.println("Error while reading the results from the stream.");
e.printStackTrace();
System.exit(1);
}
}
+
+ private void streamToSystemOut(final QueryResultStream<?> stream, final AtomicBoolean shutdownSignal) throws Exception {
--- End diff --
make static
---
[GitHub] incubator-rya issue #275: RYA-466 Update the Rya Streams Client to stream re...
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on the issue:
https://github.com/apache/incubator-rya/pull/275
asfbot build
---
[GitHub] incubator-rya issue #275: RYA-466 Update the Rya Streams Client to stream re...
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on the issue:
https://github.com/apache/incubator-rya/pull/275
I'm closing this because https://github.com/apache/incubator-rya/pull/279 takes it over.
---
[GitHub] incubator-rya pull request #275: RYA-466 Update the Rya Streams Client to st...
Posted by isper3at <gi...@git.apache.org>.
Github user isper3at commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/275#discussion_r170314969
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/utils/QueryInvestigator.java ---
@@ -0,0 +1,104 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api.utils;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.regex.Pattern;
+
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.parser.sparql.SPARQLParser;
+
+import edu.umd.cs.findbugs.annotations.DefaultAnnotation;
+import edu.umd.cs.findbugs.annotations.NonNull;
+
+/**
+ * A utility class that is used to glean insight into the structure of SPARQL queries.
+ */
+@DefaultAnnotation(NonNull.class)
+public class QueryInvestigator {
+
+ private static final SPARQLParser PARSER = new SPARQLParser();
+
+ private QueryInvestigator() { }
+
+ /**
+ * Determines whether a SPARQL command is a CONSTRUCT or not.
+ *
+ * @param sparql - The SPARQL to evaluate. (not null)
+ * @return {@code true} if the provided SPARQL is a CONSTRUCT query; otherwise {@code false}.
+ * @throws MalformedQueryException The SPARQL is neither a well formed query or update.
+ */
+ public static boolean isConstruct(final String sparql) throws MalformedQueryException {
+ requireNonNull(sparql);
+
+ try {
+ // Constructs are queries, so try to create a ParsedQuery.
+ PARSER.parseQuery(sparql, null);
+
+ // Check to see if the SPARQL looks like a CONSTRUCT query.
+ return Pattern.matches(".*?construct.*?where.*", sparql.toLowerCase());
+
+ } catch(final MalformedQueryException queryE) {
+ try {
+ // Maybe it's an update.
+ PARSER.parseUpdate(sparql, null);
+
+ // It was, so return false.
+ return false;
+
+ } catch(final MalformedQueryException updateE) {
+ // It's not. Actually malformed.
+ throw queryE;
+ }
+ }
+ }
+
+ /**
+ * Determines whether a SPARQL command is an INSERT with a WHERE clause or not.
+ *
+ * @param sparql - The SPARQL to evaluate. (not null)
+ * @return {@code true} if the provided SPARQL is an INSERT update; otherwise {@code false}.
+ * @throws MalformedQueryException The SPARQL is neither a well formed query or update.
+ */
+ public static boolean isInsertWhere(final String sparql) throws MalformedQueryException {
+ requireNonNull(sparql);
+
+ try {
+ // Inserts are updated, so try to create a ParsedUpdate.
+ PARSER.parseUpdate(sparql, null);
+
+ // Check to see if the SPARQL looks like an INSERT query.
+ return Pattern.matches(".*?insert.*?where.*", sparql.toLowerCase());
--- End diff --
odd thought, what if the query is something like INSERT { a ?where b }
---
[GitHub] incubator-rya issue #275: RYA-466 Update the Rya Streams Client to stream re...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/incubator-rya/pull/275
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/695/
---
[GitHub] incubator-rya issue #275: RYA-466 Update the Rya Streams Client to stream re...
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on the issue:
https://github.com/apache/incubator-rya/pull/275
asfbot build
---
[GitHub] incubator-rya pull request #275: RYA-466 Update the Rya Streams Client to st...
Posted by ejwhite922 <gi...@git.apache.org>.
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/275#discussion_r170329299
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/utils/QueryInvestigator.java ---
@@ -0,0 +1,104 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api.utils;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.regex.Pattern;
+
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.parser.sparql.SPARQLParser;
+
+import edu.umd.cs.findbugs.annotations.DefaultAnnotation;
+import edu.umd.cs.findbugs.annotations.NonNull;
+
+/**
+ * A utility class that is used to glean insight into the structure of SPARQL queries.
+ */
+@DefaultAnnotation(NonNull.class)
+public class QueryInvestigator {
+
+ private static final SPARQLParser PARSER = new SPARQLParser();
+
+ private QueryInvestigator() { }
+
+ /**
+ * Determines whether a SPARQL command is a CONSTRUCT or not.
+ *
+ * @param sparql - The SPARQL to evaluate. (not null)
+ * @return {@code true} if the provided SPARQL is a CONSTRUCT query; otherwise {@code false}.
+ * @throws MalformedQueryException The SPARQL is neither a well formed query or update.
+ */
+ public static boolean isConstruct(final String sparql) throws MalformedQueryException {
+ requireNonNull(sparql);
+
+ try {
+ // Constructs are queries, so try to create a ParsedQuery.
+ PARSER.parseQuery(sparql, null);
+
+ // Check to see if the SPARQL looks like a CONSTRUCT query.
+ return Pattern.matches(".*?construct.*?where.*", sparql.toLowerCase());
+
+ } catch(final MalformedQueryException queryE) {
+ try {
+ // Maybe it's an update.
+ PARSER.parseUpdate(sparql, null);
+
+ // It was, so return false.
+ return false;
+
+ } catch(final MalformedQueryException updateE) {
+ // It's not. Actually malformed.
+ throw queryE;
+ }
+ }
+ }
+
+ /**
+ * Determines whether a SPARQL command is an INSERT with a WHERE clause or not.
+ *
+ * @param sparql - The SPARQL to evaluate. (not null)
+ * @return {@code true} if the provided SPARQL is an INSERT update; otherwise {@code false}.
+ * @throws MalformedQueryException The SPARQL is neither a well formed query or update.
+ */
+ public static boolean isInsertWhere(final String sparql) throws MalformedQueryException {
+ requireNonNull(sparql);
+
+ try {
+ // Inserts are updated, so try to create a ParsedUpdate.
+ PARSER.parseUpdate(sparql, null);
+
+ // Check to see if the SPARQL looks like an INSERT query.
+ return Pattern.matches(".*?insert.*?where.*", sparql.toLowerCase());
--- End diff --
Again, this could lead to false positives. Such as:
```
DELETE
{
?bookInsert ?p ?o
}
WHERE
{
?bookInsert <urn:datePrinted> ?datePrinted .
FILTER ( ?datePrinted < "2018-01-01T00:00:00"^^xsd:dateTime )
?bookInsert ?p ?o
}
```
It should be either an INSERT or DELETE by the time it reaches the pattern matcher. However, INSERT and DELETE are both considered `ParsedUpdate`s without any extra subclasses to distinguish them so it needs to be handled differently from the CONSTRUCT query parsing above.
Maybe try:
```
PARSER.parseUpdate(sparql, null);
final String strippedOperation = QueryParserUtil.removeSPARQLQueryProlog(sparql.toLowerCase());
return strippedOperation.startsWith("insert");
```
---
[GitHub] incubator-rya pull request #275: RYA-466 Update the Rya Streams Client to st...
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/275#discussion_r171064989
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/utils/QueryInvestigator.java ---
@@ -0,0 +1,104 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api.utils;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.regex.Pattern;
+
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.parser.sparql.SPARQLParser;
+
+import edu.umd.cs.findbugs.annotations.DefaultAnnotation;
+import edu.umd.cs.findbugs.annotations.NonNull;
+
+/**
+ * A utility class that is used to glean insight into the structure of SPARQL queries.
+ */
+@DefaultAnnotation(NonNull.class)
+public class QueryInvestigator {
+
+ private static final SPARQLParser PARSER = new SPARQLParser();
+
+ private QueryInvestigator() { }
+
+ /**
+ * Determines whether a SPARQL command is a CONSTRUCT or not.
+ *
+ * @param sparql - The SPARQL to evaluate. (not null)
+ * @return {@code true} if the provided SPARQL is a CONSTRUCT query; otherwise {@code false}.
+ * @throws MalformedQueryException The SPARQL is neither a well formed query or update.
+ */
+ public static boolean isConstruct(final String sparql) throws MalformedQueryException {
+ requireNonNull(sparql);
+
+ try {
+ // Constructs are queries, so try to create a ParsedQuery.
+ PARSER.parseQuery(sparql, null);
+
+ // Check to see if the SPARQL looks like a CONSTRUCT query.
+ return Pattern.matches(".*?construct.*?where.*", sparql.toLowerCase());
--- End diff --
Any SELECT could also create a ParsedGraphQuery, so that will not work.
---
[GitHub] incubator-rya issue #275: RYA-466 Update the Rya Streams Client to stream re...
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on the issue:
https://github.com/apache/incubator-rya/pull/275
asfbot build
---
[GitHub] incubator-rya pull request #275: RYA-466 Update the Rya Streams Client to st...
Posted by ejwhite922 <gi...@git.apache.org>.
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/275#discussion_r170336478
--- Diff: extras/rya.streams/client/src/main/java/org/apache/rya/streams/client/command/StreamResultsCommand.java ---
@@ -167,17 +182,38 @@ public void run() {
throw new ExecutionException("Could not parse the SPARQL for the query: " + sparql, e);
}
- // Iterate through the results and print them to the console until the program or the stream ends.
- try (final QueryResultStream<?> stream = getQueryResultStream.fromStart(queryId)) {
- while(!finished.get()) {
- for(final Object result : stream.poll(1000)) {
- System.out.println(result);
+ // Iterate through the results and print them to the configured output mechanism.
+ try (final QueryResultStream<?> resultsStream = getQueryResultStream.fromStart(queryId)) {
+ final TupleExpr tupleExpr = new SPARQLParser().parseQuery(sparql, null).getTupleExpr();
+ if(params.outputPath != null) {
+ final Path file = Paths.get(params.outputPath);
+ try(OutputStream out = Files.newOutputStream(file)) {
--- End diff --
try(final OutputStream out ...
---
[GitHub] incubator-rya pull request #275: RYA-466 Update the Rya Streams Client to st...
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/275#discussion_r171032022
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/utils/QueryInvestigator.java ---
@@ -0,0 +1,104 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api.utils;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.regex.Pattern;
+
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.parser.sparql.SPARQLParser;
+
+import edu.umd.cs.findbugs.annotations.DefaultAnnotation;
+import edu.umd.cs.findbugs.annotations.NonNull;
+
+/**
+ * A utility class that is used to glean insight into the structure of SPARQL queries.
+ */
+@DefaultAnnotation(NonNull.class)
+public class QueryInvestigator {
+
+ private static final SPARQLParser PARSER = new SPARQLParser();
+
+ private QueryInvestigator() { }
+
+ /**
+ * Determines whether a SPARQL command is a CONSTRUCT or not.
+ *
+ * @param sparql - The SPARQL to evaluate. (not null)
+ * @return {@code true} if the provided SPARQL is a CONSTRUCT query; otherwise {@code false}.
+ * @throws MalformedQueryException The SPARQL is neither a well formed query or update.
+ */
+ public static boolean isConstruct(final String sparql) throws MalformedQueryException {
+ requireNonNull(sparql);
+
+ try {
+ // Constructs are queries, so try to create a ParsedQuery.
+ PARSER.parseQuery(sparql, null);
+
+ // Check to see if the SPARQL looks like a CONSTRUCT query.
+ return Pattern.matches(".*?construct.*?where.*", sparql.toLowerCase());
+
+ } catch(final MalformedQueryException queryE) {
+ try {
+ // Maybe it's an update.
+ PARSER.parseUpdate(sparql, null);
+
+ // It was, so return false.
+ return false;
+
+ } catch(final MalformedQueryException updateE) {
+ // It's not. Actually malformed.
+ throw queryE;
+ }
+ }
+ }
+
+ /**
+ * Determines whether a SPARQL command is an INSERT with a WHERE clause or not.
+ *
+ * @param sparql - The SPARQL to evaluate. (not null)
+ * @return {@code true} if the provided SPARQL is an INSERT update; otherwise {@code false}.
+ * @throws MalformedQueryException The SPARQL is neither a well formed query or update.
+ */
+ public static boolean isInsertWhere(final String sparql) throws MalformedQueryException {
+ requireNonNull(sparql);
+
+ try {
+ // Inserts are updated, so try to create a ParsedUpdate.
+ PARSER.parseUpdate(sparql, null);
+
+ // Check to see if the SPARQL looks like an INSERT query.
+ return Pattern.matches(".*?insert.*?where.*", sparql.toLowerCase());
--- End diff --
.*? is part of the regex. It will not match on ?where. But yea, it will def hit the 'where' part.
---
[GitHub] incubator-rya pull request #275: RYA-466 Update the Rya Streams Client to st...
Posted by ejwhite922 <gi...@git.apache.org>.
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/275#discussion_r170322179
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/utils/QueryInvestigator.java ---
@@ -0,0 +1,104 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api.utils;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.regex.Pattern;
+
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.parser.sparql.SPARQLParser;
+
+import edu.umd.cs.findbugs.annotations.DefaultAnnotation;
+import edu.umd.cs.findbugs.annotations.NonNull;
+
+/**
+ * A utility class that is used to glean insight into the structure of SPARQL queries.
+ */
+@DefaultAnnotation(NonNull.class)
+public class QueryInvestigator {
+
+ private static final SPARQLParser PARSER = new SPARQLParser();
+
+ private QueryInvestigator() { }
+
+ /**
+ * Determines whether a SPARQL command is a CONSTRUCT or not.
+ *
+ * @param sparql - The SPARQL to evaluate. (not null)
+ * @return {@code true} if the provided SPARQL is a CONSTRUCT query; otherwise {@code false}.
+ * @throws MalformedQueryException The SPARQL is neither a well formed query or update.
+ */
+ public static boolean isConstruct(final String sparql) throws MalformedQueryException {
+ requireNonNull(sparql);
+
+ try {
+ // Constructs are queries, so try to create a ParsedQuery.
+ PARSER.parseQuery(sparql, null);
+
+ // Check to see if the SPARQL looks like a CONSTRUCT query.
+ return Pattern.matches(".*?construct.*?where.*", sparql.toLowerCase());
--- End diff --
I don't think this will work in all cases. It looks like the following query would incorrectly be considered a CONSTRUCT query:
```
SELECT ?constructionCompany
WHERE {
?constructionCompany <urn:built> <urn:skyscraper> .
}
```
Instead, try checking if the result of `PARSER.parseQuery()` is a `ParsedGraphQuery`:
```
final ParsedQuery parsedQuery = PARSER.parseQuery(sparql, null);
return parsedQuery instanceof ParsedGraphQuery;
```
---